Abstract

In this contribution, we study the numerical behavior of the generalized minimal residual (GMRES) method for solving singular linear systems. It is known that GMRES determines a least squares solution without breakdown if the coefficient matrix is range-symmetric (EP) or if its range and nullspace are disjoint (GP) and the system is consistent. We show that the accuracy of GMRES iterates may deteriorate in practice due to three distinct factors: (i) the inconsistency of the linear system, (ii) the distance of the initial residual to the nullspace of the coefficient matrix, and (iii) the extremal principal angles between the ranges of the coefficient matrix and its transpose. These factors lead to poor conditioning of the extended Hessenberg matrix in the Arnoldi decomposition and affect the accuracy of the computed least squares solution. We also compare GMRES with the range restricted GMRES method. Numerical experiments show typical behaviors of GMRES for small problems with EP and GP matrices.

Keywords

  1. GMRES method
  2. singular linear systems
  3. least squares problems
  4. group inverse
  5. EP matrix
  6. GP matrix

MSC codes

  1. 65F10
  2. 15A12
  3. 15A09

Get full access to this article

View all available purchase options and get full access to this article.

References

1.
O. Awono and J. Tagoudjeu, A preconditioned minimal residual solver for a class of linear operator equations, Comput. Methods Appl. Math., 10 (2010), pp. 119--136, https://doi.org/10.2478/cmam-2010-0007.
2.
O. M. Baksalary and G. Trenkler, Characterizations of EP, normal, and Hermitian matrices, Linear Multilinear Algebra, 56 (2008), pp. 299--304, https://doi.org/10.1080/03081080600872616.
3.
O. M. Baksalary and G. Trenkler, On disjoint range matrices, Linear Algebra Appl., 435 (2011), pp. 1222--1240, https://doi.org/10.1016/j.laa.2011.03.005.
4.
A. Ben-Israel and T. N. E. Greville, Generalized Inverses: Theory and Applications, 2nd ed., Springer-Verlag, New York, 2003, https://doi.org/10.1007/b97366.
5.
\AA. Björck, Numerical Methods for Least Squares Problems, SIAM, Philadelphia, PA, 1996.
6.
P. N. Brown and H. F. Walker, GMRES on (nearly) singular systems, SIAM J. Matrix Anal. Appl., 18 (1997), pp. 37--51, https://doi.org/10.1137/S0895479894262339.
7.
D. Calvetti, B. Lewis, and L. Reichel, GMRES-type methods for inconsistent systems, Linear Algebra Appl., 316 (2000), pp. 157--169, https://doi.org/10.1016/S0024-3795(00)00064-1.
8.
D. Calvetti, B. Lewis, and L. Reichel, GMRES, L-curves, and discrete ill-posed problems, BIT, 42 (2002), pp. 44--65, https://doi.org/10.1023/a:1021918118380.
9.
D. Calvetti, B. Lewis, and L. Reichel, On the regularizing properties of the GMRES method, Numer. Math., 91 (2002), pp. 605--625.
10.
S. L. Campbell and C. D. Meyer, Generalized Inverses of Linear Transformations, SIAM, Philadelphia, PA, 2009, https://doi.org/10.1137/1.9780898719048.
11.
Z.-H. Cao and M. Wang, A note on Krylov subspace methods for singular systems, Linear Algebra Appl., 350 (2002), pp. 285--288, https://doi.org/10.1016/S0024-3795(02)00310-5.
12.
F. Chatelin, Eigenvalues of Matrices, revised ed., SIAM, Philadelphia, PA, 2012, https://doi.org/10.1137/1.9781611972467.
13.
X. Du and D. B. Szyld, Inexact GMRES for singular linear systems, BIT, 48 (2008), pp. 511--531, https://doi.org/10.1007/s10543-008-0171-2.
14.
L. Eldén and V. Simoncini, Solving ill-posed linear systems with GMRES and a singular preconditioner, SIAM J. Matrix Anal. Appl., 33 (2012), pp. 1369--1394, https://doi.org/10.1137/110832793.
15.
M. G. Gasparo, A. Papini, and A. Pasquali, Some properties of GMRES in Hilbert spaces, Numer. Funct. Anal. Optim., 29 (2008), pp. 1276--1285, https://doi.org/10.1080/01630560802580786.
16.
I. C. F. Ipsen and C. D. Meyer, The idea behind Krylov methods, Amer. Math. Monthly, 105 (1998), pp. 889--899, https://doi.org/10.2307/2589281.
17.
Z. Liao, K. Hayami, and J.-F. Yin, A stabilized GMRES method for solving inconsistent underdetermined least squares problems, in 22nd Meeting of the Japan SIAM Special Interest on Algorithm for Matrix/Eigenvalue Problems and Their Applications, Tokyo, Japan, Japan Society for Industrial and Applied Mathematics, 2016.
18.
K. Morikuni and K. Hayami, Convergence of inner-iteration GMRES methods for rank-deficient least squares problems, SIAM J. Matrix Anal. Appl., 36 (2015), pp. 225--250, https://doi.org/10.1137/130946009.
19.
C. C. Paige and M. A. Saunders, Solution of sparse indefinite systems of linear equations, SIAM J. Numer. Anal., 12 (1975), pp. 617--629, https://doi.org/10.1137/0712047.
20.
L. Reichel and Q. Ye, Breakdown-free GMRES for singular systems, SIAM J. Matrix Anal. Appl., 26 (2005), pp. 1001--1021, https://doi.org/10.1137/S0895479803437803.
21.
P. Robert, On the group-inverse of a linear transformation, J. Math. Anal. Appl., 22 (1968), pp. 658--669, https://doi.org/10.1016/0022-247x(68)90204-7.
22.
Y. Saad and M. H. Schultz, GMRES: A generalized minimal residual algorithm for solving nonsymmetric linear systems, SIAM J. Sci. Stat. Comput., 7 (1986), pp. 856--869, https://doi.org/10.1137/0907058.
23.
O. Schneider, Krylov Subspace Methods and Their Generalizations for Solving Singular Linear Operator Equations with Applications to Continuous Time Markov Chains, Ph.D. thesis, Technischen Universität Bergakademie Freiberg, 2005, http://nbn-resolving.de/urn:nbn:de:bsz:105-1148840.
24.
A. Sidi, A unified approach to Krylov subspace methods for the Drazin-inverse solution of singular nonsymmetric linear systems, Linear Algebra Appl., 298 (1999), pp. 99--113, https://doi.org/10.1016/S0024-3795(99)00153-6.
25.
A. Sidi, DGMRES: A GMRES-type algorithm for Drazin-inverse solution of singular nonsymmetric linear systems, Linear Algebra Appl., 335 (2001), pp. 189--204, https://doi.org/10.1016/S0024-3795(01)00289-0.
26.
L. Smoch, Spectral behaviour of GMRES applied to singular systems, Adv. Comput. Math., 27 (2007), pp. 151--166, https://doi.org/10.1007/s10444-007-9029-4.
27.
Z. Strakoš, On the real convergence rate of the conjugate gradient method, Linear Algebra Appl., 154--156 (1991), pp. 535--549, https://doi.org/10.1016/0024-3795(91)90393-b.
28.
H. F. Walker, Implementation of the GMRES method using Householder transformations, SIAM J. Sci. Stat. Comput., 9 (1988), pp. 152--163, https://doi.org/10.1137/0909010.
29.
H. F. Walker and L. Zhou, A simpler GMRES, Numer. Linear Algebra Appl., 1 (1994), pp. 571--581, https://doi.org/10.1002/nla.1680010605.
30.
Y.-M. Wei, Perturbation analysis of singular linear systems with index one, Int. J. Comput. Math., 74 (2000), pp. 483--491, https://doi.org/10.1080/00207160008804956.
31.
Y.-M. Wei and H. Wu, Convergence properties of Krylov subspace methods for singular linear systems with arbitrary index, J. Comput. Appl. Math., 114 (2000), pp. 305--318, https://doi.org/10.1016/S0377-0427(99)90237-6.
32.
N.-M. Zhang, A note on preconditioned GMRES for solving singular linear systems, BIT, 50 (2010), pp. 207--220, https://doi.org/10.1007/s10543-009-0247-7.
33.
N.-M. Zhang and Y.-M. Wei, Solving EP singular linear systems, Int. J. Comput. Math., 81 (2004), pp. 1395--1405, https://doi.org/10.1080/00207160412331284132.

Information & Authors

Information

Published In

cover image SIAM Journal on Matrix Analysis and Applications
SIAM Journal on Matrix Analysis and Applications
Pages: 1033 - 1048
ISSN (online): 1095-7162

History

Submitted: 8 May 2017
Accepted: 29 March 2018
Published online: 5 June 2018

Keywords

  1. GMRES method
  2. singular linear systems
  3. least squares problems
  4. group inverse
  5. EP matrix
  6. GP matrix

MSC codes

  1. 65F10
  2. 15A12
  3. 15A09

Authors

Affiliations

Miroslav Rozložník

Funding Information

Czech Science Foundation : GA17-12925S
Japan Society for the Promotion of Science https://doi.org/10.13039/501100001691 : 16K17639

Metrics & Citations

Metrics

Citations

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

Cited By

View Options

View options

PDF

View PDF

Media

Figures

Other

Tables

Share

Share

Copy the content Link

Share with email

Email a colleague

Share on social media

The SIAM Publications Library now uses SIAM Single Sign-On for individuals. If you do not have existing SIAM credentials, create your SIAM account https://my.siam.org.