Abstract

An iterative method LSMR is presented for solving linear systems $Ax=b$ and least-squares problems $\min \|Ax-b\|_2$, with A being sparse or a fast linear operator. LSMR is based on the Golub–Kahan bidiagonalization process. It is analytically equivalent to the MINRES method applied to the normal equation $A^T\! Ax = A^T\! b$, so that the quantities $\|A^T\! r_k\|$ are monotonically decreasing (where $r_k = b - Ax_k$ is the residual for the current iterate $x_k$). We observe in practice that $\|r_k\|$ also decreases monotonically, so that compared to LSQR (for which only $\|r_k\|$ is monotonic) it is safer to terminate LSMR early. We also report some experiments with reorthogonalization.

MSC codes

  1. 15A06
  2. 65F10
  3. 65F20
  4. 65F22
  5. 65F25
  6. 65F35
  7. 65F50
  8. 93E24

Keywords

  1. least-squares problem
  2. sparse matrix
  3. LSQR
  4. MINRES
  5. Krylov subspace method
  6. Golub–Kahan process
  7. conjugate-gradient method
  8. minimum-residual method
  9. iterative method

Get full access to this article

View all available purchase options and get full access to this article.

References

1.
M. Arioli and S. Gratton, Least-Squares Problems, Normal Equations, and Stopping Criteria for the Conjugate Gradient Method, Technical report RAL-TR-2008-008, Rutherford Appleton Laboratory, Oxfordshire, UK, 2008.
2.
S. J. Benbow, Solving generalized least-squares problems with LSQR, SIAM J. Matrix Anal. Appl., 21 (1999), pp. 166–177.
3.
X.-W. Chang, C. C. Paige, and D. Titley-Peloquin, Stopping criteria for the iterative solution of linear least squares problems, SIAM J. Matrix Anal. Appl., 31 (2009), pp. 831–852.
4.
S.-C. Choi, C. C. Paige, and M. A. Saunders, MINRES-QLP: A Krylov subspace method for indefinite or singular symmetric systems, SIAM J. Sci. Comput., to appear.
5.
T. A. Davis, University of Florida Sparse Matrix Collection, http://www.cise.ufl.edu/research/sparse/matrices.
6.
G. H. Golub and W. Kahan, Calculating the singular values and pseudo-inverse of a matrix, J. Soc. Indust. Appl. Math. Ser. B Numer. Anal., 2 (1965), pp. 205–224.
7.
S. Gratton, P. Jiránek, and D. Titley-Peloquin, On the Accuracy of the Karlson-Waldén Estimate of the Backward Error for Linear Least Squares Problems, CERFACS Technical report TR/PA/11/19, CERFACS, Toulouse, France, 2011.
8.
J. F. Grcar, M. A. Saunders, and Z. Su, Estimates of Optimal Backward Perturbations for Linear Least Squares Problems, Report SOL 2007-1, Department of Management Science and Engineering, Stanford University, Stanford, CA, 2007.
9.
K. Hayami, J.-F. Yin, and T. Ito, GMRES methods for least squares problems, SIAM J. Matrix Anal. Appl., 31 (2010), pp. 2400–2430.
10.
N. J. Higham, Accuracy and Stability of Numerical Algorithms, 2nd ed., SIAM, Philadelphia, 2002.
11.
LSMR software for linear systems and least squares, http://www.stanford.edu/group/SOL/software.html.
12.
D. G. Luenberger, The conjugate residual method for constrained minimization problems, SIAM J. Numer. Anal., 7 (1970), pp. 390–398.
13.
P. Jiránek and D. Titley-Peloquin, Estimating the backward error in LSQR, SIAM J. Matrix Anal. Appl., 31 (2010), pp. 2055–2074.
14.
C. C. Paige, M. Rozložník, and Z. Strakoš, Modified Gram–Schmidt (MGS), least squares, and backward stability of MGS-GMRES, SIAM J. Matrix Anal. Appl., 28 (2006), pp. 264–284.
15.
C. C. Paige and M. A. Saunders, Solution of sparse indefinite systems of linear equations, SIAM J. Numer. Anal., 12 (1975), pp. 617–629.
16.
C. C. Paige and M. A. Saunders, LSQR: An algorithm for sparse linear equations and sparse least squares, ACM Trans. Math. Software, 8 (1982), pp. 43–71.
17.
C. C. Paige and M. A. Saunders, Algorithm 583; LSQR: Sparse linear equations and least-squares problems, ACM Trans. Math. Software, 8 (1982), pp. 195–209.
18.
V. Pereyra, private communication, 2010.
19.
PROPACK software for SVD of sparse matrices, http://soi.stanford.edu/~rmunk/PROPACK/.
20.
Y. Saad and M. H. Schultz, GMRES: A generalized minimum residual algorithm for solving nonsymmetric linear systems, SIAM J. Sci. Statist. Comput., 7 (1986), pp. 856–869.
21.
G. W. Stewart, An inverse perturbation theorem for the linear least squares problem, SIGNUM Newsletter, 10 (1975), pp. 39–40.
22.
G. W. Stewart, Research, development and LINPACK, in Mathematical Software III, J. R. Rice, ed., Academic Press, New York, 1977, pp. 1–14.
23.
G. W. Stewart, The QLP approximation to the singular value decomposition, SIAM J. Sci. Comput., 20 (1999), pp. 1336–1348.
24.
Z. Su, Computational Methods for Least Squares Problems and Clinical Trials, Ph.D. thesis, SCCM, Stanford University, Stanford, CA, 2005.
25.
D. Titley-Peloquin, Backward Perturbation Analysis of Least Squares Problems, Ph.D. thesis, School of Computer Science, McGill University, Montreal, QC, Canada, 2010.
26.
B. Waldén, R. Karlson, and J.-G. Sun, Optimal backward perturbation bounds for the linear least squares problem, Numer. Linear Algebra Appl., 2 (1995), pp. 271–286.

Information & Authors

Information

Published In

cover image SIAM Journal on Scientific Computing
SIAM Journal on Scientific Computing
Pages: 2950 - 2971
ISSN (online): 1095-7197

History

Submitted: 1 June 2010
Accepted: 6 June 2011
Published online: 27 October 2011

MSC codes

  1. 15A06
  2. 65F10
  3. 65F20
  4. 65F22
  5. 65F25
  6. 65F35
  7. 65F50
  8. 93E24

Keywords

  1. least-squares problem
  2. sparse matrix
  3. LSQR
  4. MINRES
  5. Krylov subspace method
  6. Golub–Kahan process
  7. conjugate-gradient method
  8. minimum-residual method
  9. iterative method

Authors

Affiliations

Metrics & Citations

Metrics

Citations

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

Cited By

View Options

View options

PDF

View PDF

Media

Figures

Other

Tables

Share

Share

Copy the content Link

Share with email

Email a colleague

Share on social media