Abstract

Regularization algorithms are often used to produce reasonable solutions to ill-posed problems. The L-curve is a plot—for all valid regularization parameters—of the size of the regularized solution versus the size of the corresponding residual. Two main results are established. First a unifying characterization of various regularization methods is given and it is shown that the measurement of “size” is dependent on the particular regularization method chosen. For example, the 2-norm is appropriate for Tikhonov regularization, but a 1-norm in the coordinate system of the singular value decomposition (SVD) is relevant to truncated SVD regularization. Second, a new method is proposed for choosing the regularization parameter based on the L-curve, and it is shown how this method can be implemented efficiently. The method is compared to generalized cross validation and this new method is shown to be more robust in the presence of correlated errors.

MSC codes

  1. 65R30
  2. 65F20

Keywords

  1. ill-posed problems
  2. regularization
  3. L-curve
  4. parameter choice
  5. generalized cross validation
  6. discrepancy principle

Get full access to this article

View all available purchase options and get full access to this article.

References

1.
M. Bertero, C. De Mol, E. R. Pike, Heinz W. Engl, C. W. Groetsch, Applied inverse problems in opticsInverse and Ill-Posed Problems, Academic Press, New York, 1987
2.
Lars Eldén, Algorithms for the regularization of ill-conditioned least squares problems, Nordisk Tidskr. Informationsbehandling (BIT), 17 (1977), 134–145
3.
G. Farin, Curves and Surfaces for Computer Aided Geometric Design, Academic Press, New York, 1988
4.
Gene H. Golub, Michael Heath, Grace Wahba, Generalized cross-validation as a method for choosing a good ridge parameter, Technometrics, 21 (1979), 215–223
5.
C. W. Groetsch, The theory of Tikhonov regularization for Fredholm equations of the first kind, Research Notes in Mathematics, Vol. 105, Pitman (Advanced Publishing Program), Boston, MA, 1984iv+104
6.
C. W. Groetsch, C. R. Vogel, Asymptotic theory of filtering for linear operator equations with discrete noisy data, Math. Comp., 49 (1987), 499–506
7.
Per Christian Hansen, The truncated SVD as a method for regularization, BIT, 27 (1987), 534–553
8.
P. C. Hansen, Regularization tools, a Matlab package for analysis of discrete regularization problems, Tech. Report, Danish Computing Center for Research and Education, Lyngby, Denmark, 1991
9.
Per Christian Hansen, Analysis of discrete ill-posed problems by means of the L-curve, SIAM Rev., 34 (1992), 561–580
10.
Charles L. Lawson, Richard J. Hanson, Solving least squares problems, Prentice-Hall Inc., Englewood Cliffs, N.J., 1974xii+340
11.
F. Natterer, The mathematics of computerized tomography, B. G. Teubner, Stuttgart, 1986x+222
12.
Christopher C. Paige, Michael A. Saunders, LSQR: an algorithm for sparse linear equations and sparse least squares, ACM Trans. Math. Software, 8 (1982), 43–71
13.
John A. Scales, Adam Gersztenkorn, Robust methods in inverse theory, Inverse Problems, 4 (1988), 1071–1091
14.
C. B. Shaw, Jr., Improvement of the resolution of an instrument by numerical solution of an integral equation, J. Math. Anal. Appl., 37 (1972), 83–112
15.
J. Skilling, S. F. Gull, C. R. Smith, W. T. Grandy, Jr., Algorithms and applicationsMaximum-Entropy and Bayesian Methods in Inverse Problems, D. Reidel Pub. Co., Boston, 1985, 83–132
16.
Andrey N. Tikhonov, Vasiliy Y. Arsenin, Solutions of ill-posed problems, V. H. Winston & Sons, Washington, D.C.: John Wiley & Sons, New York, 1977xiii+258
17.
J. M. Varah, Pitfalls in the numerical solution of linear ill-posed problems, SIAM J. Sci. Statist. Comput., 4 (1983), 164–176
18.
Grace Wahba, Spline models for observational data, CBMS-NSF Regional Conference Series in Applied Mathematics, Vol. 59, Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 1990xii+169

Information & Authors

Information

Published In

cover image SIAM Journal on Scientific Computing
SIAM Journal on Scientific Computing
Pages: 1487 - 1503
ISSN (online): 1095-7197

History

Submitted: 23 October 1991
Accepted: 31 December 1992
Published online: 13 July 2006

MSC codes

  1. 65R30
  2. 65F20

Keywords

  1. ill-posed problems
  2. regularization
  3. L-curve
  4. parameter choice
  5. generalized cross validation
  6. discrepancy principle

Authors

Affiliations

Dianne Prost O’Leary

Metrics & Citations

Metrics

Citations

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

Cited By