We consider the class of iterative shrinkage-thresholding algorithms (ISTA) for solving linear inverse problems arising in signal/image processing. This class of methods, which can be viewed as an extension of the classical gradient algorithm, is attractive due to its simplicity and thus is adequate for solving large-scale problems even with dense matrix data. However, such methods are also known to converge quite slowly. In this paper we present a new fast iterative shrinkage-thresholding algorithm (FISTA) which preserves the computational simplicity of ISTA but with a global rate of convergence which is proven to be significantly better, both theoretically and practically. Initial promising numerical results for wavelet-based image deblurring demonstrate the capabilities of FISTA which is shown to be faster than ISTA by several orders of magnitude.

MSC codes

  1. 90C25
  2. 90C06
  3. 65F22


  1. iterative shrinkage-thresholding algorithm
  2. deconvolution
  3. linear inverse problem
  4. least squares and $l_1$ regularization problems
  5. optimal gradient method
  6. global rate of convergence
  7. two-step iterative algorithms
  8. image deblurring

Get full access to this article

View all available purchase options and get full access to this article.


A. Ben-Tal and A. Nemirovski, Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications, MPS/SIAM Ser. Optim., SIAM, Philadelphia, 2001.
D. P. Bertsekas, Nonlinear Programming, 2nd ed., Athena Scientific, Belmont, MA, 1999.
J. Bioucas-Dias and M. Figueiredo, A new TwIST: Two-step iterative shrinkage/thresholding algorithms for image restoration, IEEE Trans. Image Process., 16 (2007), pp. 2992–3004.
A. Björck, Numerical Methods for Least Squares Problems, SIAM, Philadelphia, 1996.
K. Bredies and D. Lorenz, Iterative Soft-Thresholding Converges Linearly, Technical report, 2008; available online at http://arxiv.org/abs/0709.1598v3.
R. J. Bruck, On the weak convergence of an ergodic iteration for the solution of variational inequalities for monotone operators in Hilbert space, J. Math. Anal. Appl., 61 (1977), pp. 159–164.
A. Chambolle, R. A. DeVore, N. Y. Lee, and B. J. Lucier, Nonlinear wavelet image processing: Variational problems, compression, and noise removal through wavelet shrinkage, IEEE Trans. Image Process., 7 (1998), pp. 319–335.
S. S. Chen, D. L. Donoho, and M. A. Saunders, Atomic decomposition by basis pursuit, SIAM J. Sci. Comput., 20 (1998), pp. 33–61.
P. L. Combettes and V. R. Wajs, Signal recovery by proximal forward-backward splitting, Multiscale Model. Simul., 4 (2005), pp. 1168–1200.
I. Daubechies, M. Defrise, and C. D. Mol, An iterative thresholding algorithm for linear inverse problems with a sparsity constraint, Comm. Pure Appl. Math., 57 (2004), pp. 1413–1457.
D. L. Donoho and I. M. Johnstone, Adapting to unknown smoothness via wavelet shrinkage, J. Amer. Statist. Assoc., 90 (1995), pp. 1200–1224.
M. Elad, B. Matalon, and M. Zibulevsky, Subspace optimization methods for linear least squares with non-quadratic regularization, Appl. Comput. Harmon. Anal., 23 (2007), pp. 346–367.
H. W. Engl, M. Hanke, and A. Neubauer, Regularization of Inverse Problems, Math. Appl. 375, Kluwer Academic Publishers Group, Dordrecht, The Netherlands, 1996.
F. Facchinei and J.-S. Pang, Finite-Dimensional Variational Inequalities and Complementarity Problems, Vol. II, Springer Ser. Oper. Res., Springer-Verlag, New York, 2003.
M. A. T. Figueiredo and R. D. Nowak, An EM algorithm for wavelet-based image restoration, IEEE Trans. Image Process., 12 (2003), pp. 906–916.
M. A. T. Figueiredo, R. D. Nowak, and S. J. Wright, Gradient projection for sparse reconstruction: Application to compressed sensing and other inverse problems, IEEE J. Sel. Top. Signal Process., 1 (2007), pp. 586–597.
G. H. Golub, P. C. Hansen, and D. P. O'Leary, Tikhonov regularization and total least squares, SIAM J. Matrix Anal. Appl., 21 (1999), pp. 185–194.
E. Hale, W. Yin, and Y. Zhang, A Fixed-Point Continuation Method for $l_1$-Regularized Minimization with Applications to Compressed Sensing, CAAM Technical report TR07-07, Rice University, Houston, TX, 2007.
P. C. Hansen and D. P. O'Leary, The use of the L-curve in the regularization of discrete ill-posed problems, SIAM J. Sci. Comput., 14 (1993), pp. 1487–1503.
P. C. Hansen, Regularization tools: A MATLAB package for analysis and solution of discrete ill-posed problems, Numer. Algorithms, 6 (1994), pp. 1–35.
P. C. Hansen, Rank-Deficient and Discrete Ill-Posed Problems: Numerical Aspects of Linear Inversion, SIAM, Philadelphia, 1997.
P. C. Hansen, J. G. Nagy, and D. P. O'Leary, Deblurring Images: Matrices, Spectra, and Filtering, Fundam. Algorithms 3, SIAM, Philadelphia, 2006.
E. S. Levitin and B. T. Polyak, Constrained minimization methods, Comput. Math. Math. Phys., 6 (1966), pp. 787–823.
B. Martinet, Régularisation d'inéquations variationnelles par approximations successives, Rev. Française Informat. Recherche Opérationnelle, 4 (1970), pp. 154–158.
P. Moulin and J. Liu, Analysis of multiresolution image denoising schemes using generalized Gaussian and complexity priors, IEEE Trans. Inform. Theory, 45 (1999), pp. 909–919.
A. S. Nemirovsky and D. B. Yudin, Problem Complexity and Method Efficiency in Optimization, Wiley-Interscience Series in Discrete Mathematics, John Wiley & Sons, New York, 1983.
Y. E. Nesterov, A method for solving the convex programming problem with convergence rate $O(1/k\sp{2})$, Dokl. Akad. Nauk SSSR, 269 (1983), pp. 543–547 (in Russian).
Y. E. Nesterov, Gradient Methods for Minimizing Composite Objective Function, CORE report, 2007; available at http://www.ecore.be/DPs/dp1191313936.pdf.
J. M. Ortega and W. C. Rheinboldt, Iterative Solution of Nonlinear Equations in Several Variables, Classics Appl. Math. 30, SIAM, Philadelphia, 2000.
G. B. Passty, Ergodic convergence to a zero of the sum of monotone operators in Hilbert space, J. Math. Anal. Appl., 72 (1979), pp. 383–390.
B. T. Polyak, Introduction to Optimization, Translations Series in Mathematics and Engineering, Optimization Software, Publications Division, New York, 1987.
J. L. Starck, D. L. Donoho, and E. J. Candès, Astronomical image representation by the curevelet transform, Astron. Astrophys., 398 (2003), pp. 785–800.
A. N. Tikhonov and V. Y. Arsenin, Solution of Ill-Posed Problems, V. H. Winston, Washington, DC, 1977.
C. Vonesch and M. Unser, Fast iterative thresholding algorithm for wavelet-regularized deconvolution, in Proceedings of the SPIE Optics and Photonics 2007 Conference on Mathematical Methods: Wavelet XII, Vol. 6701, San Diego, CA, 2007, pp. 1–5.
S. J. Wright, R. D. Nowak, and M. A. T. Figueiredo, Sparse reconstruction by separable approximation, in Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2008), 2008, pp. 3373–3376.

Information & Authors


Published In

cover image SIAM Journal on Imaging Sciences
SIAM Journal on Imaging Sciences
Pages: 183 - 202
ISSN (online): 1936-4954


Submitted: 25 February 2008
Accepted: 23 October 2008
Published online: 4 March 2009

MSC codes

  1. 90C25
  2. 90C06
  3. 65F22


  1. iterative shrinkage-thresholding algorithm
  2. deconvolution
  3. linear inverse problem
  4. least squares and $l_1$ regularization problems
  5. optimal gradient method
  6. global rate of convergence
  7. two-step iterative algorithms
  8. image deblurring



Metrics & Citations



If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

Cited By