Abstract

The class of L1-regularized optimization problems has received much attention recently because of the introduction of “compressed sensing,” which allows images and signals to be reconstructed from small amounts of data. Despite this recent attention, many L1-regularized problems still remain difficult to solve, or require techniques that are very problem-specific. In this paper, we show that Bregman iteration can be used to solve a wide variety of constrained optimization problems. Using this technique, we propose a “split Bregman” method, which can solve a very broad class of L1-regularized problems. We apply this technique to the Rudin–Osher–Fatemi functional for image denoising and to a compressed sensing problem that arises in magnetic resonance imaging.

MSC codes

  1. 65K05

Keywords

  1. constrained optimization
  2. L1-regularization
  3. compressed sensing
  4. total variation denoising

Get full access to this article

View all available purchase options and get full access to this article.

References

1.
R. Baraniuk and P. Steeghs, Compressive radar imaging, in Proceedings of the 2007 IEEE Radar Conference, 2007, pp. 128–133.
2.
S. Boyd and L. Vandenberghe, Convex Optimization, Cambridge University Press, Cambridge, UK, 2004.
3.
Y. Boykov, O. Veksler, and R. Zabih, Fast approximate energy minimization via graph cuts, IEEE Trans. Pattern Anal. Mach. Intell., 23 (2005), pp. 1222–1239.
4.
L. Bregman, The relaxation method of finding the common points of convex sets and its application to the solution of problems in convex optimization, USSR Comput. Math. and Math. Phys., 7 (1967), pp. 200–217.
5.
J. F. Cai, S. Osher, and Z. Shen, Linearized Bregman iterations for compressed sensing, Math. Comp., to appear.
6.
E. J. Candes and J. Romberg, Signal recovery from random projections, in Computational Imaging III, Proc. SPIE 5674, International Society for Optical Engineering, Bellingham, WA, 2005, pp. 76–86.
7.
E. J. Candes, J. Romberg, and T. Tao, Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information, IEEE Trans. Inform. Theory, 52 (2006), pp. 489–509.
8.
A. Chambolle, Total variation minimization and a class of binary MRF models, in Energy Minimization Methods in Computer Vision and Pattern Recognition, Springer-Verlag, Berlin, 2005, pp. 136–152.
9.
T. F. Chan, G. H. Golub, and P. Mulet, A nonlinear primal-dual method for total variation-based image restoration, SIAM J. Sci. Comput., 20 (1999), pp. 1964–1977.
10.
J. Darbon and M. Sigelle, A fast and exact algorithm for total variation minimization, in IbPRIA, Lecture Notes in Comput. Sci. 3522, Springer-Verlag, Berlin, 2005, pp. 351–359.
11.
D. Donoho, Compressed sensing, IEEE Trans. Inform. Theory, 52 (2006), pp. 1289–1306.
12.
D. Goldfarb and W. Yin, Parametric Maximum Flow Algorithms for Fast Total Variation Minimization, CAAM Technical Report TR07-09, Rice University, Houston, 2008.
13.
E. Hale, W. Yin, and Y. Zhang, A Fixed-Point Continuation Method for l1-Regularized Minimization with Applications to Compressed Sensing, CAAM Technical Report TR07-07, Rice University, Houston, 2007.
14.
L. He, T.-C. Chang, and S. Osher, MR image reconstruction from sparse radial samples by using iterative refinement procedures, in Proceedings of the 13th Annual Meeting of ISMRM, 2005, p. 696.
15.
S. Kim, K. Koh, M. Lustig, S. Boyd, and D. Gerinvesky, A Method for Large-Scale l1-Regularized Least Squares Problems with Applications in Signal Processing and Statistics, Technical report, Department of Electrical Engineering, Stanford University, Palo Alto, CA, 2007.
16.
Y. Li and F. Santosa, An Affine Scaling Algorithm for Minimizing Total Variation in Image Enhancement, Technical report TR94-1470, Center for Theory and Simulation in Science and Engineering, Cornell University, Ithaca, NY, 1994.
17.
M. Lustig, D. Donoho, and J. Pauly, Sparse MRI: The application of compressed sensing for rapid MR imaging, Magn. Reson. Med., 58 (2007), pp. 1182–1195.
18.
M. Lustig, J. H. Lee, D. L. Donoho, and J. M. Pauly, Faster imaging with randomly perturbed undersampled spirals and $L_1$ reconstruction, in Proceedings of the 13th Scientific Meeting of the ISMRM, 2005.
19.
G. L. Marseille, R. de Beer, M. Fuderer, A. F. Mehlkopf, and D. van Ormondt, Nonuniform phase-encode distributions for MRI scan-time reduction, J. Magn. Reson., 111 (1996), pp. 70–75.
20.
J. Nocedal and S. Wright, Numerical Optimization, 2nd ed., Springer-Verlag, New York, 2006.
21.
A. Osher, Y. Mao, B. Dong, and W. Yin, Fast Linearized Bregman Iterations for Compressed Sensing and Sparse Denoising, UCLA CAM Report, 08-37, University of California at Los Angeles, Los Angeles, 2008.
22.
S. Osher, M. Burger, D. Goldfarb, J. Xu, and W. Yin, An iterative regularization method for total variation-based image restoration, Multiscale Model. Simul., 4 (2005), pp. 460–489.
23.
L. Rudin, S. Osher, and E. Fatemi, Nonlinear total variation based noise removal algorithms, Phys. D, 60 (1992), pp. 259–268.
24.
J. Trzasko, A. Manduca, and E. Borisch, Sparse MRI reconstruction via multiscale L0-continuation, in Proceedings of the 14th IEEE/SP Workshop on Statistical Signal Processing, 2007, pp. 176–180.
25.
C. Vogel, A multigrid method for total variation-based image denoising, in Computation and Control, IV, Progr. Systems Control Theory 20, Birkhäuser-Boston, Boston, 1995, pp. 323–331.
26.
C. R. Vogel and M. E. Oman, Iterative methods for total variation denoising, SIAM J. Sci. Comput., 17 (1996), pp. 227–238.
27.
Y. Wang, W. Yin, and Y. Zhang, A Fast Algorithm for Image Deblurring with Total Variation Regularization, CAAM Technical Report TR07-10, Rice University, Houston, 2007.
28.
J. Xu and S. Osher, Iterative regularization and nonlinear inverse scale space applied to wavelet-based denoising, IEEE Trans. Image Process., 16 (2007), pp. 534–544.
29.
W. Yin, PGC: A Preflow-Push Based Graph-Cut Solver, Version 2.32, http://www.caam.rice.edu/ optimization/L1/pgc/ http://www.caam.rice.edu/ optimization/L1/pgc/.
30.
W. Yin, S. Osher, D. Goldfarb, and J. Darbon, Bregman iterative algorithms for $\ell_1$-minimization with applications to compressed sensing, SIAM J. Imaging Sci., 1 (2008), pp. 143–168.

Information & Authors

Information

Published In

cover image SIAM Journal on Imaging Sciences
SIAM Journal on Imaging Sciences
Pages: 323 - 343
ISSN (online): 1936-4954

History

Submitted: 2 June 2008
Accepted: 4 November 2008
Published online: 1 April 2009

MSC codes

  1. 65K05

Keywords

  1. constrained optimization
  2. L1-regularization
  3. compressed sensing
  4. total variation denoising

Authors

Affiliations

Metrics & Citations

Metrics

Citations

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

Cited By