The Split Bregman Method for L1-Regularized Problems

The class of L1-regularized optimization problems has received much attention recently because of the introduction of “compressed sensing,” which allows images and signals to be reconstructed from small amounts of data. Despite this recent attention, many L1-regularized problems still remain difficult to solve, or require techniques that are very problem-specific. In this paper, we show that Bregman iteration can be used to solve a wide variety of constrained optimization problems. Using this technique, we propose a “split Bregman” method, which can solve a very broad class of L1-regularized problems. We apply this technique to the Rudin–Osher–Fatemi functional for image denoising and to a compressed sensing problem that arises in magnetic resonance imaging.

  • [1]  R. Baraniuk and P. Steeghs, Compressive radar imaging, in Proceedings of the 2007 IEEE Radar Conference, 2007, pp. 128–133. Google Scholar

  • [2]  S. Boyd and L. Vandenberghe, Convex Optimization, Cambridge University Press, Cambridge, UK, 2004. Google Scholar

  • [3]  Y. Boykov, O. Veksler and , and R. Zabih, Fast approximate energy minimization via graph cuts, IEEE Trans. Pattern Anal. Mach. Intell., 23 (2005), pp. 1222–1239. ITPIDJ 0162-8828 CrossrefISIGoogle Scholar

  • [4]  L. Bregman, The relaxation method of finding the common points of convex sets and its application to the solution of problems in convex optimization, USSR Comput. Math. and Math. Phys., 7 (1967), pp. 200–217. CMMPA9 0965-5425 CrossrefGoogle Scholar

  • [5]  J. F. Cai, S. Osher, and Z. Shen, Linearized Bregman iterations for compressed sensing, Math. Comp., to appear. Google Scholar

  • [6]  E. J. Candes and J. Romberg, Signal recovery from random projections, in Computational Imaging III, Proc. SPIE 5674, International Society for Optical Engineering, Bellingham, WA, 2005, pp. 76–86. Google Scholar

  • [7]  E. J. Candes, J. Romberg and , and T. Tao, Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information, IEEE Trans. Inform. Theory, 52 (2006), pp. 489–509. IETTAW 0018-9448 CrossrefISIGoogle Scholar

  • [8]  A. Chambolle, Total variation minimization and a class of binary MRF models, in Energy Minimization Methods in Computer Vision and Pattern Recognition, Springer-Verlag, Berlin, 2005, pp. 136–152. Google Scholar

  • [9]  T. F. Chan, G. H. Golub and , and P. Mulet, A nonlinear primal-dual method for total variation-based image restoration, SIAM J. Sci. Comput., 20 (1999), pp. 1964–1977. SJOCE3 1064-8275 LinkISIGoogle Scholar

  • [10]  J. Darbon and M. Sigelle, A fast and exact algorithm for total variation minimization, in IbPRIA, Lecture Notes in Comput. Sci. 3522, Springer-Verlag, Berlin, 2005, pp. 351–359. Google Scholar

  • [11]  D. Donoho, Compressed sensing, IEEE Trans. Inform. Theory, 52 (2006), pp. 1289–1306. IETTAW 0018-9448 CrossrefISIGoogle Scholar

  • [12]  D. Goldfarb and W. Yin, Parametric Maximum Flow Algorithms for Fast Total Variation Minimization, CAAM Technical Report TR07-09, Rice University, Houston, 2008. Google Scholar

  • [13]  E. Hale, W. Yin, and Y. Zhang, A Fixed-Point Continuation Method for l1-Regularized Minimization with Applications to Compressed Sensing, CAAM Technical Report TR07-07, Rice University, Houston, 2007. Google Scholar

  • [14]  L. He, T.-C. Chang, and S. Osher, MR image reconstruction from sparse radial samples by using iterative refinement procedures, in Proceedings of the 13th Annual Meeting of ISMRM, 2005, p. 696. Google Scholar

  • [15]  S. Kim, K. Koh, M. Lustig, S. Boyd, and D. Gerinvesky, A Method for Large-Scale l1-Regularized Least Squares Problems with Applications in Signal Processing and Statistics, Technical report, Department of Electrical Engineering, Stanford University, Palo Alto, CA, 2007. Google Scholar

  • [16]  Y. Li and F. Santosa, An Affine Scaling Algorithm for Minimizing Total Variation in Image Enhancement, Technical report TR94-1470, Center for Theory and Simulation in Science and Engineering, Cornell University, Ithaca, NY, 1994. Google Scholar

  • [17]  M. Lustig, D. Donoho and , and J. Pauly, Sparse MRI: The application of compressed sensing for rapid MR imaging, Magn. Reson. Med., 58 (2007), pp. 1182–1195. MRMEEN 0740-3194 CrossrefISIGoogle Scholar

  • [18]  M. Lustig, J. H. Lee, D. L. Donoho, and J. M. Pauly, Faster imaging with randomly perturbed undersampled spirals and $L_1$ reconstruction, in Proceedings of the 13th Scientific Meeting of the ISMRM, 2005. Google Scholar

  • [19]  G. L. Marseille, R. de Beer, M. Fuderer, A. F. Mehlkopf and , and D. van Ormondt, Nonuniform phase-encode distributions for MRI scan-time reduction, J. Magn. Reson., 111 (1996), pp. 70–75. CrossrefISIGoogle Scholar

  • [20]  J. Nocedal and S. Wright, Numerical Optimization, 2nd ed., Springer-Verlag, New York, 2006. Google Scholar

  • [21]  A. Osher, Y. Mao, B. Dong, and W. Yin, Fast Linearized Bregman Iterations for Compressed Sensing and Sparse Denoising, UCLA CAM Report, 08-37, University of California at Los Angeles, Los Angeles, 2008. Google Scholar

  • [22]  S. Osher, M. Burger, D. Goldfarb, J. Xu and , and W. Yin, An iterative regularization method for total variation-based image restoration, Multiscale Model. Simul., 4 (2005), pp. 460–489. MMSUBT 1540-3459 LinkISIGoogle Scholar

  • [23]  L. Rudin, S. Osher and , and E. Fatemi, Nonlinear total variation based noise removal algorithms, Phys. D, 60 (1992), pp. 259–268. PDNPDT 0167-2789 CrossrefISIGoogle Scholar

  • [24]  J. Trzasko, A. Manduca, and E. Borisch, Sparse MRI reconstruction via multiscale L0-continuation, in Proceedings of the 14th IEEE/SP Workshop on Statistical Signal Processing, 2007, pp. 176–180. Google Scholar

  • [25]  C. Vogel, A multigrid method for total variation-based image denoising, in Computation and Control, IV, Progr. Systems Control Theory 20, Birkhäuser-Boston, Boston, 1995, pp. 323–331. Google Scholar

  • [26]  C. R. Vogel and  and M. E. Oman, Iterative methods for total variation denoising, SIAM J. Sci. Comput., 17 (1996), pp. 227–238. SJOCE3 1064-8275 LinkISIGoogle Scholar

  • [27]  Y. Wang, W. Yin, and Y. Zhang, A Fast Algorithm for Image Deblurring with Total Variation Regularization, CAAM Technical Report TR07-10, Rice University, Houston, 2007. Google Scholar

  • [28]  J. Xu and  and S. Osher, Iterative regularization and nonlinear inverse scale space applied to wavelet-based denoising, IEEE Trans. Image Process., 16 (2007), pp. 534–544. IIPRE4 1057-7149 CrossrefISIGoogle Scholar

  • [29]  W. Yin, PGC: A Preflow-Push Based Graph-Cut Solver, Version 2.32, http://www.caam.rice.edu/ optimization/L1/pgc/ http://www.caam.rice.edu/ optimization/L1/pgc/. Google Scholar

  • [30]  W. Yin, S. Osher, D. Goldfarb and , and J. Darbon, Bregman iterative algorithms for $\ell_1$-minimization with applications to compressed sensing, SIAM J. Imaging Sci., 1 (2008), pp. 143–168. LinkISIGoogle Scholar