This paper introduces a generalized forward-backward splitting algorithm for finding a zero of a sum of maximal monotone operators $B + \sum_{i=1}^n A_i$, where $B$ is cocoercive. It involves the computation of $B$ in an explicit (forward) step and the parallel computation of the resolvents of the $A_i$'s in a subsequent implicit (backward) step. We prove the algorithm's convergence in infinite dimension and its robustness to summable errors on the computed operators in the explicit and implicit steps. In particular, this allows efficient minimization of the sum of convex functions $f + \sum_{i=1}^n g_i$, where $f$ has a Lipschitz-continuous gradient and each $g_i$ is simple in the sense that its proximity operator is easy to compute. The resulting method makes use of the regularity of $f$ in the forward step, and the proximity operators of the $g_i$'s are applied in parallel in the backward step. While the forward-backward algorithm cannot deal with more than $n = 1$ nonsmooth function, we generalize it to the case of arbitrary $n$. Examples on inverse problems in imaging demonstrate the advantage of the proposed methods in comparison to other splitting algorithms.


  1. forward-backward algorithm
  2. monotone operator splitting
  3. nonsmooth convex optimization
  4. proximal splitting
  5. image processing
  6. sparsity

MSC codes

  1. 47N10
  2. 90C25
  3. 94A08

Get full access to this article

View all available purchase options and get full access to this article.


F. Acker and M.-A. Prestel, Convergence d'un schéma de minimisation altérnée, Ann. Fac. Sci. Toulouse Math. (5), 2 (1980), pp. 1--9.
M. V. Afonso, J. M. Bioucas-Dias, and M. A. T. Figueiredo, Fast image recovery using variable splitting and constrained optimization, IEEE Trans. Image Process., 19 (2010), pp. 2345--2356.
K. J. Arrow, L. Hurwicz, and H. Uzawa, Studies in Linear and Non-linear Programming, Stanford University Press, Stanford, CA, 1958.
H. Attouch, L. M. Bricen͂o-Arias, and P. L. Combettes, A parallel splitting method for coupled monotone inclusions, SIAM J. Control Optim., 48 (2010), pp. 3246--3270.
J.-B. Baillon and G. Haddad, Quelques propriétés des opérateurs angle-bornés et n-cycliquement monotones, Israel J. Math., 26 (1977), pp. 137--150.
H. H. Bauschke and P. L. Combettes, Convex Analysis and Monotone Operator Theory in Hilbert Spaces, Springer-Verlag, New York, 2011.
H. H. Bauschke, P. L. Combettes, and S. Reich, The asymptotic behavior of the composition of two resolvents, Nonlinear Anal., 60 (2005), pp. 283--301.
A. Beck and M. Teboulle, A fast iterative shrinkage-thresholding algorithm for linear inverse problems, SIAM J. Imaging Sci., 2 (2009), pp. 183--202.
K. Bredies and D. A. Lorenz, Linear convergence of iterative soft-thresholding, J. Fourier Anal. Appl., 14 (2008), pp. 813--837.
L. M. Bricen͂o-Arias and P. L. Combettes, Convex variational formulation with smooth coupling for multicomponent signal decomposition and recovery, Numer. Math. Theory Methods Appl., 2 (2009), pp. 485--508.
L. M. Bricen͂o-Arias and P. L. Combettes, A monotone + skew splitting model for composite monotone inclusions in duality, SIAM J. Optim., 21 (2011), pp. 1230--1250.
L. M. Bricen͂o-Arias, P. L. Combettes, J.-C. Pesquet, and N. Pustelnik, Proximal algorithms for multicomponent image recovery problems, J. Math. Imaging Vision, 41 (2011), pp. 3--22.
F. E. Browder, Convergence theorems for sequences of nonlinear operators in Banach spaces, Math. Z., 100 (1967), pp. 201--225.
A. Chambolle and T. Pock, A first-order primal-dual algorithm for convex problems with applications to imaging, J. Math. Imaging Vision, 40 (2011), pp. 120--145.
C. Chaux, A. Benazza-Benyahia, and J.-C. Pesquet, A block-thresholding method for multispectral image denoising, in Proceedings of the SPIE, Vol. 5914, San Diego, CA, 2005, pp. 1H-1--1H-13.
C. Chaux, P. L. Combettes, J.-C. Pesquet, and V. R. Wajs, A variational formulation for frame-based inverse problems, Inverse Problems, 23 (2007), pp. 1495--1518.
C. Chaux, J.-C. Pesquet, and N. Pustelnik, Nested iterative algorithms for convex constrained image recovery problems, SIAM J. Imaging Sci., 2 (2009), pp. 730--762.
G. Chen and M. Teboulle, A proximal-based decomposition method for convex minimization problems, Math. Programming, 64 (1994), pp. 81--101.
G. H.-G. Chen and R. T. Rockafellar, Convergence rates in forward--backward splitting, SIAM J. Optim., 7 (1997), pp. 421--444.
P. L. Combettes, Solving monotone inclusions via compositions of nonexpansive averaged operators, Optimization, 53 (2004), pp. 475--504.
P. L. Combettes, Iterative construction of the resolvent of a sum of maximal monotone operators, J. Convex Anal., 16 (2009), pp. 727--748.
P. L. Combettes, D. Du͂ng, and B. C. Vu͂, Dualization of signal recovery problems, Set-Valued Var. Anal., 18 (2010), pp. 373--404.
P. L. Combettes and J.-C. Pesquet, A Douglas-Rachford splitting approach to nonsmooth convex variational signal recovery, IEEE J. Sel. Topics Signal Process., 1 (2007), pp. 564--574.
P. L. Combettes and J.-C. Pesquet, A proximal decomposition method for solving convex variational inverse problems, Inverse Problems, 24 (2008), 065014.
P. L. Combettes and J.-C. Pesquet, Proximal splitting methods in signal processing, in Fixed-Point Algorithms for Inverse Problems in Science and Engineering, Springer-Verlag, New York, 2011, pp. 185--212.
P. L. Combettes and J.-C. Pesquet, Primal-dual splitting algorithm for solving inclusions with mixtures of composite, Lipschitzian, and parallel-sum monotone operators, Set-Valued Var. Anal., 20 (2012), pp. 307--330.
P. L. Combettes and V. R. Wajs, Signal recovery by proximal forward-backward splitting, Multiscale Model. Simul., 4 (2005), pp. 1168--1200.
L. Condat, A primal-dual splitting method for convex optimization involving Lipschitzian, proximable and linear composite terms, J. Optim. Theory Appl., to appear.
I. Daubechies, M. Defrise, and C. De Mol, An iterative thresholding algorithm for linear inverse problems with a sparsity constraint, Comm. Pure Appl. Math., 57 (2004), pp. 1413--1541.
D. L. Donoho, De-noising by soft-thresholding, IEEE Trans. Inform. Theory, 41 (1995), pp. 613--627.
J. Douglas and H. H. Rachford, On the numerical solution of heat conduction problems in two and three space variables, Trans. Amer. Math. Soc., 82 (1956), pp. 421--439.
F.-X. Dupé, J. M. Fadili, and J.-L. Starck, A proximal iteration for deconvolving Poisson noisy images using sparse representations, IEEE Trans. Image Process., 18 (2009), pp. 310--321.
F.-X. Dupé, J. M. Fadili, and J.-L. Starck, Linear inverse problems with various noise models and mixed regularizations, in Proceedings of the 1st International ICST Workshop on New Computational Methods for Inverse Problems, Paris, 2011.
F.-X. Dupé, M. J. Fadili, and J.-L. Starck, Inverse problems with Poisson noise: Primal and primal-dual splitting, in Proceedings of the 18th IEEE International Conference on Image Processing (ICIP), Brussels, Belgium, 2011.
F.-X. Dupé, M. J. Fadili, and J.-L. Starck, Deconvolution under Poisson noise using exact data fidelity and synthesis or analysis sparsity priors, Stat. Methodol., 9 (2012), pp. 4--18.
J. Eckstein, Parallel alternating direction multiplier decomposition of convex programs, J. Optim. Theory Appl., 80 (1994), pp. 39--62.
J. Eckstein and D. P. Bertsekas, On the Douglas-Rachford splitting method and the proximal point algorithm for maximal monotone operators, Math. Programming, 55 (1992), pp. 293--318.
J. Eckstein and B. F. Svaiter, General projective splitting methods for sums of maximal monotone operators, SIAM J. Control Optim., 48 (2009), pp. 787--811.
E. Elhamifar and R. Vidal, Robust classification using structured sparse representation, in Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2011, pp. 1873--1879.
M. J. Fadili and G. Peyré, Total variation projection with first order schemes, IEEE Trans. Image Process., 20 (2010), pp. 657--669.
M. J. Fadili, J.-L. Starck, and F. Murtagh, Inpainting and zooming using sparse representations, Comput. J., 52 (2009), pp. 64--79.
M. A. T. Figueiredo and J. M. Bioucas-Dias, Restoration of Poissonian images using alternating direction optimization, IEEE Trans. Image Process., 19 (2010), pp. 3133--3145.
M. A. T. Figueiredo and R. Nowak, An EM algorithm for wavelet-based image restoration, IEEE Trans. Image Process., 12 (2003), pp. 906--916.
M. Fortin and R. Glowinski, Augmented Lagrangian Methods: Applications to the Numerical Solution of Boundary-Value Problems, Elsevier Science Publishers, Amsterdam, 1983.
D. Gabay, Applications of the method of multipliers to variational inequalities, in Augmented Lagrangian Methods: Applications to the Numerical Solution of Boundary-Value Problems, M. Fortin and R. Glowinski, eds., North-Holland, Amsterdam, 1983, pp. 299--331.
D. Gabay and B. Mercier, A dual algorithm for the solution of nonlinear variational problems via finite element approximation, Comput. Math. Appl., 2 (1976), pp. 17--40.
R. Glowinski and P. Le Tallec, Augmented Lagrangian and Operator-Splitting Methods in Nonlinear Mechanics, SIAM, Philadelphia, 1989.
R. Jenatton, J. Mairal, G. Obozinski, and F. Bach, Proximal Methods for Hierarchical Sparse Coding, preprint, arXiv:1009.2139 [stat.ML], 2010.
G. M. Korpelevich, An extragradient method for finding saddle points and for other problems, Èkonom. i Mat. Metody, 12 (1976), pp. 747--756.
M. Kowalski and B. Torrésani, Sparsity and persistence: Mixed norms provide simple signal models with dependent coefficients, Signal, Image and Video Processing, 3 (2009), pp. 251--264.
B. Lemaire, Stability of the iteration method for nonexpansive mappings, Serdica Math. J., 22 (1996), pp. 331--340.
B. Lemaire, Which fixed point does the iteration method select?, in Recent Advances in Optimization: The 8th French-German Conference on Optimization, P. Gritzmann, R. Horst, E. Sachs, and R. Tichatschke, eds., Lecture Notes in Econom. and Math. Systems 452, Springer-Verlag, Berlin, 1997, pp. 154--167.
J. Lieutaud, Approximation d'Opérateurs par des Méthodes de Décomposition, Ph.D. thesis, Université de Paris, Paris, 1969.
P.-L. Lions, Une méthode itérative de résolution d'une inéquation variationnelle, Israel J. Math., 31 (1978), pp. 204--208.
P.-L. Lions and B. Mercier, Splitting algorithms for the sum of two nonlinear operators, SIAM J. Numer. Anal., 16 (1979), pp. 964--979.
S. Mallat, A Wavelet Tour of Signal Processing. The Sparse Way, 3rd ed., Elsevier/Academic Press, Amsterdam, 2009.
B. Mercier, Lectures on Topics in Finite Element Solution of Elliptic Problems, Tata Inst. Fund. Res. Lectures on Math. and Phys. 63, Tata Institute of Fundamental Research, Bombay, India, 1979.
O. J. Minty, Montone (nonlinear) operators in Hilbert space, Duke Math. J., 29 (1962), pp. 341--346.
R. D. C. Monteiro and B. F. Svaiter, Iteration-complexity of block-decomposition algorithms and the alternating direction method of multipliers, SIAM J. Optim., 23 (2013), pp. 475--507.
J.-J. Moreau, Proximité et dualité dans un espace hilbertien, Bull. Soc. Math. France, 93 (1965), pp. 273--299.
Yu. Nesterov, Gradient methods for minimizing composite functions, Math. Program., 140 (2013), pp. 125--161.
G. B. Passty, Ergodic convergence to a zero of the sum of monotone operators in Hilbert space, J. Math. Anal. Appl., 72 (1979), pp. 383--390.
J.-C. Pesquet and N. Pustelnik, A parallel inertial proximal optimization method, Pac. J. Optim., 8 (2012), pp. 273--306.
G. Peyré and J. Fadili, Group sparsity with overlapping partition functions, in Proceedings of the 19th European Signal Processing Conference (EUSIPCO 2011), 2011, pp. 303--307.
R. R. Phelps, Convex Functions, Monotone Operators and Differentiability, Lecture Notes in Math. 1364, 2nd ed., Springer-Verlag, Berlin, 1993.
L. D. Popov, A modification of the Arrow-Hurwitz method of search for saddle points, Mat. Zametki, 28 (1980), pp. 777--784.
N. Pustelnik, C. Chaux, and J.-C. Pesquet, Parallel proximal algorithm for image restoration using hybrid regularization, IEEE Trans. Image Process., 20 (2011), pp. 2450--2462.
R. T. Rockafellar, Convex Analysis, Princeton University Press, Princeton, NJ, 1970.
L. I. Rudin, S. Osher, and E. Fatemi, Nonlinear total variation based noise removal algorithms, Phys. D, 60 (1992), pp. 259--268.
M. V. Solodov, A class of decomposition methods for convex optimization and monotone variational inclusions via the hybrid inexact proximal point framework, Optim. Methods Softw., 19 (2004), pp. 557--575.
J. E. Spingarn, Partial inverse of a monotone operator, Appl. Math. Optim., 10 (1983), pp. 247--265.
J.-L. Starck, F. Murtagh, and M. J. Fadili, Sparse Signal and Image Processing: Wavelets, Curvelets and Morphological Diversity, Cambridge University Press, Cambridge, UK, 2010.
P. Tseng, Applications of a splitting algorithm to decomposition in convex programming and variational inequalities, SIAM J. Control Optim., 29 (1991), pp. 119--138.
P. Tseng, Alternating projection-proximal methods for convex programming and variational inequalities, SIAM J. Optim., 7 (1997), pp. 951--965.
P. Tseng, A modified forward-backward splitting method for maximal monotone mappings, SIAM J. Control Optim., 38 (2000), pp. 431--446.
B. C. Vu͂, A splitting algorithm for dual monotone inclusions involving cocoercive operators, Adv. Comput. Math., 38 (2013), pp. 667--681.
M. Yuan and Y. Lin, Model selection and estimation in regression with grouped variables, J. R. Stat. Soc. Ser. B Stat. Methodol., 68 (2006), pp. 49--67.
E. H. Zarantonello, Projections on convex sets in Hilbert space and spectral theory. \textupI. Projections on convex sets, in Contributions to Nonlinear Functional Analysis, Academic Press, New York, 1971, pp. 237--341.

Information & Authors


Published In

cover image SIAM Journal on Imaging Sciences
SIAM Journal on Imaging Sciences
Pages: 1199 - 1226
ISSN (online): 1936-4954


Submitted: 9 April 2012
Accepted: 29 March 2013
Published online: 9 July 2013


  1. forward-backward algorithm
  2. monotone operator splitting
  3. nonsmooth convex optimization
  4. proximal splitting
  5. image processing
  6. sparsity

MSC codes

  1. 47N10
  2. 90C25
  3. 94A08



Metrics & Citations



If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

Cited By







Copy the content Link

Share with email

Email a colleague

Share on social media

The SIAM Publications Library now uses SIAM Single Sign-On for individuals. If you do not have existing SIAM credentials, create your SIAM account https://my.siam.org.