Open access

A Multilevel Proximal Gradient Algorithm for a Class of Composite Optimization Problems


Composite optimization models consist of the minimization of the sum of a smooth (not necessarily convex) function and a nonsmooth convex function. Such models arise in many applications where, in addition to the composite nature of the objective function, a hierarchy of models is readily available. It is common to take advantage of this hierarchy of models by first solving a low fidelity model and then using the solution as a starting point to a high fidelity model. We adopt an optimization point of view and show how to take advantage of the availability of a hierarchy of models in a consistent manner. We do not use the low fidelity model just for the computation of promising starting points but also for the computation of search directions. We establish the convergence and convergence rate of the proposed algorithm. Our numerical experiments on large scale image restoration problems and the transition path problem suggest that, for certain classes of problems, the proposed algorithm is significantly faster than the state of the art.


  1. composite optimization
  2. multigrid
  3. nonsmooth optimization

MSC codes

  1. 90-08
  2. 90C25
  3. 90C26

Formats available

You can view the full content in the following formats:


A. Beck and M. Teboulle, A fast iterative shrinkage-thresholding algorithm for linear inverse problems, SIAM J. Imaging Sci., 2 (2009), pp. 183--202.
A. Beck and M. Teboulle, Smoothing and first order methods: A unified framework, SIAM J. Optim., 22 (2012), pp. 557--580.
D.P. Bertsekas, Nonlinear Programming, Optim. Comput. Ser., Athena Scientific, Belmont, MA, 1999.
A. Borz\`\i, On the convergence of the MG/OPT method, Proc. Appl. Math. Mech., 5 (2005), pp. 735--736.
A. Borz\`\i and V. Schulz, Multigrid methods for PDE optimization, SIAM Rev., 51 (2009), pp. 361--395.
A. Brandt, J. Brannick, K. Kahl, and I. Livshits, Bootstrap AMG, SIAM J. Sci. Comput., 33 (2011), pp. 612--632.
W.L. Briggs, V.E. Henson, and S.F. McCormick, A Multigrid Tutorial, 2nd ed., SIAM, Philadelphia, 2000.
C. Gräser and R. Kornhuber, Multigrid methods for obstacle problems, J. Comput. Math., 27 (2009), pp. 1--44.
C. Gräser, U. Sack, and O. Sander, Truncated nonsmooth Newton multigrid methods for convex minimization problems, in Domain Decomposition Methods in Science and Engineering XVIII, Springer, New York, 2009, pp. 129--136.
S. Gratton, M. Mouffe, A. Sartenaer, P.L. Toint, and D. Tomanos, Numerical experience with a recursive trust-region method for multilevel nonlinear bound-constrained optimization, Optim. Methods Softw., 25 (2010), pp. 359--386.
S. Gratton, M. Mouffe, P.L. Toint, and M. Weber-Mendonça, A recursive-trust-region method for bound-constrained nonlinear optimization, IMA J. Numer. Anal., 28 (2008), pp. 827--861.
S. Gratton, A. Sartenae, and P.L. Toint, Recursive trust-region methods for multiscale nonlinear optimization, SIAM J. Optim., 19 (2008), pp. 414--444.
P.C. Hansen, J.G. Nagy, and D.P. O'Leary, Deblurring Images: Matrices, Spectra, and Filtering, Vol. 3, SIAM, Philadelphia, 2006.
J.B. Hiriart-Urruty and C. Lemaréchal, Convex Analysis and Minimization Algorithms I: Fundamentals, Vol. 305, Springer, New York, 2013.
V. Hovhannisyan, P. Parpas, and S. Zafeiriou, MAGMA: Multilevel accelerated gradient mirror descent algorithm for large-scale convex composite minimization, SIAM J. Imaging Sci., 9 (2016), pp. 1829--1857.
G. Lan, An optimal method for stochastic composite optimization, Math. Program., 133 (2012), pp. 365--397.
J.D. Lee, Y. Sun, and M.A. Saunders, Proximal Newton-type Methods for Minimizing Composite Functions, SIAM J. Optim., 24 (2014), pp. 1420--1443.
A.S. Lewis and S.J. Wright, A Proximal Method for Composite Minimization, Math. Program., 158 (2016), pp. 501--546.
R.M. Lewis and S.G. Nash, Model problems for the multigrid optimization of systems governed by differential equations, SIAM J. Sci. Comput., 26 (2005), pp. 1811--1837.
J.J. Moré and T.S. Munson, Computing mountain passes and transition states, Math. Program., 100 (2004), pp. 151--182.
S.G. Nash, A multigrid approach to discretized optimization problems, Optim. Methods Softw., 14 (2000), pp. 99--116.
S.G. Nash, Properties of a class of multilevel optimization algorithms for equality-constrained problems, Optim. Methods Softw., 29 (2014), pp. 137--159.
S.G. Nash and R.M. Lewis, Assessing the performance of an optimization-based multilevel method, Optim. Methods Softw., 26 (2011), pp. 693--717.
Y. Nesterov, Gradient methods for minimizing composite objective function, Math. Program., 140 (2013), pp. 125--161.
S. Oh, A.B. Milstein, C.A. Bouman, and K.J. Webb, A general framework for nonlinear multigrid inversion, IEEE Trans. Image Process., 14 (2005), pp. 125--140.
N. Parikh and S. Boyd, Proximal algorithms, Found. Trends Optim., 1 (2014), pp. 127--239.
P. Parpas and M. Webster, A stochastic multiscale model for electricity generation capacity expansion, European J. Oper. Res., 232 (2014), pp. 359--374.
E. Polak, R.S. Womersley, and H.X. Yin, An algorithm based on active sets and smoothing for discretized semi-infinite minimax problems, J. Optim. Theory Appl., 138 (2008), pp. 311--328.
P. Richtárik and M. Takáč, Iteration complexity of randomized block-coordinate descent methods for minimizing a composite function, Math. Program., 144 (2014), pp. 1--38.
J.J. Thiagarajan, K.N. Ramamurthy, and A. Spanias, Learning stable multilevel dictionaries for sparse representation of images, IEEE Trans. Neural Netw. Learn. Syst., under review, 2013.
A. Tsoukalas, P. Parpas, and B. Rustem, A smoothing algorithm for finite min--max--min problems, Optim. Lett., 3 (2009), pp. 49--62.
Z. Wen and D. Goldfarb, A line search multigrid method for large-scale nonlinear optimization, SIAM J. Optim., 20 (2009), pp. 1478--1503.
Z. Lei, Q. Du, and Z. Zheng, Optimization-based shrinking dimer method for finding transition states, SIAM J. Sci. Comput., 38 (2016), pp. A528--A544.

Information & Authors


Published In

cover image SIAM Journal on Scientific Computing
SIAM Journal on Scientific Computing
Pages: S681 - S701
ISSN (online): 1095-7197


Submitted: 1 July 2016
Accepted: 8 May 2017
Published online: 26 October 2017


  1. composite optimization
  2. multigrid
  3. nonsmooth optimization

MSC codes

  1. 90-08
  2. 90C25
  3. 90C26



Metrics & Citations



If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

Cited By







Copy the content Link

Share with email

Email a colleague

Share on social media

The SIAM Publications Library now uses SIAM Single Sign-On for individuals. If you do not have existing SIAM credentials, create your SIAM account