A Multilevel Proximal Gradient Algorithm for a Class of Composite Optimization Problems
Abstract
Composite optimization models consist of the minimization of the sum of a smooth (not necessarily convex) function and a nonsmooth convex function. Such models arise in many applications where, in addition to the composite nature of the objective function, a hierarchy of models is readily available. It is common to take advantage of this hierarchy of models by first solving a low fidelity model and then using the solution as a starting point to a high fidelity model. We adopt an optimization point of view and show how to take advantage of the availability of a hierarchy of models in a consistent manner. We do not use the low fidelity model just for the computation of promising starting points but also for the computation of search directions. We establish the convergence and convergence rate of the proposed algorithm. Our numerical experiments on large scale image restoration problems and the transition path problem suggest that, for certain classes of problems, the proposed algorithm is significantly faster than the state of the art.
Keywords
MSC codes
Formats available
You can view the full content in the following formats:
References
1.
A. Beck and M. Teboulle, A fast iterative shrinkage-thresholding algorithm for linear inverse problems, SIAM J. Imaging Sci., 2 (2009), pp. 183--202.
2.
A. Beck and M. Teboulle, Smoothing and first order methods: A unified framework, SIAM J. Optim., 22 (2012), pp. 557--580.
3.
D.P. Bertsekas, Nonlinear Programming, Optim. Comput. Ser., Athena Scientific, Belmont, MA, 1999.
4.
A. Borz\`\i, On the convergence of the MG/OPT method, Proc. Appl. Math. Mech., 5 (2005), pp. 735--736.
5.
A. Borz\`\i and V. Schulz, Multigrid methods for PDE optimization, SIAM Rev., 51 (2009), pp. 361--395.
6.
A. Brandt, J. Brannick, K. Kahl, and I. Livshits, Bootstrap AMG, SIAM J. Sci. Comput., 33 (2011), pp. 612--632.
7.
W.L. Briggs, V.E. Henson, and S.F. McCormick, A Multigrid Tutorial, 2nd ed., SIAM, Philadelphia, 2000.
8.
C. Gräser and R. Kornhuber, Multigrid methods for obstacle problems, J. Comput. Math., 27 (2009), pp. 1--44.
9.
C. Gräser, U. Sack, and O. Sander, Truncated nonsmooth Newton multigrid methods for convex minimization problems, in Domain Decomposition Methods in Science and Engineering XVIII, Springer, New York, 2009, pp. 129--136.
10.
S. Gratton, M. Mouffe, A. Sartenaer, P.L. Toint, and D. Tomanos, Numerical experience with a recursive trust-region method for multilevel nonlinear bound-constrained optimization, Optim. Methods Softw., 25 (2010), pp. 359--386.
11.
S. Gratton, M. Mouffe, P.L. Toint, and M. Weber-Mendonça, A recursive-trust-region method for bound-constrained nonlinear optimization, IMA J. Numer. Anal., 28 (2008), pp. 827--861.
12.
S. Gratton, A. Sartenae, and P.L. Toint, Recursive trust-region methods for multiscale nonlinear optimization, SIAM J. Optim., 19 (2008), pp. 414--444.
13.
P.C. Hansen, J.G. Nagy, and D.P. O'Leary, Deblurring Images: Matrices, Spectra, and Filtering, Vol. 3, SIAM, Philadelphia, 2006.
14.
J.B. Hiriart-Urruty and C. Lemaréchal, Convex Analysis and Minimization Algorithms I: Fundamentals, Vol. 305, Springer, New York, 2013.
15.
V. Hovhannisyan, P. Parpas, and S. Zafeiriou, MAGMA: Multilevel accelerated gradient mirror descent algorithm for large-scale convex composite minimization, SIAM J. Imaging Sci., 9 (2016), pp. 1829--1857.
16.
G. Lan, An optimal method for stochastic composite optimization, Math. Program., 133 (2012), pp. 365--397.
17.
J.D. Lee, Y. Sun, and M.A. Saunders, Proximal Newton-type Methods for Minimizing Composite Functions, SIAM J. Optim., 24 (2014), pp. 1420--1443.
18.
A.S. Lewis and S.J. Wright, A Proximal Method for Composite Minimization, Math. Program., 158 (2016), pp. 501--546.
19.
R.M. Lewis and S.G. Nash, Model problems for the multigrid optimization of systems governed by differential equations, SIAM J. Sci. Comput., 26 (2005), pp. 1811--1837.
20.
J.J. Moré and T.S. Munson, Computing mountain passes and transition states, Math. Program., 100 (2004), pp. 151--182.
21.
S.G. Nash, A multigrid approach to discretized optimization problems, Optim. Methods Softw., 14 (2000), pp. 99--116.
22.
S.G. Nash, Properties of a class of multilevel optimization algorithms for equality-constrained problems, Optim. Methods Softw., 29 (2014), pp. 137--159.
23.
S.G. Nash and R.M. Lewis, Assessing the performance of an optimization-based multilevel method, Optim. Methods Softw., 26 (2011), pp. 693--717.
24.
Y. Nesterov, Gradient methods for minimizing composite objective function, Math. Program., 140 (2013), pp. 125--161.
25.
S. Oh, A.B. Milstein, C.A. Bouman, and K.J. Webb, A general framework for nonlinear multigrid inversion, IEEE Trans. Image Process., 14 (2005), pp. 125--140.
26.
N. Parikh and S. Boyd, Proximal algorithms, Found. Trends Optim., 1 (2014), pp. 127--239.
27.
P. Parpas and M. Webster, A stochastic multiscale model for electricity generation capacity expansion, European J. Oper. Res., 232 (2014), pp. 359--374.
28.
E. Polak, R.S. Womersley, and H.X. Yin, An algorithm based on active sets and smoothing for discretized semi-infinite minimax problems, J. Optim. Theory Appl., 138 (2008), pp. 311--328.
29.
P. Richtárik and M. Takáč, Iteration complexity of randomized block-coordinate descent methods for minimizing a composite function, Math. Program., 144 (2014), pp. 1--38.
30.
J.J. Thiagarajan, K.N. Ramamurthy, and A. Spanias, Learning stable multilevel dictionaries for sparse representation of images, IEEE Trans. Neural Netw. Learn. Syst., under review, 2013.
31.
A. Tsoukalas, P. Parpas, and B. Rustem, A smoothing algorithm for finite min--max--min problems, Optim. Lett., 3 (2009), pp. 49--62.
32.
Z. Wen and D. Goldfarb, A line search multigrid method for large-scale nonlinear optimization, SIAM J. Optim., 20 (2009), pp. 1478--1503.
33.
Z. Lei, Q. Du, and Z. Zheng, Optimization-based shrinking dimer method for finding transition states, SIAM J. Sci. Comput., 38 (2016), pp. A528--A544.
Information & Authors
Information
Published In
SIAM Journal on Scientific Computing
Pages: S681 - S701
DOI: 10.1137/16M1082299
ISSN (online): 1095-7197
Copyright
© 2017, Society for Industrial and Applied Mathematics. This work is licensed under a Creative Commons Attribution 4.0 International License
History
Submitted: 1 July 2016
Accepted: 8 May 2017
Published online: 26 October 2017
Keywords
MSC codes
Authors
Metrics & Citations
Metrics
Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited By
- “FISTA” in Banach spaces with adaptive discretisationsComputational Optimization and Applications, Vol. 83, No. 3 | 2 November 2022
- Newton-type multilevel optimization methodOptimization Methods and Software, Vol. 37, No. 1 | 13 December 2019
- First-Order Geometric Multilevel Optimization for Discrete TomographyScale Space and Variational Methods in Computer Vision | 30 April 2021
- Fast Multilevel Algorithms for Compressive Principal Component PursuitSIAM Journal on Imaging Sciences, Vol. 12, No. 1 | 26 March 2019
- An Approach to Multidimensional Nonlinear Optimization2018 International Multi-Conference on Industrial Engineering and Modern Technologies (FarEastCon) | 1 Oct 2018