An Inexact Bundle Algorithm for Nonconvex Nonsmooth Minimization in Hilbert Space
Abstract
In this paper a bundle method for nonconvex nonsmooth optimization in infinite-dimensional Hilbert spaces is developed and analyzed. The algorithm requires only inexact function value and subgradient information. Global convergence to approximately stationary points is proved, where the final accuracy depends on the error level in the function and subgradient data. The method is then applied to an optimal control problem governed by the obstacle problem. For adaptively controlling the inexactness, implementable conditions are developed, first on a general level and then for the concrete case of a FEM discretization for optimal control of an obstacle problem. Numerical results are presented.
Keywords
MSC codes
Get full access to this article
View all available purchase options and get full access to this article.
References
1.
P. Apkarian, D. Noll, and O. Prot, A proximity control algorithm to minimize nonsmooth and nonconvex semi-infinite maximum eigenvalue functions, J. Convex Anal., 16 (2009), pp. 641--666.
2.
P. Apkarian, D. Noll, and L. Ravanbod, Nonsmooth bundle trust-region algorithm with applications to robust stability, Set-Valued Var. Anal., 24 (2016), pp. 115--148.
3.
P. Apkarian, D. Noll, and L. Ravanbod, Non-smooth optimization for robust control of infinite-dimensional systems, Set-Valued Var. Anal., 26 (2018), pp. 405--429, https://doi.org/10.1007/s11228-017-0453-4.
4.
H. H. Bauschke and P. L. Combettes, Convex Analysis and Monotone Operator Theory in Hilbert Spaces, CMS Books Math./Ouvrages Math. SMC, Springer, New York, 2011, https://doi.org/10.1007/978-3-319-48311-5.
5.
S. Brenner and R. Scott, The Mathematical Theory of Finite Element Methods, Texts Appl. Math. 15, Springer, New York, 2007, https://doi.org/10.1007/978-0-387-75934-0.
6.
G. Choquet, Topology, Academic Press, New York, 1966.
7.
F. H. Clarke, Optimization and Nonsmooth Analysis, Classics Appl. Math., SIAM, Philadelphia, 1998, https://doi.org/10.1137/1.9781611971309.
8.
R. Correa and C. Lemaréchal, Convergence of some algorithms for convex minimization, Math. Program., 62 (1993), pp. 261--275, https://doi.org/10.1007/BF01585170.
9.
A. Daniilidis and P. Georgiev, Approximate convexity and submonotonicity, J. Math. Anal. Appl., 291 (2004), pp. 292--301, https://doi.org/10.1016/j.jmaa.2003.11.004.
10.
W. de Oliveira, C. Sagastizábal, and C. Lemaréchal, Convex proximal bundle methods in depth: A unified analysis for inexact oracles, Math. Program., 148 (2014), pp. 241--277, https://doi.org/10.1007/s10107-014-0809-6.
11.
R. Falk, Error estimates for the approximation of a class of variational inequalities, Math. Comp., 28 (1974), pp. 963--971.
12.
N. Haarala, K. Miettinen, and M. M. Mäkelä, Globally convergent limited memory bundle method for large-scale nonsmooth optimization, Math. Program., 109 (2007), pp. 181--205, https://doi.org/10.1007/s10107-006-0728-2.
13.
W. Hare and C. Sagastizábal, A redistributed proximal bundle method for nonconvex optimization, SIAM J. Optim., 20 (2010), pp. 2442--2473, https://doi.org/10.1137/090754595.
14.
W. Hare, C. Sagastizábal, and M. Solodov, A proximal bundle method for nonsmooth nonconvex functions with inexact information, Comput. Optim. Appl., 63 (2016), pp. 1--28, https://doi.org/10.1007/s10589-015-9762-4.
15.
M. Hintermüller, A proximal bundle method based on approximate subgradients, Comput. Optim. Appl., 20 (2001), pp. 245--266, https://doi.org/10.1023/A:1011259017643.
16.
N. Karmitsa and M. M. Mäkelä, Limited memory bundle method for large bound constrained nonsmooth optimization: Convergence analysis, Optim. Methods Softw., 25 (2010), pp. 895--916, https://doi.org/10.1080/10556780902842495.
17.
D. Kinderlehrer and G. Stampacchia, An Introduction to Variational Inequalities and Their Applications, Classics Appl. Math., SIAM, Philadelphia, 2000, https://doi.org/10.1137/1.9780898719451.
18.
K. Kiwiel, Methods of Descent for Nondifferentiable Optimization, Lecture Notes in Math., Springer, New York, 1985, https://doi.org/10.1007/BFB0074500.
19.
K. Kiwiel, A dual method for certain positive semidefinite quadratic programming problems, SIAM J. Sci. and Stat. Comput., 10 (1989), pp. 175--186, https://doi.org/10.1137/0910013.
20.
K. C. Kiwiel, A linearization algorithm for nonsmooth minimization, Math. Oper. Res., 10 (1985), pp. 185--194, https://doi.org/10.1287/moor.10.2.185.
21.
K. C. Kiwiel, Approximations in proximal bundle methods and decomposition of convex programs, J. Optim. Theory Appl., 84 (1995), pp. 529--548, https://doi.org/10.1007/BF02191984.
22.
K. C. Kiwiel, Restricted step and Levenberg--Marquardt techniques in proximal bundle methods for nonconvex nondifferentiable optimization, SIAM J. Optim., 6 (1996), pp. 227--249, https://doi.org/10.1137/0806013.
23.
K. C. Kiwiel, A proximal bundle method with approximate subgradient linearizations, SIAM J. Optim., 16 (2006), pp. 1007--1023, https://doi.org/10.1137/040603929.
24.
C. Lemaréchal, J.-J. Strodiot, and A. Bihain, On a bundle algorithm for nonsmooth optimization, in Nonlinear Programming 4, Academic Press, New York, 1981, pp. 245--282, https://doi.org/10.1016/B978-0-12-468662-5.50015-X.
25.
L. Lukšan and J. Vlček, A bundle-Newton method for nonsmooth unconstrained minimization, Math. Program., 83 (1998), pp. 373--391, https://doi.org/10.1016/S0025-5610(97)00108-1.
26.
M. M. Mäkelä and P. Neittaanmäki, Nonsmooth Optimization: Analysis and Algorithms with Applications to Optimal Control, World Scientific, River Edge, NJ, 1992, https://doi.org/10.1142/1493.
27.
C. Meyer and O. Thoma, A priori finite element error analysis for optimal control of the obstacle problem, SIAM J. Numer. Anal., 51 (2013), pp. 605--628, https://doi.org/10.1137/110836092.
28.
R. Mifflin, Semismooth and semiconvex functions in constrained optimization, SIAM J. Control Optim., 15 (1977), pp. 959--972, https://doi.org/10.1137/0315061.
29.
R. Mifflin, Convergence of a modification of Lemaréchal's algorithm for nonsmooth optimization, in Progress in Nondifferentiable Optimization, IIASA Collaborative Proc. Ser. CP-828, International Institute for Applied Systems Analysis, Laxenburg, 1982, pp. 85--95.
30.
H. V. Ngai, D. T. Luc, and M. Théra, Approximate convex functions, J. Nonlinear Convex Anal., 1 (2000), pp. 155--176.
31.
D. Noll, Bundle Method for Non-Convex Minimization with Inexact Subgradients and Function Values, in Computational and Analytical Mathematics, D. Bailey, H. Bauschke, et al., eds., Springer, New York, 2013, pp. 555--592, https://doi.org/10.1007/978-1-4614-7621-4_26.
32.
D. Noll, O. Prot, and A. Rondepierre, A proximity control algorithm to minimize nonsmooth and nonconvex functions, Pac. J. Optim., 4 (2008), pp. 571--604.
33.
A.-T. Rauls and S. Ulbrich, Subgradient Computation for the Solution Operator of the Obstacle Problem, Technical report, preprint SPP 1962-056, 2018, https://spp1962.wias-berlin.de/preprints/056.pdf.
34.
M. V. Solodov, On approximations with finite precision in bundle methods for nonsmooth optimization, J. Optim. Theory Appl., 119 (2003), pp. 151--165, https://doi.org/10.1023/B:JOTA.0000005046.70410.02.
35.
W. van Ackooij, J. Y. Bello Cruz, and W. de Oliveira, A strongly convergent proximal bundle method for convex minimization in Hilbert spaces, Optimization, 65 (2016), pp. 145--167, https://doi.org/10.1080/02331934.2015.1004549.
36.
E. Zeidler, Nonlinear Functional Analysis and its Applications II/A: Linear Monotone Operators, Springer, New York, 1990, https://doi.org/10.1007/978-1-4612-0985-0.
Information & Authors
Information
Published In
SIAM Journal on Control and Optimization
Pages: 3137 - 3165
DOI: 10.1137/18M1221849
ISSN (online): 1095-7138
Copyright
© 2019, Society for Industrial and Applied Mathematics.
History
Submitted: 22 October 2018
Accepted: 3 July 2019
Published online: 17 September 2019
Keywords
MSC codes
Authors
Funding Information
Deutsche Forschungsgemeinschaft https://doi.org/10.13039/501100001659 : 1962
Metrics & Citations
Metrics
Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited By
- Semismoothness for Solution Operators of Obstacle-Type Variational Inequalities with Applications in Optimal ControlSIAM Journal on Control and Optimization, Vol. 61, No. 3 | 31 May 2023
- An Approximation Scheme for Distributionally Robust PDE-Constrained OptimizationSIAM Journal on Control and Optimization, Vol. 60, No. 3 | 26 May 2022
- On the Characterization of Generalized Derivatives for the Solution Operator of the Bilateral Obstacle ProblemSIAM Journal on Control and Optimization, Vol. 59, No. 5 | 12 October 2021
- A Nonsmooth Trust-Region Method for Locally Lipschitz Functions with Application to Optimization Problems Constrained by Variational InequalitiesSIAM Journal on Optimization, Vol. 30, No. 3 | 12 August 2020
- Computation of a Bouligand Generalized Derivative for the Solution Operator of the Obstacle ProblemSIAM Journal on Control and Optimization, Vol. 57, No. 5 | 26 September 2019