A Sequential Quadratic Programming Algorithm for Nonsmooth Problems with Upper- \({\boldsymbol{\mathcal{C}^2}}\) Objective
Abstract.
An optimization algorithm for nonsmooth nonconvex constrained optimization problems with upper- \(\mathcal{C}^2\) objective functions is proposed and analyzed. Upper- \(\mathcal{C}^2\) is a weakly concave property that exists in difference of convex (DC) functions and arises naturally in many applications, particularly certain classes of solutions to parametric optimization problems e.g., recourse of stochastic programming and projection onto closed sets. The algorithm can be viewed as an extension of sequential quadratic programming (SQP) to nonsmooth problems with upper- \(\mathcal{C}^2\) objectives or a simplified bundle method. It is globally convergent with bounded algorithm parameters that are updated with a trust-region criterion. The algorithm handles general smooth constraints through linearization and uses a line search to ensure progress. The potential inconsistencies from the linearization of the constraints are addressed through a penalty method. The capabilities of the algorithm are demonstrated by solving both simple upper- \(\mathcal{C}^2\) problems and a real-world optimal power flow problem used in current power grid industry practices.
Keywords
MSC codes
Get full access to this article
View all available purchase options and get full access to this article.
References
1.
L. T. H. An and P. D. Tao, DC programming and DCA: Thirty years of developments, Math. Program., 169 (2018), pp. 5–68.
2.
P. Apkarian, D. Noll, and O. Prot, A trust region spectral bundle method for nonconvex eigenvalue optimization, SIAM J. Optim., 19 (2008), pp. 281–306, https://doi.org/10.1137/060665191.
3.
J. R. Birge and F. Louveaux, Introduction to Stochastic Programming, Springer-Verlag, New York, 1997.
4.
J. F. Bonnans and A. Shapiro, Perturbation Analysis of Optimization Problems, Springer Ser. Oper. Res., Springer-Verlag, New York, 2000.
5.
J. Burke, A sequential quadratic programming method for potentially infeasible mathematical programs, J. Math. Anal., 139 (1989), pp. 319–351.
6.
J. V. Burke, An exact penalization viewpoint of constrained optimization, SIAM J. Control Optim., 29 (1991), pp. 968–998, https://doi.org/10.1137/0329054.
7.
J. V. Burke, A robust trust region method for constrained nonlinear programming problems, SIAM J. Optim., 2 (1992), pp. 325–347, https://doi.org/10.1137/0802016.
8.
J. V. Burke and S. P. Han, A robust sequential quadratic programming method, Math. Program., 43 (1989), pp. 277–303.
9.
J. V. Burke and R. A. Poliquin, Optimality conditions for non-finite valued convex composite functions, Math. Program., 57 (1992), pp. 103–120.
10.
R. H. Byrd, N. I. M. Gould, J. Nocedal, and R. A. Waltz, On the convergence of successive linear-quadratic programming algorithms, SIAM J. Optim., 16 (2005), pp. 471–489, https://doi.org/10.1137/S1052623403426532.
11.
N. Chiang, C. G. Petra, and V. M. Zavala, Structured nonconvex optimization of large-scale energy systems using PIPS-NLP, in Proceedings of the 2014 Power Systems Computation Conference, Warsaw, Poland, 2014, pp. 1–7.
12.
F. Clarke, Optimization and Nonsmooth Analysis, John Wiley & Sons, New York, 1983.
13.
Y. Cui and J.-S. Pang, Modern Nonconvex Nondifferentiable Optimization, MOS-SIAM Ser. Optim. 29, SIAM, Philadelphia, 2021, https://doi.org/10.1137/1.9781611976748.
14.
F. E. Curtis, T. Mitchell, and M. L. Overton, A BFGS-SQP method for nonsmooth, nonconvex, constrained optimization and its evaluation using relative minimization profiles, Optim. Methods Softw., 32 (2017), pp. 148–181.
15.
F. E. Curtis and M. L. Overton, A sequential quadratic programming algorithm for nonconvex, nonsmooth constrained optimization, SIAM J. Optim., 22 (2012), pp. 474–500, https://doi.org/10.1137/090780201.
16.
A. Daniilidis and P. Georgiev, Approximate convexity and submonotonicity, J. Math. Anal., 291 (2004), pp. 292–301.
17.
M. Dao, Bundle method for nonconvex nonsmooth constrained optimization, J. Convex Anal., 22 (2015), pp. 1061–1090.
18.
M. Dao, J. Gwinner, D. Noll, and N. Ovcharova, Nonconvex bundle method with application to a delamination problem, Comput. Optim Appl., 65 (2016), pp. 173–203.
19.
R. Fletcher, Practical Methods of Optimization, John Wiley & Sons, New York, 2013.
20.
W. Hare, C. Sagastizabal, and M. Solodov, A proximal bundle method for nonsmooth nonconvex functions with inexact information, Comput. Optim Appl., 63 (2016), pp. 1–28.
21.
W. Hare and C. Sagastizábal, A redistributed proximal bundle method for nonconvex optimization, SIAM J. Optim., 20 (2010), pp. 2442–2473, https://doi.org/10.1137/090754595.
22.
M. Hong, A distributed, asynchronous, and incremental algorithm for nonconvex optimization: An ADMM approach, IEEE Trans. Control Network Syst., 5 (2018), pp. 935–945.
23.
P. Kall and S. W. Wallace, Stochastic Programming, 2nd ed., John Wiley & Sons, Chichester, 1994.
24.
K. Kiwiel, A linearization algorithm for nonsmooth minimization, Math. Oper. Res., 10 (1985), pp. 185–194.
25.
K. C. Kiwiel, Restricted step and Levenberg–Marquardt techniques in proximal bundle methods for nonconvex nondifferentiable optimization, SIAM J. Optim., 6 (1996), pp. 227–249, https://doi.org/10.1137/0806013.
26.
C. Lemaréchal, Bundle methods in nonsmooth optimization, in Nonsmooth Optimization (Proc. IIASA Workshop, Laxenburg, 1977), Vol. 3, Pergamon, Oxford, 1978, pp. 79–102.
27.
C. Lemaréchal and C. Sagastizabal, Variable metric bundle methods: From conceptual to implementable forms, Math. Program., 76 (1996), pp. 393–410.
28.
J. Liu, Y. Cui, J.-S. Pang, and S. Sen, Two-stage stochastic programming with linearly bi-parameterized quadratic recourse, SIAM J. Optim., 30 (2020), pp. 2530–2558, https://doi.org/10.1137/19M1276819.
29.
R. Mifflin, A modification and an extension of Lemarechal’s algorithm for nonsmooth minimization, in Nondifferential and Variational Techniques in Optimization, Math. Program. Stud. 17, Springer, Berlin, Heidelberg, 1982, pp. 77–90.
30.
B. Mordukhovich, Necessary conditions in nonsmooth minimization via lower and upper subgradients, Set-Valued Anal., 12 (2004), pp. 163–193.
31.
M. M. Mäkelä and P. Neittaanmäki, Nonsmooth Optimization: Analysis and Algorithms with Applications to Optimal Control, World Scientific, River Edge, NJ, 1992.
32.
J. Nocedal and S. J. Wright, Numerical Optimization, 2nd ed., Springer, New York, 2006.
33.
D. Noll, Cutting plane oracles to minimize non-smooth non-convex functions, Set-Valued Var. Anal., 18 (2009), pp. 531–568.
34.
D. Noll, Bundle method for non-convex minimization with inexact subgradients and function values, in Computational and Analytical Mathematics, Springer Proc. Math. Statist. 50, Springer, New York, 2013, pp. 555–592.
35.
C. G. Petra and I. Aravena, Solving realistic security-constrained optimal power flow problems, https://doi.org/10.1287/opre.2021.0526.
36.
C. G. Petra, O. Schenk, and M. Anitescu, Real-time stochastic optimization of complex energy systems on high performance computers, Comput. Sci. Eng., 99 (2014), pp. 1–9.
37.
C. G. Petra, O. Schenk, M. Lubin, and K. Gärtner, An augmented incomplete factorization approach for computing the Schur complement in stochastic optimization, SIAM J. Sci. Comput., 36 (2014), pp. C139–C162, https://doi.org/10.1137/130908737.
38.
W. Qiu, A. J. Flueck, and F. Tu, A parallel algorithm for security constrained optimal power flow with an interior point method, in Proceedings of the IEEE Power Engineering Society General Meeting 2005, Vol. 1, 2005, pp. 447–453.
39.
R. T. Rockafellar and R. J.-B. Wets, Variational Analysis, Grundlehren Math. Wiss. 317, Springer-Verlag, Berlin, 1998.
40.
H. Schramm and J. Zowe, A version of the bundle idea for minimizing a nonsmooth function: Conceptual idea, convergence analysis, numerical results, SIAM J. Optim., 2 (1992), pp. 121–152, https://doi.org/10.1137/0802008.
41.
A. Shapiro, D. Dentcheva, and A. Ruszczyński, Lectures on Stochastic Programming: Modeling and Theory, 2nd ed., MOS-SIAM Ser. Optim. 16, SIAM, Philadelphia, 2014, https://doi.org/10.1137/1.9781611973433.
42.
N. Z. Shor, Minimization Methods for Non-differentiable Functions, 3rd ed., Springer-Verlag, Berlin, 1985.
43.
J. Spingarn, Submonotone subdifferentials of Lipschitz functions, Trans. Amer. Math. Soc., 264 (1981), pp. 77–89.
44.
J. Wang, I. Aravena, and C. G. Petra, An Adaptive Sampling Sequential Quadratic Programming Method for Nonsmooth Stochastic Optimization with Upper- \(\mathcal{C}^2\) Objective, preprint, https://arxiv.org/abs/2304.04380, 2023.
45.
J. Wang, N. Y. Chiang, and C. G. Petra, An asynchronous distributed-memory optimization solver for two-stage stochastic programming problems, in Proceedings of the 20th International Symposium on Parallel and Distributed Computing (ISPDC), IEEE, Piscataway, NJ, 2021, pp. 33–40.
46.
J. Wang and C. G. Petra, A Simplified Nonsmooth Nonconvex Bundle Method with Applications to Security-Constrained ACOPF Problems, preprint, https://arxiv.org/abs/2203.17215, 2022.
47.
Y. Wang, W. Yin, and J. Zeng, Global convergence of ADMM in nonconvex nonsmooth optimization, J. Sci. Comput., 78 (2019), pp. 29–63.
48.
A. Wächter and L. Biegler, On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming, Math. Program., 106 (2006), pp. 25–27.
49.
M. Xu, J. J. Ye, and L. Zhang, Smoothing SQP methods for solving degenerate nonsmooth constrained optimization problems with applications to bilevel programs, SIAM J. Optim., 25 (2015), pp. 1388–1410, https://doi.org/10.1137/140971580.
50.
Y. Yang, L. Pang, X. Ma, and J. Shen, Constrained nonconvex nonsmooth optimization via proximal bundle method, J. Optim. Theory Appl., 163 (2014), pp. 900–925.
51.
P. Yu, T. K. Pong, and Z. Lu, Convergence rate analysis of a sequential convex programming method with line search for a class of constrained difference-of-convex optimization problems, SIAM J. Optim., 31 (2021), pp. 2024–2054, https://doi.org/10.1137/20M1314057.
Information & Authors
Information
Published In
Copyright
© 2023 Society for Industrial and Applied Mathematics.
History
Submitted: 21 April 2022
Accepted: 9 May 2023
Published online: 31 August 2023
Keywords
MSC codes
Authors
Funding Information
Lawrence Livermore National Laboratory (LLNL): DEAC52-07NA27344
Funding: This work was performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under contract DE-AC52-07NA27344. Release number LLNL-JRNL-833508.
Metrics & Citations
Metrics
Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited By
- An improved descent Perry‐type algorithm for large‐scale unconstrained nonconvex problems and applications to image restoration problemsNumerical Linear Algebra with Applications, Vol. 31, No. 6 | 13 July 2024
- Issues on a 2–Dimensional Quadratic Sub–Problem and Its Applications in Nonlinear Programming: Trust–Region Methods (TRMs) and Linesearch Based Methods (LBMs)Algorithms, Vol. 17, No. 12 | 9 December 2024
- Advanced Computing is at the Forefront of a New “Moonshot” Revolutionizing the North American Power GridComputing in Science & Engineering, Vol. 26, No. 2 | 1 Apr 2024
View Options
- Access via your Institution
- Questions about how to access this content? Contact SIAM at [email protected].