A Decomposition Algorithm for Two-Stage Stochastic Programs with Nonconvex Recourse Functions
Abstract.
In this paper, we have studied a decomposition method for solving a class of nonconvex two-stage stochastic programs, where both the objective and constraints of the second-stage problem are nonlinearly parameterized by the first-stage variables. Due to the failure of the Clarke regularity of the resulting nonconvex recourse function, classical decomposition approaches such as Benders decomposition and (augmented) Lagrangian-based algorithms cannot be directly generalized to solve such models. By exploring an implicitly convex-concave structure of the recourse function, we introduce a novel decomposition framework based on the so-called partial Moreau envelope. The algorithm successively generates strongly convex quadratic approximations of the recourse function based on the solutions of the second-stage convex subproblems and adds them to the first-stage master problem. Convergence has been established for both a fixed number of scenarios and a sequential internal sampling strategy. Numerical experiments are conducted to demonstrate the effectiveness of the proposed algorithm.
Keywords
MSC codes
Get full access to this article
View all available purchase options and get full access to this article.
References
1.
Z. Artstein and R. A. Vitale, A strong law of large numbers for random compact sets, Ann. Probab., 3 (1975), pp. 879–882.
2.
E. Asplund, Differentiability of the metric projection in finite-dimensional Euclidean space, Proc. Amer. Math. Soc., 38 (1973), pp. 218–219.
3.
J. F. Benders, Partitioning procedures for solving mixed-variables programming problems, Numer. Math., 4 (1962), pp. 238–252.
4.
J. R. Birge and F. Louveaux, Introduction to Stochastic Programming, 2nd ed., Springer, New York, 2011.
5.
J. Bolte, T. Le, and E. Pauwels, Subgradient Sampling for Nonsmooth Nonconvex Minimization, arXiv:2202.13744, 2022.
6.
I. M. Bomze, M. Gabl, F. Maggioni, and G. C. Pflug, Two-stage stochastic standard quadratic optimization, European J. Oper. Res., 299 (2022), pp. 21–34.
7.
J. F. Bonnans and A. Shapiro, Perturbation Analysis of Optimization Problems, Springer, New York, 2013.
8.
P. Borges, C. Sagastizábal, and M. Solodov, A regularized smoothing method for fully parameterized convex problems with applications to convex and nonconvex two-stage stochastic programming, Math. Program., 189 (2022), pp. 117–149.
9.
R. H. Byrd, J. Nocedal, and R. A. Waltz, Knitro: An integrated package for nonlinear optimization, in Large Scale Nonlinear Optimization, Springer, New York, 2006, pp. 35–59.
10.
B. J. Chun and S. M. Robinson, Scenario analysis via bundle decomposition, Ann. Oper. Res., 56 (1995), pp. 39–63.
11.
F. H. Clarke, Generalized gradients and applications, Trans. Amer. Math. Soc., 205 (1975), pp. 247–262.
12.
F. H. Clarke, Optimization and Nonsmooth Analysis, SIAM, Philadelphia, 1990.
13.
P. L. Combettes, Quasi-Fejérian analysis of some optimization algorithms, in Inherently Parallel Algorithms in Feasibility and Optimization and their Applications, Stud. Comput. Math. 8, Elsevier, Amsterdam, 2001, pp. 115–152.
14.
K. J. Cormican, D. P. Morton, and R. K. Wood, Stochastic network interdiction, Oper. Res., 46 (1998), pp. 184–197.
15.
Y. Cui and J.-S. Pang, Modern Nonconvex Nondifferentiable Optimization, SIAM, Philadelphia, 2021.
16.
D. Davis, D. Drusvyatskiy, S. Kakade, and J. D. Lee, Stochastic subgradient method converges on tame functions, Found. Comput. Math., 20 (2020), pp. 119–154.
17.
Y. M. Ermoliev and V. I. Norkin, Sample average approximation method for compound stochastic optimization problems, SIAM J. Optim., 23 (2013), pp. 2231–2263.
18.
G. B. Folland, Real Analysis: Modern Techniques and Their Applications, Pure Appl. Math. 40, John Wiley & Sons, New York, 1999.
19.
S. Ghadimi and G. Lan, Stochastic first-and zeroth-order methods for nonconvex stochastic programming, SIAM J. Optim., 23 (2013), pp. 2341–2368.
20.
V. Goel and I. E. Grossmann, A stochastic programming approach to planning of offshore gas field developments under uncertainty in reserves, Comput. Chem. Eng., 28 (2004), pp. 1409–1429.
21.
V. Goel and I. E. Grossmann, A Lagrangian duality based branch and bound for solving linear stochastic programs with decision dependent uncertainty, in European Symposium on Computer-Aided Process Engineering, Computer Aided Chemical Engineering 20, Elsevier, Amsterdam, 2005, pp. 55–60.
22.
V. Goel and I. E. Grossmann, A class of stochastic programs with decision dependent uncertainty, Math. Program., 108 (2006), pp. 355–394.
23.
M. Guignard, Lagrangean relaxation, TOP, 11 (2003), pp. 151–200.
24.
V. Guigues, Inexact stochastic mirror descent for two-stage nonlinear stochastic programs, Math. Program., 187 (2021), pp. 533–577.
25.
V. Guigues, R. Monteiro, and B. Svaiter, Inexact cuts in stochastic dual dynamic programming applied to multistage stochastic nondifferentiable problems, SIAM J. Optim., 31 (2021), pp. 2084–2110.
26.
T. Hao and J.-S. Pang, Piecewise affine parameterized value-function based bilevel non-cooperative games, Math. Program., 180 (2020), pp. 33–73.
27.
L. Hellemo, P. I. Barton, and A. Tomasgard, Decision-dependent probabilities in stochastic programs with recourse, Comput. Manag. Sci., 15 (2018), pp. 369–395.
28.
J. L. Higle and S. Sen, Stochastic Decomposition: A Statistical Method for Large Scale Stochastic Linear Programming, Nonconvex Optim. Appl. 8, Springer, New York, 2013.
29.
T. Homem-de Mello, Variable-sample methods for stochastic optimization, ACM Trans. Model. Comput. Simul., 13 (2003), pp. 10–133.
30.
T. Homem-de Mello and G. Bayraksan, Monte Carlo sampling-based methods for stochastic optimization, Surv. Oper. Res. Manage. Sci., 19 (2014), pp. 56–85.
31.
T. W. Jonsbråten, R. J.-B. Wets, and D. L. Woodruff, A class of stochastic programs with decision dependent random elements, Ann. Oper. Res., 82 (1998), pp. 83–106.
32.
J. Liu, Y. Cui, J.-S. Pang, and S. Sen, Two-stage stochastic programming with linearly bi-parameterized quadratic recourse, SIAM J. Optim., 30 (2020), pp. 2530–2558.
33.
F. V. Louveaux and Y. Smeers, Optimal investments for electricity generation: A stochastic model and a test problem, in Numerical Techniques for Stochastic Optimization Problems, Y. Ermoliev and R. J.-B. Wets, eds., Springer-Verlag, Berlin, 1988, pp. 445–452.
34.
Y. Lucet, Fast Moreau envelope computation I: Numerical algorithms, Numer. Algorithms, 43 (2006), pp. 235–249.
35.
B. S. Mordukhovich, Variational Analysis and Generalized Differentiation I: Basic Theory, Grundlehren Math. Wiss. 330, Springer, Berlin, 2006.
36.
A. Nemirovski, A. Juditsky, G. Lan, and A. Shapiro, Robust stochastic approximation approach to stochastic programming, SIAM J. Optim., 19 (2009), pp. 1574–1609.
37.
O. Nohadani and K. Sharma, Optimization under decision-dependent uncertainty, SIAM J. Optim., 28 (2018), pp. 1773–1795.
38.
R. Pasupathy and Y. Song, Adaptive sequential sample average approximation for solving two-stage stochastic linear programs, SIAM J. Optim., 31 (2021), pp. 1017–1048.
39.
C. Planiden and X. Wang, Proximal mappings and Moreau envelopes of single-variable convex piecewise cubic functions and multivariable gauge functions, in Nonsmooth Optimization and Its Applications, Springer, New York, 2019, pp. 89–130.
40.
B. T. Polyak, New stochastic approximation type procedures, Automat. i Telemekh., 7 (1990).
41.
D. P. Bertsekas, Control of Uncertain Systems with a Set-Membership Description of the Uncertainty, Ph.D. thesis, Massachusetts Institute of Technology, 1971.
42.
J. M. Danskin, The Theory of Max-Min and Its Application to Weapons Allocation Problems, Economics and Operations Research 5, Springer, Berlin, 1967.
43.
H. Robbins and S. Monro, A stochastic approximation method, Ann. Math. Statist., 22 (1951), pp. 400–407.
44.
H. Robbins and D. Siegmund, A convergence theorem for nonnegative almost supermartingales and some applications, in Optimizing Methods in Statistics, Academic Press, New York, 1971, pp. 233–257.
45.
R. T. Rockafellar, Convex Analysis, Princeton University Press, Princeton, NJ, 1970.
46.
R. T. Rockafellar, Progressive decoupling of linkages in optimization and variational inequalities with elicitable convexity or monotonicity, Set-Valued Var. Anal., 27 (2019), pp. 863–893.
47.
R. T. Rockafellar, Augmented Lagrangians and hidden convexity in sufficient conditions for local optimality, Math. Program., 198 (2022), pp. 159–194.
48.
R. T. Rockafellar and R. J.-B. Wets, Scenarios and policy aggregation in optimization under uncertainty, Math. Oper. Res., 16 (1991), pp. 119–147.
49.
R. T. Rockafellar and R. J.-B. Wets, Variational Analysis, Grundlehren Math. Wiss. 317, Springer, Berlin, 2009.
50.
J. Royset and E. Polak, Extensions of stochastic optimization results to problems with system failure probability functions, J. Optim. Theory Appl., 133 (2007), pp. 1–18.
51.
A. Ruszczyński, Decomposition methods in stochastic programming, Math. Program., 79 (1997), pp. 333–353.
52.
A. Ruszczyński, Convergence of a stochastic subgradient method with averaging for nonsmooth nonconvex constrained optimization, Optim. Lett., 14 (2020), pp. 1615–1625.
53.
A. Shapiro, Monte Carlo sampling methods, in Stochastic Programming, Handbooks in Operations Research and Management Science 10, A. Ruszczyński and A. Shapiro, eds., Elsevier, Amsterdam, 2003, pp. 353–425.
54.
A. Shapiro, D. Dentcheva, and A. Ruszczyński, Lectures on Stochastic Programming: Modeling and Theory, 3rd ed., SIAM, Philadelphia, 2021.
55.
A. Shapiro and H. Xu, Uniform laws of large numbers for set-valued mappings and subdifferentials of random functions, J. Math. Anal. Appl., 325 (2007), pp. 1390–1399.
56.
B. Tarhan, I. E. Grossmann, and V. Goel, Stochastic programming approach for the planning of offshore oil or gas field infrastructure under decision-dependent uncertainty, Ind. Eng. Chem. Res., 48 (2009), pp. 3078–3097.
57.
R. M. Van Slyke and R. J.-B, Wets, L-shaped linear programs with applications to optimal control and stochastic programming, SIAM J. Appl. Math., 17 (1969), pp. 638–663.
58.
A. Wächter and L. T. Biegler, On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming, Math. Program., 106 (2006), pp. 25–57.
59.
R. J.-B. Wets, Large scale linear programming techniques in stochastic programming, in Numerical Methods in Stochastic Programming, Yu. Ermoliev and R. J.-B. Wets, eds., Springer, Berlin, 1988, pp. 65–94.
Information & Authors
Information
Published In
Copyright
© 2024 Society for Industrial and Applied Mathematics.
History
Submitted: 4 April 2022
Accepted: 27 June 2023
Published online: 19 January 2024
Keywords
MSC codes
Authors
Funding Information
National Science Foundation (NSF): CCF-2153352, DMS-2309729
Funding: The authors are partially supported by the National Science Foundation under grants CCF-2153352 and DMS-2309729.
Metrics & Citations
Metrics
Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited By
- Variational Analysis of a Nonconvex and Nonsmooth Optimization Problem: An IntroductionSet-Valued and Variational Analysis, Vol. 33, No. 2 | 15 May 2025
- MUSE-BB: a decomposition algorithm for nonconvex two-stage problems using strong multisection branchingJournal of Global Optimization, Vol. 161 | 5 May 2025
- A Progressive Decoupling Algorithm for Minimizing the Difference of Convex and Weakly Convex FunctionsJournal of Optimization Theory and Applications, Vol. 204, No. 3 | 23 January 2025
- Variational Theory and Algorithms for a Class of Asymptotically Approachable Nonconvex ProblemsMathematics of Operations Research, Vol. 40 | 16 January 2025
- Meal pickup and delivery problem with appointment time and uncertainty in order cancellationTransportation Research Part E: Logistics and Transportation Review, Vol. 193 | 1 Jan 2025
- An Approximate-and-Optimize Method for Security-Constrained AC Optimal Power FlowLearning and Intelligent Optimization | 3 January 2025
- Towards Global Solutions for Nonconvex Two-Stage Stochastic Programs: A Polynomial Lower Approximation ApproachSIAM Journal on Optimization, Vol. 34, No. 4 | 10 October 2024
View Options
- Access via your Institution
- Questions about how to access this content? Contact SIAM at [email protected].