Abstract

The stochastic dual dynamic programming (SDDP) algorithm has become one of the main tools used to address convex multistage stochastic optimal control problems. Recently a large amount of work has been devoted to improving the convergence speed of the algorithm through cut selection and regularization, and to extending the field of applications to nonlinear, integer, or risk-averse problems. However, one of the main downsides of the algorithm remains the difficulty in giving an upper bound of the optimal value, usually estimated through Monte Carlo methods and therefore difficult to use in the stopping criterion of the algorithm. In this paper we present a dual SDDP algorithm that yields a converging exact upper bound for the optimal value of the optimization problem. As an easy consequence of our approach, we show how to compute an alternative control policy based on an inner approximation of Bellman value functions instead of the outer approximation given by the standard SDDP algorithm. We illustrate the approach on an energy production problem involving zones of production and transportation links between the zones. The numerical experiments we carry out on this example show the effectiveness of the method.

Keywords

  1. stochastic programming
  2. dynamic programming
  3. SDDP
  4. Fenchel conjugacy

MSC codes

  1. 90C15
  2. 90C39
  3. 49N15

Get full access to this article

View all available purchase options and get full access to this article.

References

1.
R. Baucke, A. Downward, and G. Zakeri, A deterministic algorithm for solving multistage stochastic programming problems, Optimization Online, 2017.
2.
H. H. Bauschke and P. L. Combettes, Convex Analysis and Monotone Operator Theory in Hilbert Spaces, 2nd ed., Springer, 2017.
3.
R. Bellman, Dynamic Programming, Princeton University Press, NJ, 1957.
4.
D. P. Bertsekas, Dynamic Programming and Optimal Control. Vol. II, 3rd ed., Athena Scientific, 2007.
5.
A. R. Bo Lincoln, Relaxing dynamic programming, IEEE Trans. Automat. Control, 51 (2006), pp. 1249--1260.
6.
J. Borwein and A. S. Lewis, Convex Analysis and Nonlinear Optimization: Theory and Examples, Springer, 2010.
7.
V. L. de Matos, A. B. Philpott, and E. C. Finardi, Improving the performance of stochastic dual dynamic programming, J. Comput. Appl. Math., 290 (2015), pp. 196--208.
8.
I. Dunning, J. Huchette, and M. Lubin, JuMP: A modeling language for mathematical optimization, SIAM Rev., 59 (2017), pp. 295--320, https://doi.org/10.1137/15M1020575.
9.
A. Georghiou, A. Tsoukalas, and W. Wiesemann, Robust dual dynamic programming, Oper. Res., 67 (2019), pp. 813--830.
10.
P. Girardeau, V. Leclére, and A. B. Philpott, On the convergence of decomposition methods for multistage stochastic convex programs, Math. Oper. Res., 40 (2014), pp. 130--145.
11.
V. Guigues, Convergence analysis of sampling-based decomposition methods for risk-averse multistage stochastic convex programs, SIAM J. Optim., 26 (2016), pp. 2468--2494, https://doi.org/10.1137/140983136.
12.
V. Guigues, Dual dynamic programming with cut selection: Convergence proof and numerical experiments, European J. Oper. Res., 258 (2017), pp. 47--57.
13.
V. Guigues, Inexact cuts in stochastic dual dynamic programming, SIAM J. Optim., 30 (2020), pp. 407--438, https://doi.org/10.1137/18M1211799.
14.
T. Homem-de Mello, V. L. De Matos, and E. C. Finardi, Sampling strategies and stopping criteria for stochastic dual dynamic programming: A case study in long-term hydrothermal scheduling, Energy Syst., 2 (2011), pp. 1--31.
15.
P. Mahey, J. Koko, and A. Lenoir, Decomposition methods for a spatial model for long-term energy pricing problem, Math. Methods Oper. Res., 85 (2017), pp. 137--153.
16.
M. V. Pereira and L. M. Pinto, Multi-stage stochastic optimization applied to energy planning, Math. Program., 52 (1991), pp. 359--375.
17.
A. Philpott, V. de Matos, and E. Finardi, On solving multistage stochastic programs with coherent risk measures, Oper. Res., 61 (2013), pp. 957--970.
18.
A. Philpott and Z. Guan, On the convergence of stochastic dual dynamic programming and related methods, Oper. Res. Lett., 36 (2008), pp. 450--455.
19.
R. T. Rockafellar, Convex Analysis, Princeton University Press, 1970.
20.
R. T. Rockafellar and R. J.-B. Wets, Stochastic convex programming: Relatively complete recourse and induced feasibility, SIAM J. Control Optim., 14 (1976), pp. 574--589, https://doi.org/10.1137/0314038.
21.
A. Shapiro, Analysis of stochastic dual dynamic programming method, European J. Oper. Research, 209 (2011), pp. 63--72.
22.
A. Shapiro, D. Dentcheva, and A. Ruszczyński, Lectures on Stochastic Programming: Modeling and Theory, SIAM, Philadelphia, 2009.
23.
A. Shapiro, W. Tekaya, J. P. da Costa, and M. P. Soares, Final Report for Technical Cooperation between Georgia Institute of Technology and ONS--Operador Nacional do Sistema Elétrico, Georgia Tech. ISyE Report, 2012.
24.
W. Van Ackooij, W. de Oliveira, and Y. Song, On regularization with normal solutions in decomposition methods for multistage stochastic programming, Optimization Online, 2017.

Information & Authors

Information

Published In

cover image SIAM Journal on Optimization
SIAM Journal on Optimization
Pages: 1223 - 1250
ISSN (online): 1095-7189

History

Submitted: 29 April 2019
Accepted: 24 February 2020
Published online: 28 April 2020

Keywords

  1. stochastic programming
  2. dynamic programming
  3. SDDP
  4. Fenchel conjugacy

MSC codes

  1. 90C15
  2. 90C39
  3. 49N15

Authors

Affiliations

Funding Information

Êlectricité de France https://doi.org/10.13039/501100006289
Fondation Mathématique Jacques Hadamard https://doi.org/10.13039/501100007493

Metrics & Citations

Metrics

Citations

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

Cited By

View Options

View options

PDF

View PDF

Media

Figures

Other

Tables

Share

Share

Copy the content Link

Share with email

Email a colleague

Share on social media

The SIAM Publications Library now uses SIAM Single Sign-On for individuals. If you do not have existing SIAM credentials, create your SIAM account https://my.siam.org.