Weakly Chained Matrices, Policy Iteration, and Impulse Control
Abstract
This work is motivated by numerical solutions to Hamilton--Jacobi--Bellman quasi-variational inequalities (HJBQVIs) associated with combined stochastic and impulse control problems. In particular, we consider (i) direct control, (ii) penalized, and (iii) semi-Lagrangian discretization schemes applied to the HJBQVI problem. Scheme (i) takes the form of a Bellman problem involving an operator which is not necessarily contractive. We consider the well-posedness of the Bellman problem and give sufficient conditions for convergence of the corresponding policy iteration. To do so, we use weakly chained diagonally dominant matrices, which give a graph-theoretic characterization of weakly diagonally dominant M-matrices. We compare schemes (i)--(iii) under the following examples: (a) optimal control of the exchange rate, (b) optimal consumption with fixed and proportional transaction costs, and (c) pricing guaranteed minimum withdrawal benefits in variable annuities. We find that one should abstain from using scheme (i). (An erratum is attached.)
Keywords
MSC codes
Get full access to this article
View all available purchase options and get full access to this article.
References
1.
P. Azimzadeh and P. A. Forsyth, The existence of optimal bang-bang controls for GMxB contracts, SIAM J. Financial Math., 6 (2015), pp. 117--139.
2.
J. Babbin, P. A. Forsyth, and G. Labahn, A comparison of iterated optimal stopping and local policy iteration for American options under regime switching, J. Sci. Comput., 58 (2014), pp. 409--430.
3.
G. Barles and P. E. Souganidis, Convergence of approximation schemes for fully nonlinear second order equations, Asymptot. Anal., 4 (1991), pp. 271--283.
4.
E. Bayraktar and H. Xing, Pricing Asian options for jump diffusion, Math. Finance, 21 (2011), pp. 117--143.
5.
A. Bensoussan and J. L. Lions, Impulse Control and Quasi-Variational Inequalities, Gauthier-Villars, Paris, 1984.
6.
O. Bokanowski, S. Maroso, and H. Zidani, Some convergence results for Howard's algorithm, SIAM J. Numer. Anal., 47 (2009), pp. 3001--3026.
7.
J. H. Bramble and B. E. Hubbard, On a finite difference analogue of an elliptic boundary problem which is neither diagonally dominant nor of non-negative type, J. Math. Phys., 43 (1964), p. 117.
8.
A. Cadenillas and F. Zapatero, Optimal central bank intervention in the foreign exchange market, J. Econom. Theory, 87 (1999), pp. 218--242.
9.
J. P. Chancelier, M. Messaoud, and A. Sulem, A policy iteration algorithm for fixed point problems with nonexpansive operators, Math. Methods Oper. Res., 65 (2007), pp. 239--259.
10.
J. P. Chancelier, B. Øksendal, and A. Sulem, Combined stochastic control and optimal stopping, and application to numerical approximation of combined stochastic and impulse control, Proc. Steklov Inst. Math., 237 (2002), pp. 149--172.
11.
Z. Chen and P. A. Forsyth, A numerical scheme for the impulse control formulation for pricing variable annuities with a guaranteed minimum withdrawal benefit (GMWB), Numer. Math., 109 (2008), pp. 535--569.
12.
R. Cont and E. Voltchkova, A finite difference scheme for option pricing in jump diffusion and exponential Lévy models, SIAM J. Numer. Anal., 43 (2005), pp. 1596--1626.
13.
M. Dai, Y. K. Kwok, and J. Zong, Guaranteed minimum withdrawal benefit in variable annuities, Math. Finance, 18 (2008), pp. 595--611.
14.
P. A. Forsyth and G. Labahn, Numerical methods for controlled Hamilton-Jacobi-Bellman PDEs in finance, J. Comput. Finance, 11 (2007).
15.
Y. Huang, P. A. Forsyth, and G. Labahn, Inexact arithmetic considerations for direct control and penalty methods: American options under jump diffusion, Appl. Numer. Math., 72 (2013), pp. 33--51.
16.
K. Ishii, Viscosity solutions of nonlinear second order elliptic PDEs associated with impulse control problems, Funkcial. Ekvac., 36 (1993), pp. 123--141.
17.
I. Kharroubi, J. Ma, H. Pham, J. Zhang, et al., Backward SDEs with constrained jumps and quasi-variational inequalities, Ann. Appl. Probab., 38 (2010), pp. 794--840.
18.
H. J. Kushner and P. G. Dupuis, Numerical Methods for Stochastic Control Problems in Continuous Time, Springer-Verlag, New York, 1992.
19.
H. Le and C. Wang, A finite time horizon optimal stopping problem with regime switching, SIAM J. Control Optim., 48 (2010), pp. 5193--5213.
20.
M. A. Milevsky and T. S. Salisbury, Financial valuation of guaranteed minimum withdrawal benefits, Insurance Math. Econom., 38 (2006), pp. 21--38.
21.
G. Mundaca and B. Øksendal, Optimal stochastic intervention control with application to the exchange rate, J. Math. Econom., 29 (1998), pp. 225--243.
22.
A. M. Oberman, Convergent difference schemes for degenerate elliptic and parabolic equations: Hamilton-Jacobi equations and free boundary problems, SIAM J. Numer. Anal., 44 (2006), pp. 879--895.
23.
B. Øksendal and A. Sulem, Applied Stochastic Control of Jump Diffusions, Probab. Theory Stochastic Processes 498, Springer, New York, 2005.
24.
R. J. Plemmons, M-matrix characterizations. I---Nonsingular M-matrices, Linear Algebra Appl., 18 (1977), pp. 175--188.
25.
R. C. Seydel, Existence and uniqueness of viscosity solutions for QVI associated with impulse control of jump-diffusions, Stochastic Process. Appl., 119 (2009), pp. 3719--3748.
26.
P. N. Shivakumar and K. H. Chew, A sufficient condition for nonvanishing of determinants, Proc. Amer. Math. Soc., 43 (1974), pp. 63--66.
27.
J. H. Witte and C. Reisinger, Penalty methods for the solution of discrete HJB equations: Continuous control and obstacle problems, SIAM J. Numer. Anal., 50 (2012), pp. 595--625.
Information & Authors
Information
Published In
Copyright
© 2016, Society for Industrial and Applied Mathematics.
History
Submitted: 12 October 2015
Accepted: 18 March 2016
Published online: 5 May 2016
Keywords
MSC codes
Authors
Funding Information
Natural Sciences and Engineering Research Council of Canadahttp://dx.doi.org/10.13039/501100000038: 36828
Metrics & Citations
Metrics
Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited By
- Monetizing Positive Externalities to Mitigate the Infrastructure Underinvestment ProblemOperations Research, Vol. 73, No. 2 | 1 Mar 2025
- Parameter Estimation and Identifiability in Kinetic Flux Profiling Models of MetabolismBulletin of Mathematical Biology, Vol. 87, No. 1 | 27 November 2024
- Renewable energy investment under stochastic interest rate with regime-switching volatilityEnergy Economics, Vol. 136 | 1 Aug 2024
- Markov Decision Processes with Observation Costs: Framework and Computation with a Penalty SchemeMathematics of Operations Research, Vol. 1 | 23 May 2024
- Impasse Surface of Differential–Algebraic Power System Models: An Interpretation Based on Admittance MatricesIEEE Transactions on Automatic Control, Vol. 68, No. 10 | 1 Oct 2023
- Feedback Nash Equilibria in Differential Games With Impulse ControlIEEE Transactions on Automatic Control, Vol. 68, No. 8 | 1 Aug 2023
- Predicting DNA kinetics with a truncated continuous-time Markov chain methodComputational Biology and Chemistry, Vol. 104 | 1 Jun 2023
- Managing Recession RiskSSRN Electronic Journal, Vol. 36 | 1 January 2023
- The cover time of a (multiple) Markov chain with rational transition probabilities is rationalStatistics & Probability Letters, Vol. 187 | 1 Aug 2022
- Optimal investment for retail investorsMathematical Finance, Vol. 32, No. 2 | 14 August 2021
- The power of two choices for random walksCombinatorics, Probability and Computing, Vol. 31, No. 1 | 28 May 2021
- Discrete analysis of Schwarz waveform relaxation for a diffusion reaction problem with discontinuous coefficientsThe SMAI Journal of computational mathematics, Vol. 8 | 8 April 2022
- A Fixed-Point Policy-Iteration-Type Algorithm for Symmetric Nonzero-Sum Stochastic Impulse Control GamesApplied Mathematics & Optimization, Vol. 84, No. 2 | 24 June 2020
- Multigrid Methods for Image Registration Model Based on Optimal Mass TransportMathematical Methods in Image Processing and Inverse Problems | 26 September 2021
- Scalable load balancing in the presence of heterogeneous serversPerformance Evaluation, Vol. 145 | 1 Jan 2021
- The Cash-Cap Model: A Two-State Model of Firm DynamicsSSRN Electronic Journal, Vol. 84 | 1 Jan 2021
- Finite Difference Methods for the Hamilton–Jacobi–Bellman Equations Arising in Regime Switching Utility MaximizationJournal of Scientific Computing, Vol. 85, No. 3 | 17 November 2020
- Nonzero-Sum Stochastic Differential Games with Impulse Controls: A Verification Theorem with ApplicationsMathematics of Operations Research, Vol. 45, No. 1 | 1 Feb 2020
- Error Estimates of Penalty Schemes for Quasi-Variational Inequalities Arising from Impulse Control ProblemsSIAM Journal on Control and Optimization, Vol. 58, No. 1 | 21 January 2020
- Multigrid methods for convergent mixed finite difference scheme for Monge–Ampère equationComputing and Visualization in Science, Vol. 22, No. 1-4 | 5 October 2017
- Stability Analysis of DC-Microgrids: A Gradient FormulationJournal of Control, Automation and Electrical Systems, Vol. 30, No. 6 | 3 October 2019
- A Zero-Sum Stochastic Differential Game with Impulses, Precommitment, and Unrestricted Cost FunctionsApplied Mathematics & Optimization, Vol. 79, No. 2 | 29 August 2017
- High Order Bellman Equations and Weakly Chained Diagonally Dominant TensorsSIAM Journal on Matrix Analysis and Applications, Vol. 40, No. 1 | 19 February 2019
- A Penalty Scheme for Monotone Systems with Interconnected Obstacles: Convergence and Error EstimatesSIAM Journal on Numerical Analysis, Vol. 57, No. 4 | 9 July 2019
- Utility maximisation in a factor model with constant and proportional transaction costsFinance and Stochastics, Vol. 23, No. 1 | 19 December 2018
- A Fixed-Point Policy-Iteration-Type Algorithm for Symmetric Nonzero-Sum Stochastic Impulse Control GamesSSRN Electronic Journal, Vol. 56 | 1 Jan 2019
- Monotone Mixed Finite Difference Scheme for Monge–Ampère EquationJournal of Scientific Computing, Vol. 76, No. 3 | 3 March 2018
- Enzyme sequestration by the substrate: An analysis in the deterministic and stochastic domainsPLOS Computational Biology, Vol. 14, No. 5 | 17 May 2018
- Convergence of Implicit Schemes for Hamilton--Jacobi--Bellman Quasi-Variational InequalitiesSIAM Journal on Control and Optimization, Vol. 56, No. 6 | 6 November 2018
- Collocation Method for Linear BVPs via B-spline Based Fuzzy TransformInformation Processing and Management of Uncertainty in Knowledge-Based Systems. Theory and Foundations | 18 May 2018
- A Primer on Portfolio Choice with Small Transaction CostsAnnual Review of Financial Economics, Vol. 9, No. 1 | 1 Nov 2017
- A Primer on Portfolio Choice with Small Transaction CostsSSRN Electronic Journal, Vol. 19 | 1 Jan 2016
View Options
- Access via your Institution
- Questions about how to access this content? Contact SIAM at [email protected].