High Order Bellman Equations and Weakly Chained Diagonally Dominant Tensors
Abstract
We introduce high order Bellman equations, extending classical Bellman equations to the tensor setting. We introduce weakly chained diagonally dominant (w.c.d.d.) tensors and show that a sufficient condition for the existence and uniqueness of a positive solution to a high order Bellman equation is that the tensors appearing in the equation are w.c.d.d. M-tensors. In this case, we give a policy iteration algorithm to compute this solution. We also prove that a weakly diagonally dominant Z-tensor with nonnegative diagonals is a strong M-tensor if and only if it is w.c.d.d. This last point is analogous to a corresponding result in the matrix setting and tightens a result from [L. Zhang, L. Qi, and G. Zhou, SIAM J. Matrix Anal. Appl., 35 (2014), pp. 437--452]. We apply our results to obtain a provably convergent numerical scheme for an optimal control problem using an “optimize then discretize” approach which outperforms (in both computation time and accuracy) a classical “discretize then optimize” approach. To the best of our knowledge, a link between M-tensors and optimal control has not been previously established.
Keywords
MSC codes
Get full access to this article
View all available purchase options and get full access to this article.
References
1.
P. Azimzadeh, A fast and stable test to check if a weakly diagonally dominant matrix is a nonsingular M-matrix, Math. Comp., 88 (2019), pp. 783--800.
2.
P. Azimzadeh and P. A. Forsyth, Weakly chained matrices, policy iteration, and impulse control, SIAM J. Numer. Anal., 54 (2016), pp. 1341--1364, https://doi.org/10.1137/15M1043431.
3.
G. Barles and P. E. Souganidis, Convergence of approximation schemes for fully nonlinear second order equations, Asymptot. Anal., 4 (1991), pp. 271--283.
4.
J. T. Betts and S. L. Campbell, Discretize then optimize, in Mathematics for Industry: Challenges and Frontiers, SIAM, Philadelphia, 2005, pp. 140--157.
5.
O. Bokanowski, S. Maroso, and H. Zidani, Some convergence results for Howard's algorithm, SIAM J. Numer. Anal., 47 (2009), pp. 3001--3026, https://doi.org/10.1137/08073041X.
6.
J. H. Bramble and B. E. Hubbard, On a finite difference analogue of an elliptic boundary problem which is neither diagonally dominant nor of non-negative type, J. Math. Phys., 43 (1964), pp. 117--132.
7.
Y. Chen, J. W. L. Wan, and J. Lin, Monotone mixed finite difference scheme for Monge-Ampère equation, J. Sci. Comput., 176 (2018).
8.
M. G. Crandall, H. Ishii, and P.-L. Lions, User's guide to viscosity solutions of second order partial differential equations, Bull. Amer. Math. Soc. (N.S.), 27 (1992), pp. 1--67, https://doi.org/10.1090/S0273-0979-1992-00266-5.
9.
W. Ding, L. Qi, and Y. Wei, M-tensors and nonsingular M-tensors, Linear Algebra Appl., 439 (2013), pp. 3264--3278, https://doi.org/10.1016/j.laa.2013.08.038.
10.
W. Ding and Y. Wei, Solving multi-linear systems with M-tensors, J. Sci. Comput., 68 (2016), pp. 689--715, https://doi.org/10.1007/s10915-015-0156-7.
11.
P. A. Forsyth and G. Labahn, Numerical methods for controlled Hamilton-Jacobi-Bellman PDEs in finance, J. Comput. Finance, 11 (2007).
12.
S. Friedland, S. Gaubert, and L. Han, Perron-Frobenius theorem for nonnegative multilinear forms and extensions, Linear Algebra Appl., 438 (2013), pp. 738--749, https://doi.org/10.1016/j.laa.2011.02.042.
13.
S. Hu, Z. Huang, and L. Qi, Strictly nonnegative tensors and nonnegative tensor partition, Sci. China Math., 57 (2014), pp. 181--195, https://doi.org/10.1007/s11425-013-4752-4.
14.
H. J. Kushner and P. G. Dupuis, Numerical methods for stochastic control problems in continuous time, Appl. Math. 24, Springer, New York, 1992, https://doi.org/10.1007/978-1-4684-0441-8.
15.
A. M. Oberman, Convergent difference schemes for degenerate elliptic and parabolic equations: Hamilton--Jacobi equations and free boundary problems, SIAM J. Numer. Anal., 44 (2006), pp. 879--895, https://doi.org/10.1137/S0036142903435235.
16.
L. Qi, Eigenvalues of a real supersymmetric tensor, J. Symbolic Comput., 40 (2005), pp. 1302--1324, https://doi.org/10.1016/j.jsc.2005.05.007.
17.
L. Qi and Z. Luo, Tensor analysis: Spectral theory and special tensors, SIAM, Philadelphia, 2017, https://doi.org/10.1137/1.9781611974751.
18.
P. N. Shivakumar, J. J. Williams, Q. Ye, and C. A. Marinov, On two-sided bounds related to weakly diagonally dominant $M$-matrices with application to digital circuit dynamics, SIAM J. Matrix Anal. Appl., 17 (1996), pp. 298--312, https://doi.org/10.1137/S0895479894276370.
19.
N. Touzi, Optimal Stochastic Control, Stochastic Target Problems, and Backward SDE, Fields Instit. Monogr. 29, Springer, New York, 2013, https://doi.org/10.1007/978-1-4614-4286-8.
20.
L. Zhang, L. Qi, and G. Zhou, M-tensors and some applications, SIAM J. Matrix Anal. Appl., 35 (2014), pp. 437--452, https://doi.org/10.1137/130915339.
Information & Authors
Information
Published In
SIAM Journal on Matrix Analysis and Applications
Pages: 276 - 298
DOI: 10.1137/18M1196923
ISSN (online): 1095-7162
Copyright
© 2019, Society for Industrial and Applied Mathematics.
History
Submitted: 27 June 2018
Accepted: 20 December 2018
Published online: 19 February 2019
Keywords
MSC codes
Authors
Funding Information
National Science Foundation https://doi.org/10.13039/100000001 : DMS-1613170
Metrics & Citations
Metrics
Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited By
- An ADMM-based interior point method for solving nonnegative tensor least squares problems and its applicationsJournal of Computational and Applied Mathematics, Vol. 472 | 1 Jan 2026
- Dynamic Approaches for Finding Least Squares Solution of Time-Varying Multi-Linear SystemsIEEE Transactions on Automation Science and Engineering, Vol. 22 | 1 Jan 2025
- New preconditioned Gauss-Seidel type methods for solving multi-linear systems with $ \mathcal{M} $-tensorsElectronic Research Archive, Vol. 33, No. <![CDATA[6]]> | 1 Jan 2025
- Restarted Nonnegativity Preserving Tensor Splitting Methods via Relaxed Anderson Acceleration for Solving Multilinear SystemsCommunications on Applied Mathematics and Computation, Vol. 40 | 25 October 2024
- Iterative methods for solving tensor equations based on exponential accelerationNumerical Algorithms, Vol. 97, No. 1 | 13 November 2023
- Solving the system of nonsingular tensor equations via randomized Kaczmarz-like methodJournal of Computational and Applied Mathematics, Vol. 421 | 1 Mar 2023
- A noise-suppressing discrete-time neural dynamics model for solving time-dependent multi-linear M -tensor equationNeurocomputing, Vol. 520 | 1 Feb 2023
- Existence and uniqueness of positive solution for multilinear systems with generalized strong M -tensorApplied Mathematics Letters, Vol. 133 | 1 Nov 2022
- A proximal point like method for solving tensor least-squares problemsCalcolo, Vol. 59, No. 1 | 26 November 2021
- Predefined-time convergent neural networks for solving the time-varying nonsingular multi-linear tensor equationsNeurocomputing, Vol. 472 | 1 Feb 2022
- Accelerated dynamical approaches for finding the unique positive solution of $\mathcal {K}\mathcal {S}$-tensor equationsNumerical Algorithms, Vol. 88, No. 4 | 2 April 2021
- An accelerated monotonic convergent algorithm for a class of non-Lipschitzian NCP( F ) involving an M -matrixJournal of Computational and Applied Mathematics, Vol. 397 | 1 Dec 2021
- Alternating Minimization Methods for Solving Multilinear SystemsMathematical Problems in Engineering, Vol. 2021 | 12 Jul 2021
- A two-step accelerated Levenberg–Marquardt method for solving multilinear systems in tensor-train formatJournal of Computational and Applied Mathematics, Vol. 382 | 1 Jan 2021
- Preconditioned tensor splitting AOR iterative methods for ℋ‐tensor equationsNumerical Linear Algebra with Applications, Vol. 27, No. 6 | 9 September 2020
- Neural network approach for solving nonsingular multi-linear tensor systemsJournal of Computational and Applied Mathematics, Vol. 368 | 1 Apr 2020
View Options
- Access via your Institution
- Questions about how to access this content? Contact SIAM at [email protected].