Low-Memory Krylov Subspace Methods for Optimal Rational Matrix Function Approximation
Abstract.
We describe a Lanczos-based algorithm for approximating the product of a rational matrix function with a vector. This algorithm, which we call the Lanczos method for optimal rational matrix function approximation (Lanczos-OR), returns the optimal approximation from a given Krylov subspace in a norm depending on the rational function’s denominator, and it can be computed using the information from a slightly larger Krylov subspace. We also provide a low-memory implementation which only requires storing a number of vectors proportional to the denominator degree of the rational function. Finally, we show that Lanczos-OR can be used to derive algorithms for computing other matrix functions, including the matrix sign function and quadrature-based rational function approximations. In many cases, it improves on the approximation quality of prior approaches, including the standard Lanczos method, with little additional computational overhead.
Keywords
MSC codes
Get full access to this article
View all available purchase options and get full access to this article.
References
1.
M. Afanasjew, M. Eiermann, O. G. Ernst, and S. Güttel, Implementation of a restarted Krylov subspace method for the evaluation of matrix functions, Linear Algebra Appl., 429 (2008), pp. 2293–2314, https://doi.org/10.1016/j.laa.2008.06.029.
2.
M. Berljafa, S. Elsworth, and S. Güttel, A Rational Krylov Toolbox for MATLAB, MIMS preprint, http://eprints.maths.manchester.ac.uk/2773/, 2020.
3.
A. Boriçi, Fast methods for computing the Neuberger operator, in Numerical Challenges in Lattice Quantum Chromodynamics, Lecture Notes in Comput. Sci. Engrg. 15, A. Frommer et al., eds., Springer, Berlin, Heidelberg, 2000, pp. 40–47, pp. https://doi.org/10.1007/978-3-642-58333-9_4.
4.
T. Chen, A. Greenbaum, C. Musco, and C. Musco, Error bounds for lanczos-based matrix function approximation, SIAM J. Matrix Anal. Appl., 43 (2022), pp. 787–811, https://doi.org/10.1137/21m1427784.
5.
T. Chen, T. Trogdon, and S. Ubaru, Randomized Matrix-Free Quadrature for Spectrum and Spectral Sum Approximation, preprint, https://arxiv.org/abs/2204.01941, 2022.
6.
J. Cullum and A. Greenbaum, Relations between Galerkin and norm-minimizing iterative methods for solving linear systems, SIAM J. Matrix Anal. Appl., 17 (1996), pp. 223–247, https://doi.org/10.1137/S0895479893246765.
7.
V. Druskin, A. Greenbaum, and L. Knizhnerman, Using nonorthogonal Lanczos vectors in the computation of matrix functions, SIAM J. Sci. Comput., 19 (1998), pp. 38–54, https://doi.org/10.1137/S1064827596303661.
8.
V. Druskin and L. Knizhnerman, Two polynomial methods of calculating functions of symmetric matrices, USSR Comput. Math. Math. Phys., 29 (1989), pp. 112–121, https://doi.org/10.1016/s0041-5553(89)80020-5.
9.
B. Fischer, Polynomial Based Iteration Methods for Symmetric Linear Systems, Vieweg+Teubner Verlag, 1996, https://doi.org/10.1007/978-3-663-11108-5.
10.
R. W. Freund, Conjugate gradient-type methods for linear systems with complex symmetric coefficient matrices, SIAM J. Sci. Comput., 13 (1992), pp. 425–448, https://doi.org/10.1137/0913023.
11.
A. Frommer, S. Güttel, and M. Schweitzer, Convergence of restarted Krylov subspace methods for Stieltjes functions of matrices, SIAM J. Matrix Anal. Appl., 35 (2014), pp. 1602–1624, https://doi.org/10.1137/140973463.
12.
A. Frommer, S. Güttel, and M. Schweitzer, Efficient and stable Arnoldi restarts for matrix functions based on quadrature, SIAM J. Matrix Anal. Appl., 35 (2014), pp. 661–683, https://doi.org/10.1137/13093491x.
13.
A. Frommer and M. Schweitzer, Error bounds and estimates for Krylov subspace approximations of Stieltjes matrix functions, BIT Numer. Math., 56 (2016), pp. 865–892, https://doi.org/10.1007/s10543-015-0596-3.
14.
A. Frommer and V. Simoncini, Matrix functions, in Model Order Reduction: Theory, Research Aspects and Applications, W. H. A. Schilders, H. A. Vorst, and J. Rommes, eds., Math. Ind. 13, Springer, Berlin, 2008, pp. 275–303, https://doi.org/10.1007/978-3-540-78841-6_13.
15.
A. Frommer and V. Simoncini, Error bounds for Lanczos approximations of rational functions of matrices, in Numerical Validation in Current Hardware Architectures, Springer, Berlin, 2009, pp. 203–216.
16.
A. Greenbaum, Behavior of slightly perturbed Lanczos and conjugate-gradient recurrences, Linear Algebra Appl., 113 (1989), pp. 7–63, https://doi.org/10.1016/0024-3795(89)90285-1.
17.
A. Greenbaum, Iterative Methods for Solving Linear Systems, Frontiers Appl. Math. 17, SIAM, Philadelphia, 1997, https://doi.org/10.1137/1.9781611970937.
18.
A. Greenbaum, V. Druskin, and L. A. Knizhnerman, On solving indefinite symmetric linear systems by means of the Lanczos method, Zh. Vychisl. Mat. Mat. Fiz., 39 (1999), pp. 371–377.
19.
S. Güttel and M. Schweitzer, A comparison of limited-memory Krylov methods for Stieltjes functions of Hermitian matrices, SIAM J. Matrix Anal. Appl., 42 (2021), pp. 83–107, https://doi.org/10.1137/20m1351072.
20.
N. Hale, N. J. Higham, and L. N. Trefethen, Computing \({A}^\alpha, \log ({A})\) , and related matrix functions by contour integrals, SIAM J. Numer. Anal., 46 (2008), pp. 2505–2523, https://doi.org/10.1137/070700607.
21.
M. R. Hestenes and E. Stiefel, Methods of conjugate gradients for solving linear systems, J. Research Nat. Bur. Standards, 49 (1952), pp. 409–436.
22.
N. J. Higham, Accuracy and Stability of Numerical Algorithms, SIAM, Philadelphia, 2002, https://doi.org/10.1137/1.9780898718027.
23.
M. D. Ilić, I. W. Turner, and D. P. Simpson, A restarted Lanczos approximation to functions of a symmetric matrix, IMA J. Numer. Anal., 30 (2010), pp. 1044–1061, https://doi.org/10.1093/imanum/drp003.
24.
J. Liesen and Z. Strakoš, Krylov Subspace Methods: Principles and Analysis, Numer. Math. Sci. Comput., Oxford University Press, Oxford, UK, 2013.
25.
L. Lopez and V. Simoncini, Analysis of projection methods for rational function approximation to the matrix exponential, SIAM J. Numer. Anal., 44 (2006), pp. 613–635, https://doi.org/10.1137/05062590.
26.
G. Meurant and Z. Strakoš, The Lanczos and conjugate gradient algorithms in finite precision arithmetic, Acta Numer., 15 (2006), pp. 471–542.
27.
C. Musco, C. Musco, and A. Sidford, Stability of the Lanczos method for matrix function approximation, in Proceedings of the Twenty-Ninth Annual ACM-SIAM Symposium on Discrete Algorithms (SODA ’18), SIAM, Philadelphia, ACM, New York, 2018, pp. 1605–1624.
28.
J. Niehoff, Projektionsverfahren zur Approximation von Matrixfunktionen mit Anwendungen auf die Implementierung exponentieller Integratoren, Ph.D. thesis, Heinrich-Heine Universität Düsseldorf, Mathematisches Institut, 2006.
29.
C. C. Paige, The Computation of Eigenvalues and Eigenvectors of Very Large Sparse Matrices, Ph.D. thesis, University of London, 1971.
30.
C. C. Paige, Error analysis of the Lanczos algorithm for tridiagonalizing a symmetric matrix, IMA J. Appl. Math., 18 (1976), pp. 341–349, https://doi.org/10.1093/imamat/18.3.341.
31.
C. C. Paige, Accuracy and effectiveness of the Lanczos algorithm for the symmetric eigenproblem, Linear Algebra Appl., 34 (1980), pp. 235–258, https://doi.org/10.1016/0024-3795(80)90167-6.
32.
C. C. Paige, B. N. Parlett, and H. A. Van der Vorst, Approximate solutions and eigenvalue bounds from Krylov subspaces, Numer. Linear Algebra Appl., 2 (1995), pp. 115–133.
33.
C. C. Paige and M. A. Saunders, Solution of sparse indefinite systems of linear equations, SIAM J. Numer. Anal., 12 (1975), pp. 617–629, https://doi.org/10.1137/0712047.
34.
G. Pleiss, M. Jankowiak, D. Eriksson, A. Damle, and J. R. Gardner, Fast Matrix Square Roots with Applications to Gaussian Processes and Bayesian Optimization, preprint, https://arxiv.org/abs/2006.11267, 2020.
35.
Y. Saad, Analysis of some Krylov subspace approximations to the matrix exponential operator, SIAM J. Numer. Anal., 29 (1992), pp. 209–228, https://doi.org/10.1137/0729014.
36.
Y. Saad, Iterative Methods for Sparse Linear Systems, SIAM, Philadelphia, 2003, https://doi.org/10.1137/1.9780898718003.
37.
K. Schiefermayr, Estimates for the asymptotic convergence factor of two intervals, J. Comput. Appl. Math., 236 (2011), pp. 28–38, https://doi.org/10.1016/j.cam.2010.06.008.
38.
D. Šimonová and P. Tichý, When Does the Lanczos Algorithm Compute Exactly?, preprint, https://arxiv.org/abs/2106.02068, 2021.
39.
Z. Strakos, On the real convergence rate of the conjugate gradient method, Linear Algebra Appl., 154/156 (1991), pp. 535–549, https://doi.org/10.1016/0024-3795(91)90393-B.
40.
Z. Strakos and A. Greenbaum, Open Questions in the Convergence Analysis of the Lanczos Process for the Real Symmetric Eigenvalue Problem, IMA Preprint Series 934, University of Minnesota, 1992.
41.
J. van den Eshof, A. Frommer, Th. Lippert, K. Schilling, and H. van der Vorst, Numerical methods for the QCDd overlap operator I: Sign-function and error bounds, Comput. Phys. Commun., 146 (2002), pp. 203–224, https://doi.org/10.1016/S0010-4655(02)00455-1.
42.
E. Zolotarev, Application of elliptic functions to questions of functions deviating least and most from zero, Zap Imp. Akad. Nauk. St. Petersburg, 30 (1877), pp. 1–59.
Information & Authors
Information
Published In
SIAM Journal on Matrix Analysis and Applications
Pages: 670 - 692
DOI: 10.1137/22M1479853
ISSN (online): 1095-7162
Copyright
© 2023 Society for Industrial and Applied Mathematics.
History
Submitted: 22 February 2022
Accepted: 8 December 2022
Published online: 19 May 2023
Keywords
MSC codes
Authors
Funding Information
National Science Foundation (NSF): DGE-1762114, CCF-2045590, CCF-2046235
Funding: The work of the authors was supported by National Science Foundation grants DGE-1762114, CCF-2045590, and CCF-2046235 and by an Adobe Research grant.
Metrics & Citations
Metrics
Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited By
- Numerical experiments using the barycentric Lagrange treecode to compute correlated random displacements for Brownian dynamics simulationsJournal of Computational Physics, Vol. 525 | 1 Mar 2025
View Options
- Access via your Institution
- Questions about how to access this content? Contact SIAM at [email protected].