Randomized Sketching for Krylov Approximations of Large-Scale Matrix Functions
Abstract.
The computation of \(f(A){\boldsymbol{b}}\) , the action of a matrix function on a vector, is a task arising in many areas of scientific computing. In many applications, the matrix \(A\) is sparse but so large that only a rather small number of Krylov basis vectors can be stored. Here we discuss a new approach to overcome this limitation by randomized sketching combined with an integral representation of \(f(A){\boldsymbol{b}}\) . Two different approximation methods are introduced, one based on sketched FOM and another based on sketched GMRES. The convergence of the latter method is analyzed for Stieltjes functions of positive real matrices. We also derive a closed-form expression for the sketched FOM approximant and bound its distance to the full FOM approximant. Numerical experiments demonstrate the potential of the presented sketching approaches.
Keywords
MSC codes
Get full access to this article
View all available purchase options and get full access to this article.
Acknowledgments.
We are grateful for insightful discussions with Oleg Balabanov, Alice Cortinovis, Andreas Frommer, Daniel Kressner, and Yuji Nakatsukasa. We also thank the two anonymous referees for their insightful suggestions.
References
1.
M. Afanasjew, M. Eiermann, O. G. Ernst, and S. Güttel, A generalization of the steepest descent method for matrix functions, Electron. Trans. Numer. Anal., 28 (2008), pp. 206–222.
2.
M. Afanasjew, M. Eiermann, O. G. Ernst, and S. Güttel, Implementation of a restarted Krylov subspace method for the evaluation of matrix functions, Linear Algebra Appl., 429 (2008), pp. 2293–2314.
3.
W. E. Arnoldi, The principle of minimized iteration in the solution of the matrix eigenvalue problem, Q. Appl. Math., 9 (1951), pp. 17–29.
4.
O. Balabanov and L. Grigori, Randomized Block Gram-Schmidt Process for Solution of Linear Systems and Eigenvalue Problems, preprint, arXiv:2111.14641, 2021.
5.
O. Balabanov and L. Grigori, Randomized Gram-Schmidt process with application to GMRES, SIAM J. Sci. Comput., 44 (2022), pp. A1450–A1474.
6.
O. Balabanov and A. Nouy, Randomized linear algebra for model reduction. Part I: Galerkin methods and error estimation, Adv. Comput. Math., 45 (2019), pp. 2969–3019.
7.
O. Balabanov and A. Nouy, Randomized linear algebra for model reduction. Part II: Minimal residual methods and dictionary-based approximation, Adv. Comput. Math., 47 (2021), pp. 1–54.
8.
B. Beckermann, S. A. Goreinov, and E. E. Tyrtyshnikov, Some remarks on the Elman estimate for GMRES, SIAM J. Matrix Anal. Appl., 27 (2005), pp. 772–778.
9.
D. Black, @HPCpodcast: Jack Dongarra Talks Turing Award, the TOP500 and the Past and Future of Supercomputing, 2022, https://insidehpc.com/2022/05/hpcpodcast-jack-dongarra-talks-turing-award-the-top500-and-the-past-and-future-of-supercomputing/.
10.
A. Boriçi, Fast methods for computing the Neuberger operator, in Numerical Challenges in Lattice Quantum Chromodynamics, A. Frommer, T. Lippert, B. Medeke, and K. Schilling, eds., Springer-Verlag, Berlin, 2000, pp. 40–47.
11.
J. Brannick, A. Frommer, K. Kahl, B. Leder, M. Rottmann, and A. Strebel, Multigrid preconditioning for the overlap operator in lattice QCD, Numer. Math., 132 (2016), pp. 463–490.
12.
A. Cortinovis, D. Kressner, and Y. Nakatsukasa, Speeding Up Krylov Subspace Methods for Computing \(f (a) b\) via Randomization, preprint, arXiv:2212.12758, 2022.
13.
V. Druskin and L. Knizhnerman, Two polynomial methods of calculating functions of symmetric matrices, U.S.S.R. Comput. Math. Math. Phys., 29 (1989), pp. 112–121.
14.
V. Druskin and L. Knizhnerman, Extended Krylov subspaces: Approximation of the matrix square root and related functions, SIAM J. Matrix Anal. Appl., 19 (1998), pp. 755–771.
15.
M. Eiermann and O. G. Ernst, A restarted Krylov subspace method for the evaluation of matrix functions, SIAM J. Numer. Anal., 44 (2006), pp. 2481–2504.
16.
M. Eiermann, O. G. Ernst, and S. Güttel, Deflated restarting for matrix functions, SIAM J. Matrix Anal. Appl., 32 (2011), pp. 621–641.
17.
H. C. Elman, Iterative Methods for Large Sparse Nonsymmetric Systems of Linear Equations, Ph.D. thesis, Yale University, New Haven, CT, 1982.
18.
J. van den Eshof, A. Frommer, Th. Lippert, K. Schilling, and H. A. van der Vorst, Numerical methods for the QCD overlap operator. I. Sign-function and error bounds, Comput. Phys. Commun., 146 (2002), pp. 203–224.
19.
A. Frommer, S. Güttel, and M. Schweitzer, Convergence of restarted Krylov subspace methods for Stieltjes functions of matrices, SIAM J. Matrix Anal. Appl., 35 (2014), pp. 1602–1624.
20.
A. Frommer, S. Güttel, and M. Schweitzer, Efficient and stable Arnoldi restarts for matrix functions based on quadrature, SIAM J. Matrix Anal. Appl., 35 (2014), pp. 661–683.
21.
A. Frommer and P. Maass, Fast CG-based methods for Tikhonov-Phillips regularization, SIAM J. Sci. Comput., 20 (1999), pp. 1831–1850.
22.
A. Frommer and V. Simoncini, Matrix functions, in Model Order Reduction: Theory, Research Aspects and Applications, W. H. A. Schilders, H. A. van der Vorst, and J. Rommes, eds., Springer-Verlag, Berlin, 2008, pp. 275–303.
23.
S. Gaudreault, G. Rainwater, and M. Tokman, KIOPS: A fast adaptive Krylov subspace solver for exponential integrators, J. Comput. Phys., 372 (2018), pp. 236–255.
24.
S. Güttel, Rational Krylov approximation of matrix functions: Numerical methods and optimal pole selection, GAMM-Mitt., 36 (2013), pp. 8–31.
25.
S. Güttel and L. Knizhnerman, A black-box rational Arnoldi variant for Cauchy-Stieltjes matrix functions, BIT, 53 (2013), pp. 595–616.
26.
S. Güttel, D. Kressner, and K. Lund, Limited-memory polynomial methods for large-scale matrix functions, GAMM-Mitt., 43 (2020), e202000019, https://onlinelibrary.wiley.com/doi/abs/10.1002/gamm.202000019.
27.
S. Güttel and M. Schweitzer, A comparison of limited-memory Krylov methods for Stieltjes functions of Hermitian matrices, SIAM J. Matrix Anal. Appl., 42 (2021), pp. 83–107, https://doi.org/10.1137/20M1351072.
28.
P. Henrici, Applied and Computational Complex Analysis, Vol. 2, John Wiley & Sons, New York, 1977.
29.
M. R. Hestenes and E. Stiefel, Methods of conjugate gradients for solving linear systems, J. Res. Natl. Bur. Stand., 49 (1952), pp. 409–436.
30.
N. J. Higham, Functions of Matrices: Theory and Computation, SIAM, Philadelphia, 2008.
31.
M. Ilić, I. W. Turner, and D. P. Simpson, A restarted Lanczos approximation to functions of a symmetric matrix, IMA J. Numer. Anal., 30 (2010), pp. 1044–1061.
32.
W. D. Joubert and G. F. Carey, Parallelizable restarted iterative methods for nonsymmetric linear systems. Part I: Theory, Int. J. Comput. Math., 44 (1992), pp. 243–267.
33.
A. Koskela, Approximating the matrix exponential of an advection-diffusion operator using the incomplete orthogonalization method, in Numerical Mathematics and Advanced Applications-ENUMATH 2013, Springer-Verlag, Berlin, 2015, pp. 345–353.
34.
C. Lanczos, An iteration method for the solution of the eigenvalue problem of linear differential and integral operators, J. Res. Natl. Stand., 45 (1950), pp. 255–282.
35.
J. Leskovec and A. Krevl, SNAP Datasets: Stanford Large Network Dataset Collection, 2014, http://snap.stanford.edu/data.
36.
Y. Nakatsukasa and J. A. Tropp, Fast & Accurate Randomized Algorithms for Linear Systems and Eigenvalue Problems, preprint, arXiv:2111.00113, 2021.
37.
H. Neuberger, Exactly massless quarks on the lattice, Phys. Lett., B, 417 (1998), pp. 141–144.
38.
B. Philippe and L. Reichel, On the generation of Krylov subspace bases, Appl. Numer. Math., 62 (2012), pp. 1171–1186.
39.
V. Rokhlin and M. Tygert, A fast randomized algorithm for overdetermined linear least-squares regression, Proc. Natl. Acad. Sci. USA, 105 (2008), pp. 13212–13217.
40.
Y. Saad, Variations on Arnoldi’s method for computing eigenelements of large unsymmetric matrices, Linear Algebra Appl., 34 (1980), pp. 269–295.
41.
Y. Saad, Krylov subspace methods for solving large unsymmetric linear systems, Math. Comput., 37 (1981), pp. 105–126.
42.
Y. Saad, Analysis of some Krylov subspace approximations to the matrix exponential operator, SIAM J. Numer. Anal., 29 (1992), pp. 209–228.
43.
Y. Saad, Iterative Methods for Sparse Linear Systems, 2nd ed., SIAM, Philadelphia, 2000.
44.
Y. Saad and M. Schultz, GMRES: A generalized minimal residual algorithm for solving nonsymmetric linear systems, SIAM J. Sci. Stat. Comput., 7 (1986), pp. 856–869.
45.
T. Sarlos, Improved approximation algorithms for large matrices via random projections, in 47th Annual IEEE Symposium on Foundations of Computer Science (FOCS’06), IEEE, New York, 2006, pp. 143–152.
46.
M. Schweitzer, Restarting and Error Estimation in Polynomial and Extended Krylov Subspace Methods for the Approximation of Matrix Functions, Ph.D. thesis, Bergische Universität Wuppertal, 2016.
47.
H. Tal-Ezer, On restart and error estimation for Krylov approximation of \(w=f(A)v\) , SIAM J. Sci. Comput., 29 (2007), pp. 2426–2441.
48.
E. Timsit, L. Grigori, and O. Balabanov, Randomized Orthogonal Projection Methods for Krylov Subspace Solvers, preprint, arXiv:2302.07466, 2023.
49.
J. van den Eshof and M. Hochbruck, Preconditioning Lanczos approximations to the matrix exponential, SIAM J. Sci. Comput., 27 (2006), pp. 1438–1457, https://doi.org/10.1137/040605461.
50.
D. P. Woodruff, Sketching as a tool for numerical linear algebra, Found. Trends Theor. Comput. Sci., 10 (2014), pp. 1–157.
51.
F. Woolfe, E. Liberty, V. Rokhlin, and M. Tygert, A fast randomized algorithm for the approximation of matrices, Appl. Comput. Harmon. Anal., 25 (2008), pp. 335–366.
Information & Authors
Information
Published In
SIAM Journal on Matrix Analysis and Applications
Pages: 1073 - 1095
DOI: 10.1137/22M1518062
ISSN (online): 1095-7162
Copyright
© 2023 Society for Industrial and Applied Mathematics.
History
Submitted: 24 August 2022
Accepted: 17 March 2023
Published online: 25 July 2023
Keywords
MSC codes
Authors
Funding Information
Funding: The first author acknowledges a fellowship from The Alan Turing Institute under EPSRC grant EP/W001381/1 and a Royal Society Industry Fellowship IF/R1/231032.
Metrics & Citations
Metrics
Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited By
- A Sketch-and-Select Arnoldi ProcessSIAM Journal on Scientific Computing, Vol. 46, No. 4 | 21 August 2024
- Sketched and truncated polynomial Krylov subspace methods: Matrix Sylvester equationsMathematics of Computation, Vol. 6 | 25 July 2024
- A Fast Monte Carlo Algorithm for Evaluating Matrix Functions with Application in Complex NetworksJournal of Scientific Computing, Vol. 99, No. 2 | 1 April 2024
View Options
Get Access
- Access via your Institution
- Questions about how to access this content? Contact SIAM at [email protected].