Efficient and Stable Arnoldi Restarts for Matrix Functions Based on Quadrature
Abstract
When using the Arnoldi method for approximating $f(A){\mathbf b}$, the action of a matrix function on a vector, the maximum number of iterations that can be performed is often limited by the storage requirements of the full Arnoldi basis. As a remedy, different restarting algorithms have been proposed in the literature, none of which has been universally applicable, efficient, and stable at the same time. We utilize an integral representation for the error of the iterates in the Arnoldi method which then allows us to develop an efficient quadrature-based restarting algorithm suitable for a large class of functions, including the so-called Stieltjes functions and the exponential function. Our method is applicable for functions of Hermitian and non-Hermitian matrices, requires no a priori spectral information, and runs with essentially constant computational work per restart cycle. We comment on the relation of this new restarting approach to other existing algorithms and illustrate its efficiency and numerical stability by various numerical experiments.
Keywords
MSC codes
Get full access to this article
View all available purchase options and get full access to this article.
References
1.
M. Afanasjew, M. Eiermann, O. G. Ernst, and S. Güttel, A generalization of the steepest descent method for matrix functions, Electron. Trans. Numer. Anal., 28 (2008), pp. 206--222.
2.
M. Afanasjew, M. Eiermann, O. G. Ernst, and S. Güttel, Implementation of a restarted Krylov subspace method for the evaluation of matrix functions, Linear Algebra Appl., 429 (2008), pp. 229--314.
3.
G. A. Baker and P. Graves-Morris, Padé Approximants, Cambridge University Press, Cambridge, UK, 1996.
4.
B. Beckermann and L. Reichel, Error estimation and evaluation of matrix functions via the Faber transform, SIAM J. Numer. Anal., 47 (2009), pp. 3849--3883.
5.
C. Berg and G. Forst, Potential Theory on Locally Compact Abelian Groups, Springer-Verlag, Heidelberg, 1975.
6.
J. Bloch, A. Frommer, B. Lang, and T. Wettig, An iterative method to compute the sign function of a non-Hermitian matrix and its application to the overlap Dirac operator at nonzero chemical potential, Comput. Phys. Comm., 177 (2007), pp. 933--943.
7.
K. Burrage, N. Hale, and D. Kay, An efficient implicit FEM scheme for fractional-in-space reaction--diffusion equations, SIAM J. Sci. Comput., 34 (2012), pp. A2145--A2172.
8.
J. R. Cardoso, Computation of the matrix $p$th root and its Fréchet derivative by integrals, Electron. Trans. Numer. Anal., 39 (2012), pp. 414--436.
9.
P. Davies and N. Higham, A Schur--Parlett algorithm for computing matrix functions, SIAM J. Matrix Anal. Appl., 25 (2003), pp. 464--485.
10.
P. J. Davis and P. Rabinowitz, Methods of Numerical Integration, Academic Press, New York, 1975.
11.
C. de Boor, Divided differences, Surv. Approx. Theory, 1 (2005), pp. 46--69.
12.
V. Druskin and L. Knizhnerman, Two polynomial methods of calculating functions of symmetric matrices, Comput. Math. Math. Physi., 29 (1989), pp. 112--121.
13.
V. Druskin and L. Knizhnerman, Extended Krylov subspaces: Approximation of the matrix square root and related functions, SIAM J. Matrix Anal. Appl., 19 (1998), pp. 775--771.
14.
A. Dutt, L. Greengard, and V. Rokhlin, Spectral deferred correction methods for ordinary differential equations, BIT, 40 (2000), pp. 241--266.
15.
M. Eiermann and O. G. Ernst, A restarted Krylov subspace method for the evaluation of matrix functions, SIAM J. Numer. Anal., 44 (2006), pp. 2481--2504.
16.
M. Eiermann, O. G. Ernst, and S. Güttel, Deflated restarting for matrix functions, SIAM J. Matrix Anal. Appl., 32 (2011), pp. 621--641.
17.
T. Ericsson, Computing Functions of Matrices Using Krylov Subspace Methods, Technical report, Chalmers University of Technology, Gothenburg, Sweden, 1990.
18.
A. Frommer, S. Güttel, and M. Schweitzer, FUNM_QUAD: An Implementation of a Stable, Quadrature-Based Restarted Arnoldi Method for Matrix Functions, Technical report, IMACM Preprint 14/04, 2014.
19.
A. Frommer and V. Simoncini, Marix functions, in Model Order Reduction: Theory, Research Aspects and Applications, H. A. van der Vorst, W. H. A. Schilders, and J. Rommes, eds., Mathematics in Industry, Springer-Verlag, Berlin, 2008.
20.
E. Gallopoulos and Y. Saad, Efficient solution of parabolic equations by Krylov approximation methods, SIAM J. Sci. Statist. Comput., 13 (1992), pp. 1236--1264.
21.
W. Gautschi, Quadrature formulae on half-infinite intervals, BIT, 31 (1991), pp. 437--446.
22.
S. Güttel, Rational Krylov Methods for Operator Functions, Ph.D. thesis, Fakultät für Mathematik und Informatik der Technischen Universität Bergakademie Freiberg, 2010.
23.
S. Güttel, Rational Krylov approximation of matrix functions: Numerical methods and optimal pole selection, GAMM-Mitt., 36 (2013), pp. 8--31.
24.
S. Güttel and L. Knizhnerman, A black-box rational Arnoldi variant for Cauchy--Stieltjes matrix functions, BIT, 24 (2013), pp. 595--616.
25.
P. Henrici, Applied and Computational Complex Analysis, vol. 2, John Wiley & Sons, New York, 1977.
26.
N. J. Higham, Functions of Matrices: Theory and Computation, SIAM, Philadelphia, 2008.
27.
N. J. Higham and A. H. Al-Mohy, Computing matrix functions, Acta Numer., 19 (2010), pp. 159--208.
28.
M. Hochbruck and C. Lubich, On Krylov subspace approximations to the matrix exponential operator, SIAM J. Numer. Anal., 34 (1997), pp. 1911--1925.
29.
M. Hochbruck and A. Ostermann, Exponential integrators, Acta Numer., 19 (2010), pp. 209--286.
30.
M. Ilić, I. W. Turner, and D. P. Simpson, A restarted Lanczos approximation to functions of a symmetric matrix, IMA J. Numer. Anal., 30 (2010), pp. 1044--1061.
31.
L. Knizhnerman, Calculation of functions of unsymmetric matrices using Arnoldi's method, Comput. Math. Math. Phys., 31 (1991), pp. 1--9.
32.
L. Knizhnerman and V. Simoncini, A new investigation of the extended Krylov subspace method for matrix function evaluations, Numer. Linear Algebra Appl., 17 (2010), pp. 615--638.
33.
C. Lanczos, An iteration method for the solution of the eigenvalue problem of linear differential and integral operators, J. Res. National Bureau of Standards, 45 (1950), pp. 255--282.
34.
G. Lastman and N. Sinha, Infinite series for logarithm of matrix, applied to identification of linear continuous-time multivariable systems from discrete-time models, Electron. Lett., 27 (1991), pp. 1468--1470.
35.
I. Moret and P. Novati, RD-rational approximations of the matrix exponential., BIT, 44 (2004), pp. 595--615.
36.
H. Neuberger, Exactly massless quarks on the lattice, Phys. Lett. B, 417 (1998), pp. 141--144.
37.
C. C Paige, B. N Parlett, and H. A. Van der Vorst, Approximate solutions and eigenvalue bounds from Krylov subspaces, Numer. Linear Algebra Appl., 1 (1993), pp. 1--7.
38.
Y. Saad, Analysis of some Krylov subspace approximations to the exponential operator, SIAM J. Numer. Anal., 29 (1992), pp. 209--228.
39.
Y. Saad, Iterative Methods for Sparse Linear Systems, 2nd ed., SIAM, Philadelphia, 2003.
40.
Y. Saad and M. Schultz, GMRES: A generalized minimal residual algorithm for solving nonsymmetric linear systems, SIAM J. Sci. Statist. Comput., 7 (1986), pp. 856--869.
41.
L. F Shampine, Vectorized adaptive quadrature in Matlab, J. Comput. Appl. Math., 211 (2008), pp. 131--140.
42.
G. E. Shilov, Elementary Real and Complex Analysis, MIT Press, Cambridge, MA, 1973.
43.
B. Singer and S. Spilerman, The representation of social processes by Markov models, Amer. J. Sociol., 82 (1976), pp. 1--54.
44.
H. Tal-Ezer, On restart and error estimation for Krylov approximation of $w=f(A)v$, SIAM J. Sci. Comput., 29 (2007), pp. 2426--2441.
45.
L. N. Trefethen, J. A. C. Weideman, and T. Schmelzer, Talbot quadratures and rational approximations, BIT, 46 (2006), pp. 653--670.
46.
J. van den Eshof, A. Frommer, Th. Lippert, K. Schilling, and H. A. van der Vorst, Numerical methods for the QCD overlap operator, I. Sign-function and error bounds, Comput. Phys. Comm., 146 (2002), pp. 203--224.
47.
J. van den Eshof and M. Hochbruck, Preconditioning Lanczos approximations to the matrix exponential, SIAM J. Sci. Comput., 27 (2006), pp. 1438--1457.
48.
J. L. Walsh, Interpolation and Approximation by Rational Functions in the Complex Domain, AMS, Providence, RI, 1969.
49.
J. A. C. Weideman, Optimizing Talbot's contours for the inversion of the Laplace transform, SIAM J. Numer. Anal., 44 (2006), pp. 2342--2362.
50.
J. A. C. Weideman and L. N. Trefethen, Parabolic and hyperbolic contours for computing the Bromwich integral, Math. Comp., 76 (2007), pp. 1341--1356.
51.
E. I. Zolotarev, Application of elliptic functions to the question of functions deviating least and most from zero, Zap. Imp. Akad. Nauk. St. Petersburg, 30 (1877), pp. 1--59.
Information & Authors
Information
Published In
SIAM Journal on Matrix Analysis and Applications
Pages: 661 - 683
DOI: 10.1137/13093491X
ISSN (online): 1095-7162
Copyright
© 2014, Society for Industrial and Applied Mathematics.
History
Submitted: 29 August 2013
Accepted: 12 February 2014
Published online: 22 May 2014
Keywords
MSC codes
Authors
Metrics & Citations
Metrics
Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited By
- Quadrature-Based Restarted Arnoldi Method for Fast 3-D TEM Forward Modeling of Large-Scale ModelsIEEE Geoscience and Remote Sensing Letters, Vol. 22 | 1 Jan 2025
- Krylov Subspace Recycling with Randomized Sketching for Matrix FunctionsSIAM Journal on Matrix Analysis and Applications, Vol. 45, No. 4 | 2 December 2024
- Computational aspects of the geometric mean of two matrices: a surveyActa Scientiarum Mathematicarum, Vol. 90, No. 3-4 | 4 September 2024
- Error bounds for the approximation of matrix functions with rational Krylov methodsNumerical Linear Algebra with Applications, Vol. 31, No. 5 | 1 July 2024
- Theoretical error estimates for computing the matrix logarithm by Padé-type approximantsLinear and Multilinear Algebra | 24 September 2024
- A Sketch-and-Select Arnoldi ProcessSIAM Journal on Scientific Computing, Vol. 46, No. 4 | 21 August 2024
- Krylov Subspace Residual and Restarting for Certain Second Order Differential EquationsSIAM Journal on Scientific Computing, Vol. 46, No. 2 | 19 July 2023
- Speeding Up Krylov Subspace Methods for Computing \(\boldsymbol{{f}(A){b}}\) via RandomizationSIAM Journal on Matrix Analysis and Applications, Vol. 45, No. 1 | 9 February 2024
- Fast numerical computation on the harmonic Arnoldi algorithm for large-scale φ functions2024 IEEE 7th Advanced Information Technology, Electronic and Automation Control Conference (IAEAC) | 15 Mar 2024
- Low‐rank updates of matrix square rootsNumerical Linear Algebra with Applications, Vol. 31, No. 1 | 9 August 2023
- Randomized Sketching for Krylov Approximations of Large-Scale Matrix FunctionsSIAM Journal on Matrix Analysis and Applications, Vol. 44, No. 3 | 25 July 2023
- Coarse Grid Corrections in Krylov Subspace Evaluations of the Matrix ExponentialSIAM Journal on Scientific Computing, Vol. 45, No. 3 | 27 June 2023
- Low-Memory Krylov Subspace Methods for Optimal Rational Matrix Function ApproximationSIAM Journal on Matrix Analysis and Applications, Vol. 44, No. 2 | 19 May 2023
- Krylov Subspace Restarting for Matrix Laplace TransformsSIAM Journal on Matrix Analysis and Applications, Vol. 44, No. 2 | 19 May 2023
- Integral representations for higher-order Fréchet derivatives of matrix functions: Quadrature algorithms and new results on the level-2 condition numberLinear Algebra and its Applications, Vol. 656 | 1 Jan 2023
- Coarse grid corrections in Krylov subspace evaluations of the matrix exponentialKeldysh Institute Preprints, No. 75 | 1 January 2023
- Error Bounds for Lanczos-Based Matrix Function ApproximationSIAM Journal on Matrix Analysis and Applications, Vol. 43, No. 2 | 12 May 2022
- Some methods for solving equations with an operator function and applications for problems with a fractional power of an operatorJournal of Computational and Applied Mathematics, Vol. 407 | 1 Jun 2022
- A study of defect-based error estimates for the Krylov approximation of φ-functionsNumerical Algorithms, Vol. 90, No. 1 | 8 November 2021
- Block Gram-Schmidt algorithms and their stability propertiesLinear Algebra and its Applications, Vol. 638 | 1 Apr 2022
- Factorized Schemes for First and Second Order Evolution Equations with Fractional Powers of OperatorsComputational Methods in Applied Mathematics, Vol. 22, No. 2 | 20 October 2021
- 3-D Large-Scale TEM Modeling Using Restarting Polynomial Krylov MethodIEEE Transactions on Geoscience and Remote Sensing, Vol. 60 | 1 Jan 2022
- An approximate representation of a solution to fractional elliptical BVP via solution of parabolic IVPJournal of Computational and Applied Mathematics, Vol. 391 | 1 Aug 2021
- Splitting schemes for non-stationary problems with a rational approximation for fractional powers of the operatorApplied Numerical Mathematics, Vol. 165 | 1 Jul 2021
- An Accurate Restarting for Shift-and-Invert Krylov Subspaces Computing Matrix Exponential Actions of Nonsymmetric MatricesComputational Mathematics and Mathematical Physics, Vol. 61, No. 5 | 1 July 2021
- Rational Krylov for Stieltjes matrix functions: convergence and pole selectionBIT Numerical Mathematics, Vol. 61, No. 1 | 4 August 2020
- A Comparison of Limited-memory Krylov Methods for Stieltjes Functions of Hermitian MatricesSIAM Journal on Matrix Analysis and Applications, Vol. 42, No. 1 | 21 January 2021
- Residual and Restarting in Krylov Subspace Evaluation of the $\varphi$ FunctionSIAM Journal on Scientific Computing, Vol. 43, No. 6 | 4 November 2021
- Fast Rational Lanczos Method for the Toeplitz Symmetric Positive Semidefinite Matrix FunctionsSimulation Tools and Techniques | 27 April 2021
- Exponential Time Integrators for Unsteady Advection–Diffusion Problems on Refined MeshesNumerical Geometry, Grid Generation and Scientific Computing | 7 May 2021
- Approximation of the matrix exponential for matrices with a skinny field of valuesBIT Numerical Mathematics, Vol. 60, No. 4 | 15 May 2020
- Limited‐memory polynomial methods for large‐scale matrix functionsGAMM-Mitteilungen, Vol. 43, No. 3 | 10 September 2020
- High order numerical schemes for solving fractional powers of elliptic operatorsJournal of Computational and Applied Mathematics, Vol. 372 | 1 Jul 2020
- Approximation of a fractional power of an elliptic operatorNumerical Linear Algebra with Applications, Vol. 27, No. 3 | 31 January 2020
- The tensor t‐function: A definition for functions of third‐order tensorsNumerical Linear Algebra with Applications, Vol. 27, No. 3 | 20 February 2020
- Computing performability measures in Markov chains by means of matrix functionsJournal of Computational and Applied Mathematics, Vol. 368 | 1 Apr 2020
- Computable upper error bounds for Krylov approximations to matrix exponentials and associated $${\varvec{\varphi }}$$-functionsBIT Numerical Mathematics, Vol. 60, No. 1 | 11 September 2019
- Block Krylov Subspace Methods for Functions of Matrices II: Modified Block FOMSIAM Journal on Matrix Analysis and Applications, Vol. 41, No. 2 | 20 May 2020
- ART: Adaptive residual-time restarting for Krylov subspace matrix exponential evaluationsJournal of Computational and Applied Mathematics, Vol. 364 | 1 Jan 2020
- Comparison of exponential and implicit time integrators for unsteady advection-diffusion problems on refined meshesKeldysh Institute Preprints, No. 119 | 1 January 2020
- Inexact Arnoldi residual estimates and decay properties for functions of non-Hermitian matricesBIT Numerical Mathematics, Vol. 59, No. 4 | 15 June 2019
- Rational approximations to fractional powers of self-adjoint positive operatorsNumerische Mathematik, Vol. 143, No. 1 | 11 May 2019
- A block rational Krylov method for 3-D time-domain marine controlled-source electromagnetic modellingGeophysical Journal International, Vol. 218, No. 1 | 8 March 2019
- Multigrid approach in shifted linear systems for the non-degenerated twisted mass operatorComputer Physics Communications, Vol. 236 | 1 Mar 2019
- Adaptive residual-time restarting for Krylov subspace matrix exponential evaluationsKeldysh Institute Preprints, No. 127 | 1 January 2019
- Exponential Krylov time integration for modeling multi-frequency optical response with monochromatic sourcesJournal of Computational and Applied Mathematics, Vol. 340 | 1 Oct 2018
- Efficient Implementation of Rational Approximations to Fractional Differential OperatorsJournal of Scientific Computing, Vol. 76, No. 1 | 22 December 2017
- Enhanced Dependability Evaluation Through Krylov Methods and Matrix Functions: The Case of Load-Sharing Systems2018 48th Annual IEEE/IFIP International Conference on Dependable Systems and Networks Workshops (DSN-W) | 1 Jun 2018
- A shifted block FOM algorithm with deflated restarting for matrix exponential computationsApplied Numerical Mathematics, Vol. 127 | 1 May 2018
- Scabies in residential care homes: Modelling, inference and interventions for well-connected population sub-unitsPLOS Computational Biology, Vol. 14, No. 3 | 26 March 2018
- Computing the Weighted Geometric Mean of Two Large-Scale Matrices and Its Inverse Times a VectorSIAM Journal on Matrix Analysis and Applications, Vol. 39, No. 1 | 1 February 2018
- Numerical Solution of Time-Dependent Problems with Fractional Power Elliptic OperatorComputational Methods in Applied Mathematics, Vol. 18, No. 1 | 17 August 2017
- Acceleration of contour integration techniques by rational Krylov subspace methodsJournal of Computational and Applied Mathematics, Vol. 316 | 1 May 2017
- Rational Approximation to the Fractional Laplacian Operator in Reaction-Diffusion ProblemsSIAM Journal on Scientific Computing, Vol. 39, No. 1 | 17 January 2017
- The Radau--Lanczos Method for Matrix FunctionsSIAM Journal on Matrix Analysis and Applications, Vol. 38, No. 3 | 1 August 2017
- Handling Fermions on the LatticeLattice Quantum Chromodynamics | 21 October 2016
- Approximation of functions of large matrices with Kronecker structureNumerische Mathematik, Vol. 135, No. 1 | 22 March 2016
- APPROXIMATION OF THE LINEAR COMBINATION OF <i>φ</i>-FUNCTIONS USING THE BLOCK SHIFT-AND-INVERT KRYLOV SUBSPACE METHODJournal of Applied Analysis & Computation, Vol. 7, No. 4 | 1 Jan 2017
- Error bounds and estimates for Krylov subspace approximations of Stieltjes matrix functionsBIT Numerical Mathematics, Vol. 56, No. 3 | 17 December 2015
- Block Krylov subspace methods for approximating the linear combination of φ -functions arising in exponential integratorsComputers & Mathematics with Applications, Vol. 72, No. 4 | 1 Aug 2016
- Parallel matrix function evaluation via initial value ODE modelingComputers & Mathematics with Applications, Vol. 72, No. 1 | 1 Jul 2016
- A framework of the harmonic Arnoldi method for evaluating φ-functions with applications to exponential integratorsAdvances in Computational Mathematics, Vol. 42, No. 3 | 1 December 2015
- Matching exponential-based and resolvent-based centrality measuresJournal of Complex Networks, Vol. 4, No. 2 | 29 June 2015
- Multigrid preconditioning for the overlap operator in lattice QCDNumerische Mathematik, Vol. 132, No. 3 | 27 May 2015
- Estimating the Largest Elements of a MatrixSIAM Journal on Scientific Computing, Vol. 38, No. 5 | 19 October 2016
- Convergence of Restarted Krylov Subspace Methods for Stieltjes Functions of MatricesSIAM Journal on Matrix Analysis and Applications, Vol. 35, No. 4 | 18 December 2014
View Options
- Access via your Institution
- Questions about how to access this content? Contact SIAM at [email protected].