Abstract

Linear matrix equations, such as the Sylvester and Lyapunov equations, play an important role in various applications, including the stability analysis and dimensionality reduction of linear dynamical control systems and the solution of partial differential equations. In this work, we present and analyze a new algorithm, based on tensorized Krylov subspaces, for quickly updating the solution of such a matrix equation when its coefficients undergo low-rank changes. We demonstrate how our algorithm can be utilized to accelerate the Newton method for solving continuous-time algebraic Riccati equations. Our algorithm also forms the basis of a new divide-and-conquer approach for linear matrix equations with coefficients that feature hierarchical low-rank structure, such as hierarchically off-diagonal low-rank structures, hierarchically semiseparable, and banded matrices. Numerical experiments demonstrate the advantages of divide-and-conquer over existing approaches, in terms of computational time and memory consumption.

Keywords

  1. Sylvester equation
  2. Lyapunov equation
  3. low-rank update
  4. divide-and-conquer
  5. hierarchical matrices

MSC codes

  1. 15A06
  2. 93C20

Get full access to this article

View all available purchase options and get full access to this article.

References

1.
J. Abels and P. Benner, CAREX---A Collection of Benchmark Examples for Continuous-Time Algebraic Riccati Equations (Version 2.0), SLICOT working note 1999-14, 1999, http://www.slicot.org.
2.
S. Ambikasaran and E. Darve, An $\mathcal{O}(N\log N)$ fast direct solver for partial hierarchically semi-separable matrices: With application to radial basis function interpolation, J. Sci. Comput., 57 (2013), pp. 477--501, https://doi.org/10.1007/s10915-013-9714-z.
3.
A. C. Antoulas, Approximation of Large-Scale Dynamical Systems, SIAM, Philadelphia, 2005.
4.
A. C. Antoulas, D. C. Sorensen, and Y. Zhou, On the decay rate of Hankel singular values and related issues, Systems Control Lett., 46 (2002), pp. 323--342, https://doi.org/10.1016/S0167-6911(02)00147-0.
5.
J. Baker, M. Embree, and J. Sabino, Fast singular value decay for Lyapunov solutions with nonnormal coefficients, SIAM J. Matrix Anal. Appl., 36 (2015), pp. 656--668, https://doi.org/10.1137/140993867.
6.
J. Ballani and D. Kressner, Matrices with hierarchical low-rank structures, in Exploiting Hidden Structure in Matrix Computations: Algorithms and Applications, Lecture Notes in Math. 2173, Springer, New York, 2016, pp. 161--209.
7.
R. H. Bartels and G. W. Stewart, Algorithm 432: The solution of the matrix equation $AX + XB = C$, Commun. ACM, 15 (1972), pp. 820--826, http://doi.org/10.1145/361573.361582.
8.
U. Baur and P. Benner, Factorized solution of Lyapunov equations based on hierarchical matrix arithmetic, Computing, 78 (2006), pp. 211--234, http://doi.org/10.1007/s00607-006-0178-y.
9.
B. Beckermann, An error analysis for rational Galerkin projection applied to the Sylvester equation, SIAM J. Numer. Anal., 49 (2011), pp. 2430--2450, https://doi.org/10.1137/110824590.
10.
B. Beckermann and A. Townsend, On the singular values of matrices with displacement structure, SIAM J. Matrix Anal. Appl., 38 (2017), pp. 1227--1248, https://doi.org/10.1137/16M1096426.
11.
P. Benner, P. Ezzatti, D. Kressner, E. S. Quintana-Ortí, and A. Remón, A mixed-precision algorithm for the solution of Lyapunov equations on hybrid CPU-GPU platforms, Parallel Comput., 37 (2011), pp. 439--450, https://doi.org/10.1016/j.parco.2010.12.002.
12.
P. Benner, P. Kürschner, and J. Saak, Self-generating and efficient shift parameters in ADI methods for large Lyapunov and Sylvester equations, Electron. Trans. Numer. Anal., 43 (2014/15), pp. 142--162.
13.
P. Benner and J. Saak, Numerical solution of large and sparse continuous time algebraic matrix Riccati and Lyapunov equations: A state of the art survey, GAMM-Mitt., 36 (2013), pp. 32--52, http://doi.org/10.1002/gamm.201310003.
14.
D. A. Bini, S. Massei, and L. Robol, On the decay of the off-diagonal singular values in cyclic reduction, Linear Algebra Appl., 519 (2017), pp. 27--53, http://doi.org/10.1016/j.laa.2016.12.027.
15.
S. Birk, Deflated Shifted Block Krylov Subspace Methods for Hermitian Positive Definite Matrices, Ph.D. thesis, Wuppertal University, 2015.
16.
A. Bonnafé, Estimates and asymptotic expansions for condenser p-capacities. The anisotropic case of segments, Quaest. Math., 39 (2016), pp. 911--944, http://doi.org/10.2989/16073606.2016.1241955.
17.
S. Börm, Efficient Numerical Methods for Non-Local Operators: ${\mathcal{H}}{^{2}}$-Matrix Compression, Algorithms and Analysis, EMS Tracts Math. 14, European Mathematical Society, Zürich, 2010, https://doi.org/10.4171/091.
18.
D. Braess and W. Hackbusch, Approximation of $1/x$ by exponential sums in $[1,\infty)$, IMA J. Numer. Anal., 25 (2005), pp. 685--697, https://doi.org/10.1093/imanum/dri015.
19.
T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein, Introduction to Algorithms, MIT Press, Cambridge, MA, 2009.
20.
M. Crouzeix and C. Palencia, The numerical range is a $(1+\sqrt{2})$-spectral set, SIAM J. Matrix Anal. Appl., 38 (2017), pp. 649--655, https://doi.org/10.1137/17M1116672.
21.
M. Dahleh, M. A. Dahleh, and G. Verghese, Lectures on Dynamic Systems and Control, Department of Electrical Engineering and Computer Science, MIT, Cambridge, MA, 2004.
22.
T. Damm, Direct methods and ADI-preconditioned Krylov subspace methods for generalized Lyapunov equations, Numer. Linear Algebra Appl., 15 (2008), pp. 853--871, https://doi.org/10.1002/nla.603.
23.
E. D. Denman and A. N. Beavers, Jr., The matrix sign function and computations in systems, Appl. Math. Comput., 2 (1976), pp. 63--94, https://doi.org/10.1016/0096-3003(76)90020-5.
24.
H. C. Elman, K. Meerbergen, A. Spence, and M. Wu, Lyapunov inverse iteration for identifying Hopf bifurcations in models of incompressible flow, SIAM J. Sci. Comput., 34 (2012), pp. A1584--A1606, https://doi.org/10.1137/110827600.
25.
T. Ganelius, Rational Functions, Capacities, and Approximation, in Aspects of Contemporary Complex Analysis (Proc. NATO Adv. Study Inst., Univ. Durham, Durham, 1979), Academic Press, London, New York, 1980, pp. 409--414.
26.
I. P. Gavrilyuk, W. Hackbusch, and B. N. Khoromskij, Hierarchical tensor-product approximation to the inverse and related operators for high-dimensional elliptic problems, Computing, 74 (2005), pp. 131--157, https://doi.org/10.1007/s00607-004-0086-y.
27.
A. A. Gonchar, The problems of E. I. Zolotarev which are connected with rational functions, Mat. Sb. (N.S.), 78 (1969), pp. 640--654.
28.
L. Grasedyck, Existence of a low rank or $\mathcal H$-matrix approximant to the solution of a Sylvester equation, Numer. Linear Algebra Appl., 11 (2004), pp. 371--389, https://doi.org/10.1002/nla.366.
29.
L. Grasedyck and W. Hackbusch, A multigrid method to solve large scale Sylvester equations, SIAM J. Matrix Anal. Appl., 29 (2007), pp. 870--894, https://doi.org/10.1137/040618102.
30.
L. Grasedyck, W. Hackbusch, and B. N. Khoromskij, Solution of large scale algebraic matrix Riccati equations by use of hierarchical matrices, Computing, 70 (2003), pp. 121--165, https://doi.org/10.1007/s00607-002-1470-0.
31.
L. Grubišić and D. Kressner, On the eigenvalue decay of solutions to operator Lyapunov equations, Systems Control Lett., 73 (2014), pp. 42--47, https://doi.org/10.1016/j.sysconle.2014.09.006.
32.
M. H. Gutknecht, Block Krylov space methods for linear systems with multiple right-hand sides: An introduction, in Modern Mathematical Models, Methods, and Algorithms for Real World Systems, Anamaya Publishers, New Delhi, India, 2007, pp. 420--447.
33.
A. Haber and M. Verhaegen, Sparse solution of the Lyapunov equation for large-scale interconnected systems, Automatica J. IFAC, 73 (2016), pp. 256--268, https://doi.org/10.1016/j.automatica.2016.06.002.
34.
W. Hackbusch, Hierarchical Matrices: Algorithms and Analysis, Springer Ser. Comput. Math. 49, Springer, New York, 2015, http://doi.org/10.1007/978-3-662-47324-5.
35.
M. Heyouni, Extended Arnoldi methods for large low-rank Sylvester matrix equations, Appl. Numer. Math., 60 (2010), pp. 1171--1182, https://doi.org/10.1016/j.apnum.2010.07.005.
36.
N. J. Higham, Accuracy and Stability of Numerical Algorithms, 2nd ed., SIAM, Philadelphia, 2002.
37.
O. Kameník, Solving SDGE models: A new algorithm for the Sylvester equation, Comput. Econom., 25 (2005), pp. 167--187, https://doi.org/10.1007/s10614-005-6280-y.
38.
D. L. Kleinman, On an iterative technique for Riccati equation computations, IEEE Trans. Automat. Control, AC-13 (1968), pp. 114--115, https://doi.org/10.1109/TAC.1968.1098829.
39.
L. Knizhnerman and V. Simoncini, Convergence analysis of the extended Krylov subspace method for the Lyapunov equation, Numer. Math., 118 (2011), pp. 567--586, https://doi.org/10.1007/s00211-011-0366-3.
40.
J. G. Korvink and B. R. Evgenii, Oberwolfach benchmark collection, in Dimension Reduction of Large-Scale Systems, Lect. Notes Comput. Sci. Eng. 45, P. Benner, V. Mehrmann, and D. C. Sorensen, eds., Springer, New York, 2005, pp. 311--316, https://sparse.tamu.edu/Oberwolfach.
41.
D. Kressner and C. Tobler, Krylov subspace methods for linear systems with tensor product structure, SIAM J. Matrix Anal. Appl., 31 (2010), pp. 1688--1714, https://doi.org/10.1137/090756843.
42.
V. Kučera, Algebraic Riccati equation: Hermitian and definite solutions, in The Riccati Equation, Springer, New York, 1991, pp. 53--88.
43.
I. Kuzmanović and N. Truhar, Sherman-Morrison-Woodbury formula for Sylvester and $T$-Sylvester equations with applications, Int. J. Comput. Math., 90 (2013), pp. 306--324, https://doi.org/10.1080/00207160.2012.716154.
44.
P. Lancaster and L. Rodman, The Algebraic Riccati Equation, Oxford University Press, Oxford, UK, 1995.
45.
B. Le Bailly and J. P. Thiran, Optimal rational functions for the generalized Zolotarev problem in the complex plane, SIAM J. Numer. Anal., 38 (2000), pp. 1409--1424, https://doi.org/10.1137/S0036142999360688.
46.
J.-R. Li and J. White, Low-rank solution of Lyapunov equations, SIAM Rev., 46 (2004), pp. 693--713, https://doi.org/10.1137/S0895479801384937.
47.
S. Massei, D. Palitta, and L. Robol, Solving rank-structured Sylvester and Lyapunov equations, SIAM J. Matrix Anal. Appl., 39 (2018), pp. 1564--1590, https://doi.org/10.1137/17M1157155.
48.
D. Palitta and V. Simoncini, Numerical methods for large-scale Lyapunov equations with symmetric banded data, SIAM J. Sci. Comput., 40 (2018), pp. A3581--A3608, https://doi.org/10.1137/17M1156575.
49.
S. Pauli, A Numerical Solver for Lyapunov Equations Based on the Matrix Sign Function Iteration in HSS Arithmetic, Semester thesis, ETH Zurich, 2010, available from http://anchp.epfl.ch/students.
50.
T. Penzl, Eigenvalue decay bounds for solutions of Lyapunov equations: The symmetric case, Systems Control Lett., 40 (2000), pp. 139--144, https://doi.org/10.1016/S0167-6911(00)00010-4.
51.
J. K. Rice and M. Verhaegen, Distributed control: A sequentially semi-separable approach for spatially heterogeneous linear systems, IEEE Trans. Automat. Control, 54 (2009), pp. 1270--1283, https://doi.org/10.1109/TAC.2009.2019802.
52.
S. Richter, L. D. Davis, and E. G. Collins, Efficient computation of the solutions to modified Lyapunov equations, SIAM J. Matrix Anal. Appl., 14 (1993), pp. 420--431, https://doi.org/10.1137/0614030.
53.
E. Ringh, G. Mele, J. Karlsson, and E. Jarlebring, Sylvester-based preconditioning for the waveguide eigenvalue problem, Linear Algebra Appl., 542 (2018), pp. 441--463, https://doi.org/10.1016/j.laa.2017.06.027.
54.
J. Sabino, Solution of Large-Scale Lyapunov Equations via the Block Modified Smith Methods, Ph.D. thesis, Department of Computational and Applied Mathematics, Rice University, Houston, TX, 2006.
55.
V. Simoncini, A new iterative method for solving large-scale Lyapunov matrix equations, SIAM J. Sci. Comput., 29 (2007), pp. 1268--1288, https://doi.org/10.1137/06066120X.
56.
V. Simoncini, Computational methods for linear matrix equations, SIAM Rev., 58 (2016), pp. 377--441, https://doi.org/10.1137/130912839.
57.
C. F. Van Loan, The ubiquitous Kronecker product, J. Comput. Appl. Math., 123 (2000), pp. 85--100, https://doi.org/10.1016/S0377-0427(00)00393-9.
58.
Y. Xi, J. Xia, S. Cauley, and V. Balakrishnan, Superfast and stable structured solvers for Toeplitz least squares via randomized sampling, SIAM J. Matrix Anal. Appl., 35 (2014), pp. 44--72, https://doi.org/10.1137/120895755.
59.
J. Xia, S. Chandrasekaran, M. Gu, and X. S. Li, Fast algorithms for hierarchically semiseparable matrices, Numer. Linear Algebra Appl., 17 (2010), pp. 953--976, https://doi.org/10.1002/nla.691.
60.
J. Xia, Y. Xi, and M. Gu, A superfast structured solver for Toeplitz linear systems via randomized sampling, SIAM J. Matrix Anal. Appl., 33 (2012), pp. 837--858, https://doi.org/10.1137/110831982.

Information & Authors

Information

Published In

cover image SIAM Journal on Scientific Computing
SIAM Journal on Scientific Computing
Pages: A848 - A876
ISSN (online): 1095-7197

History

Submitted: 12 December 2017
Accepted: 7 February 2019
Published online: 26 March 2019

Keywords

  1. Sylvester equation
  2. Lyapunov equation
  3. low-rank update
  4. divide-and-conquer
  5. hierarchical matrices

MSC codes

  1. 15A06
  2. 93C20

Authors

Affiliations

Funding Information

Gruppo Nazionale per il Calcolo Scientifico

Metrics & Citations

Metrics

Citations

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

Cited By

Media

Figures

Other

Tables

Share

Share

Copy the content Link

Share with email

Email a colleague

Share on social media

The SIAM Publications Library now uses SIAM Single Sign-On for individuals. If you do not have existing SIAM credentials, create your SIAM account https://my.siam.org.