Computing the Generalized Singular Value Decomposition
Abstract
A variation of Paige’s algorithm is presented for computing the generalized singular value decomposition (GSVD) of two matrices A and B. There are two innovations. The first is a new preprocessing step which reduces A and B to upper triangular forms satisfying certain rank conditions. The second is a new $2 \times 2$ triangular GSVD algorithm, which constitutes the inner loop of Paige’s algorithm. Proofs of stability and high accuracy of the $2 \times 2$ GSVD algorithm are presented and are demonstrated using examples on which all previous algorithms fail.
MSC codes
Keywords
Get full access to this article
View all available purchase options and get full access to this article.
References
1.
IEEE Standard for Binary Floating Point Arithmetic, New York, 1985
2.
Zhaojun Bai, Note on the quadratic convergence of Kogbetliantz's algorithm for computing the singular value decomposition, Linear Algebra Appl., 104 (1988), 131–140
3.
Zhaojun Bai, Numerical treatment of restricted Gauss-Markov model, Comm. Statist. Simulation Comput., 17 (1988), 569–579
4.
Z. Bai, Ph.D. Thesis, The direct GSVD algorithm and its parallel implementation, Fudan University, China, 1987, Also available as Comput. Sci. TR-1901, Univ. of Maryland, College Park, MD, 1987
5.
Jesse L. Barlow, Error analysis and implementation aspects of deferred correction for equality constrained least squares problems, SIAM J. Numer. Anal., 25 (1988), 1340–1358
6.
A. W. Bojanczyk, M. Ewerbring, F. T. Luk, P. Van Dooren, R. J. Vaccaro, An accurate product SVD algorithmSVD and Signal Processing, II, Algorithms, Analysis and Applications, Elsevier Sci. Publishers, North Holland, Amsterdam, 1991
7.
J. P. Charlier, P. Van Dooren, On Kogbetliantz's SVD algorithm in the presence of clusters, Linear Algebra Appl., 95 (1987), 135–160
8.
B. L. R. De Moor, G. H. Golub, Generalized singular value decomposition: A proposal for a standardized nomenclature, Manuscript, NA-89-05, Stanford Univ., Stanford, CA, 1989
9.
B. L. R. De Moor, H. Zha, A tree of generalizations of the ordinary singular value decomposition, ESAT-SISTA Report, 1989-21, Katholieke Universiteit Leuven, Belgium, 1989
10.
James Demmel, W. Kahan, Accurate singular values of bidiagonal matrices, SIAM J. Sci. Statist. Comput., 11 (1990), 873–912
11.
J. Demmel, A. Mckenney, LAPACK working notes # 9, a test matrix generation suite, MCS-P69-0389, Argonne National Lab, Argonne, IL, 1989
12.
J. J. Dongarra, J. R. Bunch, C. B. Moler, G. W. Stewart, LINPACK User's Guide, Society for Industrial and Applied Mathematics, Philadelphia, PA, 1978
13.
L. Magnus Ewerbring, Franklin T. Luk, Canonical correlations and generalized SVD: applications and new algorithms, J. Comput. Appl. Math., 27 (1989), 37–52
14.
K. Vince Fernando, Linear convergence of the row cyclic Jacobi and Kogbetliantz methods, Numer. Math., 56 (1989), 73–91
15.
K. V. Fernando, S. J. Hammarling, B. N. Datta, C. R. Johnson, M. A. Kaashoek, R. J. Plemmons, E. D. Sontag, A product induced singular value decomposition for two matrices and balanced realizationLinear Algebra in Signals Systems and Control, Society for Industrial and Applied Mathematics, Philadelphia, PA, 1988, 128–140
16.
G. E. Forsythe, P. Henrici, The cyclic Jacobi method for computing the principal values of a complex matrix, Trans. Amer. Math. Soc., 94 (1960), 1–23
17.
Gene H. Golub, Charles F. Van Loan, Matrix computations, Johns Hopkins Series in the Mathematical Sciences, Vol. 3, Johns Hopkins University Press, Baltimore, MD, 1989xxii+642, 2nd ed.
18.
S. J. Hammarling, 1989, private communications
19.
V. Hari, K. Veselíc, On Jacobi methods for singular value decompositions, SIAM J. Sci. Statist. Comput., 8 (1987), 741–754
20.
M. T. Heath, A. J. Laub, C. C. Paige, R. C. Ward, Computing the singular value decomposition of a product of two matrices, SIAM J. Sci. Statist. Comput., 7 (1986), 1147–1159
21.
S. V. Huffel, J. Vandewalle, Analysis and properties of the generalized total least squares problem $AX\approx B$ when some or all columns in A are subject to error, SIAM J. Matrix Anal. Appl., 10 (1989), 294–315
22.
B. K. Ångström, The generalized singular value decomposition and the general $(A-\lambda B)$-problem, BIT, 24 (1984), 568–583
23.
E. G. Kogbetliantz, Solution of linear equations by diagonalization of coefficient matrix, Quart. Appl. Math., 13 (1955), 123–132
24.
F. T. Luk, H. T. Park, On parallel Jacobi orderings, SIAM J. Sci. Statist. Comput., 10 (1989), 18–26
25.
C. C. Paige, M. A. Saunders, Towards a generalized singular value decomposition, SIAM J. Numer. Anal., 18 (1981), 398–405
26.
C. C. Paige, A note on a result of Sun Ji Guang: sensitivity of the CS and GSV decompositions, SIAM J. Numer. Anal., 21 (1984), 186–191
27.
C. C. Paige, The general linear model and the generalized singular value decomposition, Linear Algebra Appl., 70 (1985), 269–284
28.
C. C. Paige, Computing the generalized singular value decomposition, SIAM J. Sci. Statist. Comput., 7 (1986), 1126–1146
29.
C. C. Paige, P. Van Dooren, On the quadratic convergence of Kogbetliantz's algorithm for computing the singular value decomposition, Linear Algebra Appl., 77 (1986), 301–313
30.
J. M. Speiser, C. F. Van Loan, Signal processing computations using the generalized singular value decomposition, Proc. Real Time Signal Processing VII, Vol. 495, SPIE, 1984, 47–55
31.
G. W. Stewart, Computing the $CS$ decomposition of a partitioned orthonormal matrix, Numer. Math., 40 (1982), 297–306
32.
J. Sun, Perturbation analysis for the generalized singular value problem, SIAM J. Numer. Anal., 20 (1983), 611–625
33.
Charles F. Van Loan, Generalizing the singular value decomposition, SIAM J. Numer. Anal., 13 (1976), 76–83
34.
C. F. Van Loan, Computing the CS and the generalized singular value decompositions, Numer. Math., 46 (1985), 479–491
35.
C. F. Van Loan, On the method of weighting for equality-constrained least-squares problems, SIAM J. Numer. Anal., 22 (1985), 851–864
36.
H. Zha, A numerical algorithm for computing the restricted singular value decomposition of matrix triplets, Linear Algebra Appl., 168 (1992), 1–25
Information & Authors
Information
Published In
Copyright
Copyright © 1993 Society for Industrial and Applied Mathematics.
History
Submitted: 19 August 1991
Accepted: 17 December 1992
Published online: 13 July 2006
MSC codes
Keywords
Authors
Metrics & Citations
Metrics
Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited By
- A New Matrix Maximization Model for Computing Ratios of Generalized Singular Values from High-Order GSVDJournal of Scientific Computing, Vol. 94, No. 2 | 6 January 2023
- Geometric Inexact Newton Method for Generalized Singular Values of Grassmann Matrix PairSIAM Journal on Matrix Analysis and Applications, Vol. 43, No. 2 | 5 April 2022
- Neural network for computing GSVD and RSVDNeurocomputing, Vol. 444 | 1 Jul 2021
- On worst-case condition numbers of a multiple nonzero finite generalized singular valueLinear Algebra and its Applications, Vol. 616 | 1 May 2021
- Randomized Generalized Singular Value DecompositionCommunications on Applied Mathematics and Computation, Vol. 3, No. 1 | 7 March 2020
- Parallel reduction of four matrices to condensed form for a generalized matrix eigenvalue algorithmNumerical Algorithms, Vol. 86, No. 1 | 2 March 2020
- Generalized singular value decomposition with iterated Tikhonov regularizationJournal of Computational and Applied Mathematics, Vol. 373 | 1 Aug 2020
- Secrecy Performance of the MIMO VLC Wiretap Channel With Randomly Located EavesdropperIEEE Transactions on Wireless Communications, Vol. 19, No. 1 | 1 Jan 2020
- The Analytic Solutions of a Class of Constrained Matrix Minimization and Maximization Problems with ApplicationsSIAM Journal on Optimization, Vol. 29, No. 2 | 20 June 2019
- Tikhonov RegularizationParameter Estimation and Inverse Problems | 1 Jan 2019
- BibliographyParameter Estimation and Inverse Problems | 1 Jan 2019
- Mode-Matching-Based Sound Field Recording and Synthesis with Circular Double-Layer ArraysApplied Sciences, Vol. 8, No. 7 | 4 July 2018
- On the Explicit Expression of Chordal Metric between Generalized Singular Values of Grassmann Matrix Pairs with ApplicationsSIAM Journal on Matrix Analysis and Applications, Vol. 39, No. 4 | 11 October 2018
- A Backward Stable Algorithm for Computing the CS Decomposition via the Polar DecompositionSIAM Journal on Matrix Analysis and Applications, Vol. 39, No. 3 | 25 September 2018
- A Generalized Singular Value Decomposition Strategy for Estimating the Block Recursive Simultaneous Equations ModelComputational Economics, Vol. 50, No. 3 | 9 June 2016
- Learning regularization parameters for general-form TikhonovInverse Problems, Vol. 33, No. 7 | 21 June 2017
- Recurrent neural network for computation of generalized eigenvalue problem with real diagonalizable matrix pair and its applicationsNeurocomputing, Vol. 216 | 1 Dec 2016
- Rank Decisions in Matrix Quotient DecompositionsSIAM Journal on Matrix Analysis and Applications, Vol. 37, No. 4 | 14 December 2016
- Blocking and parallelization of the Hari–Zimmermann variant of the Falk–Langemeyer algorithm for the generalized SVDParallel Computing, Vol. 49 | 1 Nov 2015
- Rescaling the GSVD with application to ill-posed problemsNumerical Algorithms, Vol. 68, No. 3 | 8 May 2014
- Matrix Eigenvalue ProblemsNumerical Methods in Matrix Computations | 7 October 2014
- Simultaneous multidiagonalization for the CS decompositionNumerical Algorithms, Vol. 66, No. 3 | 1 August 2013
- Simplified GSVD computations for the solution of linear discrete ill-posed problemsJournal of Computational and Applied Mathematics, Vol. 255 | 1 Jan 2014
- A note on simple nonzero finite generalized singular valuesLinear Algebra and its Applications, Vol. 438, No. 8 | 1 Apr 2013
- Divide and Conquer the CS DecompositionSIAM Journal on Matrix Analysis and Applications, Vol. 34, No. 2 | 25 April 2013
- BibliographyParameter Estimation and Inverse Problems | 1 Jan 2013
- Sensitivity analysis for the generalized singular value decompositionNumerical Linear Algebra with Applications, Vol. 20, No. 1 | 23 February 2012
- On the reduction of Tikhonov minimization problems and the construction of regularization matricesNumerical Algorithms, Vol. 60, No. 4 | 17 June 2012
- ReferencesSystem Identification | 13 April 2012
- Stable Computation of the CS Decomposition: Simultaneous BidiagonalizationSIAM Journal on Matrix Analysis and Applications, Vol. 33, No. 1 | 5 January 2012
- Frequency-domain weighted non-linear least-squares estimation of continuous-time, time-varying systemsIET Control Theory & Applications, Vol. 5, No. 7 | 5 May 2011
- Measurement of jet fragmentation p + p collision at = 200 GeV with the PHENIX detectorJournal of Physics: Conference Series, Vol. 230 | 8 July 2010
- A Note on Backward Error Analysis of the Generalized Singular Value DecompositionSIAM Journal on Matrix Analysis and Applications, Vol. 30, No. 4 | 16 October 2008
- Computing the complete CS decompositionNumerical Algorithms, Vol. 50, No. 1 | 8 July 2008
- Evolutionarily Conserved Transcriptional Co-Expression Guiding Embryonic Stem Cell DifferentiationPLoS ONE, Vol. 3, No. 10 | 15 October 2008
- Pole assignment via the schur formSystems & Control Letters, Vol. 56, No. 4 | 1 Apr 2007
- GENERALIZED SINGULAR VALUE DECOMPOSITION AND ITS APPLICATIONS IN MODEL ANALYSISInternational Journal of Theoretical and Applied Finance, Vol. 09, No. 02 | 21 November 2011
- Efficient Algorithms for Solution of Regularized Total Least SquaresSIAM Journal on Matrix Analysis and Applications, Vol. 26, No. 2 | 31 July 2006
- An implicit Jacobi-like method for computing generalized hyperbolic SVDLinear Algebra and its Applications, Vol. 358, No. 1-3 | 1 Jan 2003
- Inversion of multiangle sky radiance measurements for the retrieval of atmospheric optical properties 1. AlgorithmJournal of Geophysical Research: Atmospheres, Vol. 107, No. D22 | 27 November 2002
- A computationally efficient discrete Backus-Gilbert footprint-matching algorithmIEEE Transactions on Geoscience and Remote Sensing, Vol. 40, No. 8 | 1 Aug 2002
- On differentiability of the matrix trace operator and its applicationsKorean Journal of Computational & Applied Mathematics, Vol. 8, No. 1 | 1 Jan 2001
- On a variational formulation of the QSVD and the RSVDLinear Algebra and its Applications, Vol. 311, No. 1-3 | 1 May 2000
- On the Computation of the Restricted Singular Value Decomposition via the Cosine-Sine DecompositionSIAM Journal on Matrix Analysis and Applications, Vol. 22, No. 2 | 31 July 2006
- Condition Number and Backward Error for the Generalized Singular Value DecompositionSIAM Journal on Matrix Analysis and Applications, Vol. 22, No. 2 | 31 July 2006
- The decompositional approach to matrix computationComputing in Science & Engineering, Vol. 2, No. 1 | 1 Jan 2000
- Computing approximate stationary distributions for discrete Markov processes with banded infinitesimal generatorsJournal of Applied Probability, Vol. 36, No. 04 | 14 July 2016
- Computing approximate stationary distributions for discrete Markov processes with banded infinitesimal generatorsJournal of Applied Probability, Vol. 36, No. 4 | 14 July 2016
- Verification Algorithms for Generalized Singular ValuesMathematische Nachrichten, Vol. 208, No. 1 | 20 September 2010
- Analyses, Development, and Applications of TLS Algorithms in Frequency Domain System IdentificationSIAM Journal on Matrix Analysis and Applications, Vol. 19, No. 4 | 31 July 2006
- On a Variational Formulation of the Generalized Singular Value DecompositionSIAM Journal on Matrix Analysis and Applications, Vol. 18, No. 4 | 31 July 2006
- Accuracy of Gram-Schmidt orthogonalization and Householder transformation for the solution of linear least squares problemsNumerical Linear Algebra with Applications, Vol. 4, No. 4 | 1 Jul 1997
- A Gauss-Newton-like optimization algorithm for "weighted" nonlinear least-squares problemsIEEE Transactions on Signal Processing, Vol. 44, No. 9 | 1 Sep 1996
- Best conditioned parametric identification of transfer function models in the frequency domainIEEE Transactions on Automatic Control, Vol. 40, No. 11 | 1 Nov 1995
- Parametric identification of transfer functions in the frequency domain-a surveyIEEE Transactions on Automatic Control, Vol. 39, No. 11 | 1 Nov 1994
- Generalized Epsilon-PseudospectraSIAM Journal on Numerical Analysis, Vol. 31, No. 4 | 14 July 2006
- Parametric identification of transfer functions in the frequency domain: a surveyProceedings of 32nd IEEE Conference on Decision and Control
- On the use of orthogonal polynomials in high order frequency domain system identification and its application to modal parameter estimationProceedings of 1994 33rd IEEE Conference on Decision and Control