The Restricted Singular Value Decomposition: Properties and Applications
Abstract
The restricted singular value decomposition (RSVD) is the factorization of a given matrix, relative to two other given matrices. It can be interpreted as the ordinary singular value decomposition with different inner products in row and column spaces. Its properties and structure, as well as its connection to generalized eigenvalue problems, canonical correlation analysis, and other generalizations of the singular value decomposition, are investigated in detail.
Applications that are discussed include the analysis of the extended shorted operator, unitarily invariant norm minimization with rank constraints, rank minimization in matrix balls, the analysis and solution of linear matrix equations, rank minimization of a partitioned matrix, and the connection with generalized Schur complements, constrained linear and total linear least squares problems with mixed exact and noisy data, including a generalized Gauss–Markov estimation scheme.
MSC codes
Keywords
Get full access to this article
View all available purchase options and get full access to this article.
References
1.
L. Autonne, Sur les groupes linéaires, réels et orthogonaux, Bull. Sci. Math., France, 30 (1902), 121–133
2.
E. Beltrami, G. Battagline, E. Fergola, Sulle funzioni bilineari, Giornale di Mathematiche, Vol. 11, 1873, 98–106
3.
ke Björck, Gene H. Golub, Numerical methods for computing angles between linear subspaces, Math. Comp., 27 (1973), 579–594
4.
David Carlson, What are Schur complements, anyway?, Linear Algebra Appl., 74 (1986), 257–275
5.
John S. Chipman, Estimation and aggregation in econometrics: an application of the theory of generalized inversesGeneralized inverses and applications (Proc. Sem., Mat. Res. Center, Univ. Wisconsin, Madison, Wis., 1973), Academic Press, New York, 1976, 549–769. Publ. Math. Res. Center Univ. Wisconsin, No. 32
6.
Chandler Davis, W. M. Kahan, H. F. Weinberger, Norm-preserving dilations and their applications to optimal error bounds, SIAM J. Numer. Anal., 19 (1982), 445–469
7.
James Weldon Demmel, The smallest perturbation of a submatrix which lowers the rank and constrained total least squares problems, SIAM J. Numer. Anal., 24 (1987), 199–206
8.
B. De Moor, G. H. Golub, Generalized singular value decompositions: A proposal for a standardized nomenclature, Numerical Analysis Project Manuscript, NA-89-03, Department of Computer Science, Stanford University, Stanford, CA, 1989, April, also in ESAT-SISTA Report 1989-10, Department of Electrical Engineering, Katholieke Universiteit Leuven, Leuven, Belgium, April 1989.
9.
B. De Moor, G. H. Golub, The restricted singular value decomposition: Properties and applications, Numerical Analysis Project Manuscript, NA-89-04, Department of Computer Science, Stanford University, Stanford, CA, 1989, April; also in ESAT-SISTA Report 1989-09, Department of Electrical Engineering, Katholieke Universiteit Leuven, Leuven, Belgium, April 1989
10.
B. De Moor, On the structure and geometry of the product singular value decomposition, Numerical Analysis Project Manuscript, NA-89-05, Department of Computer Science, Stanford University, Stanford, CA, 1989, May, also in ESAT-SISTA Report 1989-12, Department of Electrical Engineering, Katholieke Universiteit Leuven, Leuven, Belgium, May 1989
11.
Bart De Moor, Hong Yuan Zha, A tree of generalizations of the ordinary singular value decomposition, Linear Algebra Appl., 147 (1991), 469–500
12.
C. Eckart, G. Young, The approximation of one matrix by another of lower rank, Psychometrika, 1 (1936), 211–218
13.
L. M. Ewerbring, F. T. Luk, Canonical correlations and generalized SVD; Applications and new algorithms, Proc. SPIE, Vol. 977, Real Time Signal Processing XI, paper 23, 1988
14.
K. V. Fernando, S. J. Hammarling, B. N. Datta, C. R. Johnson, M. A. Kaashoek, R. Plemmons, E. Sontag, A product induced singular value decomposition for two matrices and balanced realisationLinear Algebra in Signal Systems and Control, Society for Industrial and Applied Mathematics, Philadelphia, PA, 1988, 128–140
15.
G. H. Golub, C. Van Loan, Matrix computations, Johns Hopkins Series in the Mathematical Sciences, Vol. 3, Johns Hopkins University Press, Baltimore, MD, 1983xvi+476
16.
G. H. Golub, C. Van Loan, An analysis of the total least squares problem, SIAM J. Numer. Anal., 17 (1980), 883–893
17.
G. H. Golub, Alan Hoffman, G. W. Stewart, A generalization of the Eckart-Young-Mirsky matrix approximation theorem, Linear Algebra Appl., 88/89 (1987), 317–327
18.
W. E. Larimore, Identification of nonlinear systems using canonical variate analysis, Proc. 26th Conference on Decision and Control, Los Angeles, CA, 1987, December
19.
C. Lawson, R. Hanson, Solving least squares problems, Prentice-Hall Inc., Englewood Cliffs, N.J., 1974xii+340
20.
Sujit Kumar Mitra, Madan L. Puri, Shorted matrices—an extended concept and some applications, Linear Algebra Appl., 42 (1982), 57–79
21.
M. Z. Nashed, Generalized inverses and applications, Academic Press [Harcourt Brace Jovanovich Publishers], New York, 1976xiv+1054
22.
C. C. Paige, M. A. Saunders, Towards a generalized singular value decomposition, SIAM J. Numer. Anal., 18 (1981), 398–405
23.
C. C. Paige, The general linear model and the generalized singular value decomposition, Linear Algebra Appl., 70 (1985), 269–284
24.
R. Penrose, A generalized inverse for matrices, Proc. Cambridge Philos. Soc., 51 (1955), 406–413
25.
R. Penrose, On best approximation solutions of linear matrix equations, Proc. Cambridge Philos. Soc., 52 (1956), 17–19
26.
J. J. Sylvester, Sur la réduction biorthogonale d'une forme linéo-linéaire à sa forme canonique, Comptes Rendus, CVIII (1889), 651–653
27.
J. Vandewalle, B. De Moor, E. Deprettere, A variety of applications of the singular value decompositionSVD and Signal Processing: Algorithms, Applications and Architectures, North-Holland, Amsterdam, 1988, 43–91
28.
Sabine Van Huffel, Joos Vandewalle, Analysis and properties of the generalized total least squares problem $AX\approx B$ when some or all columns in A are subject to error, SIAM J. Matrix Anal. Appl., 10 (1989), 294–315
29.
S. Van Huffel, H. Zha, Restricted total least squares: A unified approach for solving (generalized) (total) least squares problems with(out) equality constraints, ESAT-SISTA Report, 1989-05, Department of Electrical Engineering, Katholieke Universiteit Leuven, Leuven, Belgium, 1989, March
30.
Charles F. Van Loan, Generalizing the singular value decomposition, SIAM J. Numer. Anal., 13 (1976), 76–83
31.
G. A. Watson, The smallest perturbation of a submatrix that lowers the rank of the matrix, IMA J. Numer. Anal., 8 (1988), 295–303
32.
H. Zha, Restricted SVD for matrix triplets and rank determination of matrices, Scientific Report, 89-2, Konrad-Zuse-Zentrum für Informationstechnik, Berlin, Germany, 1989
33.
H. Zha, A numerical algorithm for computing the RSVD for matrix triplets, Scientific Report, 89-1, Konrad-Zuse-Zentrum für Informationstechnik, Berlin, Germany, 1989
Information & Authors
Information
Published In
SIAM Journal on Matrix Analysis and Applications
Pages: 401 - 425
DOI: 10.1137/0612029
ISSN (online): 1095-7162
Copyright
Copyright © 1991 Society for Industrial and Applied Mathematics.
History
Submitted: 8 June 1989
Accepted: 12 September 1990
Published online: 17 July 2006
MSC codes
Keywords
Authors
Metrics & Citations
Metrics
Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited By
- RSVD for Three Quaternion Tensors with Applications in Color Video Watermark ProcessingAxioms, Vol. 12, No. 3 | 22 February 2023
- A Survey of Unsupervised Generative Models for Exploratory Data Analysis and Representation LearningACM Computing Surveys, Vol. 54, No. 5 | 9 July 2021
- Some applications of a decomposition for five quaternion matrices in control system and color image processingComputational and Applied Mathematics, Vol. 40, No. 6 | 11 August 2021
- Neural network for computing GSVD and RSVDNeurocomputing, Vol. 444 | 1 Jul 2021
- Detecting the Most Insightful Parts of Documents Using a Regularized Attention-Based ModelComputational Science – ICCS 2020 | 15 June 2020
- A simultaneous decomposition for seven matrices with applicationsJournal of Computational and Applied Mathematics, Vol. 349 | 1 Mar 2019
- An Evaluation Metric for Content Providing Models, Recommendation Systems, and Online CampaignsComputational Science – ICCS 2019 | 8 June 2019
- The complete equivalence canonical form of four matrices over an arbitrary division ringLinear and Multilinear Algebra, Vol. 66, No. 1 | 31 January 2017
- An Efficient Deep Learning Model for Recommender SystemsComputational Science – ICCS 2018 | 12 June 2018
- SSA Analysis of One-Dimensional Time SeriesSingular Spectrum Analysis with R | 15 June 2018
- Upper and lower bounds for the rank of the matrix-valued function when X has a fixed rankLinear and Multilinear Algebra, Vol. 65, No. 10 | 4 May 2017
- Optimization methods for regularization-based ill-posed problems: a survey and a multi-objective frameworkFrontiers of Computer Science, Vol. 11, No. 3 | 21 June 2017
- Simultaneous decomposition of quaternion matrices involving η-Hermicity with applicationsApplied Mathematics and Computation, Vol. 298 | 1 Apr 2017
- Deconvolution of a discrete uniform distributionStatistics & Probability Letters, Vol. 118 | 1 Nov 2016
- Two-sided coupled generalized Sylvester matrix equations solving using a simultaneous decomposition for fifteen matricesLinear Algebra and its Applications, Vol. 496 | 1 May 2016
- Algorithmic Approach for Learning a Comprehensive View of Online UsersProcedia Computer Science, Vol. 80 | 1 Jan 2016
- Recommendation System Based on Complete PersonalizationProcedia Computer Science, Vol. 80 | 1 Jan 2016
- Minimum rank (skew) Hermitian solutions to the matrix approximation problem in the spectral normJournal of Computational and Applied Mathematics, Vol. 288 | 1 Nov 2015
- The additive and block decompositions about the WLSEs of parametric functions for a multiple partitioned linear regression modelJournal of Multivariate Analysis, Vol. 133 | 1 Jan 2015
- Computation of Recommender System Using Localized RegularizationProcedia Computer Science, Vol. 51 | 1 Jan 2015
- Unsupervised Feature Extraction Using Singular Value DecompositionProcedia Computer Science, Vol. 51 | 1 Jan 2015
- Optimization problems on the rank of the solution to left and right inverse eigenvalue problemJournal of Industrial & Management Optimization, Vol. 11, No. 1 | 1 Jan 2015
- On relations between weighted least-squares estimators of parametric functions under a general partitioned linear model and its small modelsMetrika, Vol. 76, No. 5 | 18 October 2012
- Relations between least-squares and least-rank solutions of the matrix equationApplied Mathematics and Computation, Vol. 219, No. 20 | 1 Jun 2013
- On properties of BLUEs under general linear regression modelsJournal of Statistical Planning and Inference, Vol. 143, No. 4 | 1 Apr 2013
- Hermitian‐type generalized singular value decomposition with applicationsNumerical Linear Algebra with Applications, Vol. 20, No. 1 | 17 January 2012
- Minimum Rank Solutions to the Matrix Approximation Problems in the Spectral NormSIAM Journal on Matrix Analysis and Applications, Vol. 33, No. 3 | 28 August 2012
- Extremal ranks of a quadratic matrix expression with applicationsLinear and Multilinear Algebra, Vol. 59, No. 6 | 1 Jun 2011
- Maximization and minimization of the rank and inertia of the Hermitian matrix expression A-BX-(BX)* with applicationsLinear Algebra and its Applications, Vol. 434, No. 10 | 1 May 2011
- Relations between matrix sets generated from linear matrix expressions and their applicationsComputers & Mathematics with Applications, Vol. 61, No. 6 | 1 Mar 2011
- Weighted least-squares estimators of parametric functions of the regression coefficients under a general linear modelAnnals of the Institute of Statistical Mathematics, Vol. 62, No. 5 | 9 September 2008
- Equalities for orthogonal projectors and their operationsCentral European Journal of Mathematics, Vol. 8, No. 5 | 24 August 2010
- On additive and block decompositions of WLSEs under a multiple partitioned regression modelStatistics, Vol. 44, No. 4 | 1 Aug 2010
- How to use RSVD to solve the matrix equation A = BXCLinear and Multilinear Algebra, Vol. 58, No. 5 | 1 Jul 2010
- Restricted real perturbation values with applications to the structured real controllability radius of LTI systemsProceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference | 1 Dec 2009
- A unified balanced approach to multichannel blind deconvolutionSignal, Image and Video Processing, Vol. 1, No. 4 | 18 July 2007
- On least squares g-inverses and minimum norm g-inverses of a bordered matrixLinear Algebra and its Applications, Vol. 416, No. 2-3 | 1 Jul 2006
- Partial inverse problemsInverse Problems, Vol. 22, No. 2 | 6 March 2006
- Relationships among Various Kinds of Eigenvalue and Singular Value DecompositionsNew Developments in Psychometrics | 1 Jan 2003
- The Extended STLS Algorithm for Minimizing the Extended LS CriterionTotal Least Squares and Errors-in-Variables Modeling | 1 Jan 2002
- On a variational formulation of the QSVD and the RSVDLinear Algebra and its Applications, Vol. 311, No. 1-3 | 1 May 2000
- Two Purposes for Matrix Factorization: A Historical AppraisalSIAM Review, Vol. 42, No. 1 | 4 August 2006
- On the Computation of the Restricted Singular Value Decomposition via the Cosine-Sine DecompositionSIAM Journal on Matrix Analysis and Applications, Vol. 22, No. 2 | 31 July 2006
- Necessary and sufficient conditions for the output feedback regularization of descriptor systemsIEEE Transactions on Automatic Control, Vol. 44, No. 2 | 1 Feb 1999
- Convergence of an algorithm for the Riemannian SVDOpen Problems in Mathematical Systems and Control Theory | 1 Jan 1999
- Identification of nonlinear systems using a B-splines parametric subspace approachProceedings of the 1999 American Control Conference (Cat. No. 99CH36251) | 1 Jan 1999
- Structured Total Least Squares for Hankel MatricesCommunications, Computation, Control, and Signal Processing | 1 Jan 1997
- The riemannian singular value decompositionSVD and Signal Processing III | 1 Jan 1995
- Total least squares for affinely structured matrices and the noisy realization problemIEEE Transactions on Signal Processing, Vol. 42, No. 11 | 1 Nov 1994
- Dynamic Total Linear Least SquaresIFAC Proceedings Volumes, Vol. 27, No. 8 | 1 Jul 1994
- On the Structure of Generalized Singular Value and QR DecompositionsSIAM Journal on Matrix Analysis and Applications, Vol. 15, No. 1 | 31 July 2006
- Analysis of the generalized total least squares problemAX?B when some columns ofA are free of errorNumerische Mathematik, Vol. 65, No. 1 | 1 Dec 1993
- Structured total least squares and L2 approximation problemsLinear Algebra and its Applications, Vol. 188-189 | 1 Jul 1993
- H 2 Model Reduction for SISO SystemsIFAC Proceedings Volumes, Vol. 26, No. 2 | 1 Jul 1993
- Generalizations of the Singular Value and QR-DecompositionsSIAM Journal on Matrix Analysis and Applications, Vol. 13, No. 4 | 17 July 2006
- On the structure and geometry of the product singular value decompositionLinear Algebra and its Applications, Vol. 168 | 1 Apr 1992
- Generalizations of the singular value and QR decompositionsSignal Processing, Vol. 25, No. 2 | 1 Nov 1991
- The Restricted Total Least Squares Problem: Formulation, Algorithm, and PropertiesSIAM Journal on Matrix Analysis and Applications, Vol. 12, No. 2 | 17 July 2006
- L/sub 2/-optimal linear system identification structured total least squares for SISO systemsProceedings of 1994 33rd IEEE Conference on Decision and Control
- Continuous-time algorithms for the Riemannian SVDProceedings of 1995 34th IEEE Conference on Decision and Control
- Space-time treatment of inverse bioelectric field problemsProceedings of the 22nd Annual International Conference of the IEEE Engineering in Medicine and Biology Society (Cat. No.00CH37143)