Accurate Downdating of Least Squares Solutions
Abstract
Solutions to a sequence of modified least squares problems, where either a new observation is added (updating) or an old observation is deleted (downdating), are required in many applications. Stable algorithms for downdating can be constructed if the complete QR factorization of the data matrix is available. Algorithms that only downdate R and do not store Q require less operations. However, they do not give good accuracy and may not recover accuracy after an ill-conditioned problem has occurred. The authors describe a new algorithm for accurate downdating of least squares solutions and compare it to existing algorithms. Numerical test results are also presented using the sliding window method, where a number of updatings and downdatings occur repeatedly.
MSC codes
Keywords
Get full access to this article
View all available purchase options and get full access to this article.
References
1.
S. T. Alexander, C.-T. Pan, R. J. Plemmons, Analysis of a recursive least squares hyperbolic rotation algorithm for signal processing, Linear Algebra Appl., 98 (1988), 3–40
2.
Andrew A. Anda, Haesun Park, Fast plane rotations with dynamic scaling, SIAM J. Matrix Anal. Appl., 15 (1994), 162–174
3.
M. G. Bellanger, J. McWhirter, The family of fast least squares algorithms for adaptive filteringMathematics in Signal Processing, Clarendon Press, Oxford, 1990, 415–434
4.
Å. Björck, Stability analysis of the method of seminormal equations for linear least squares problems, Linear Algebra Appl., 88/89 (1987), 31–48
5.
J. Daniel, W. B. Gragg, L. Kaufman, G. W. Stewart, Reorthogonalization and stable algorithms for updating the Gram-Schmidt $QR$ factorization, Math. Comp., 30 (1976), 772–795
6.
L. Eldén, J. McWhirter, Downdating $QR$ decompositionMathematics in Signal Processing, Clarendon Press, Oxford, 1990, 561–574
7.
William R. Ferng, Gene H. Golub, Robert J. Plemmons, Adaptive Lanczos methods for recursive condition estimation, Numer. Algorithms, 1 (1991), 1–19
8.
Gene H. Golub, Charles F. Van Loan, Matrix computations, Johns Hopkins Series in the Mathematical Sciences, Vol. 3, Johns Hopkins University Press, Baltimore, MD, 1989xxii+642, 2nd ed.
9.
C. C. Paige, Error analysis of some techniques for updating orthogonal decompositions, Math. Comp., 34 (1980), 465–471
10.
C.-T. Pan, A perturbation analysis of the problem of downdating a Cholesky factorization, Linear Algebra Appl., 183 (1993), 103–115
11.
C.-T. Pan, R. J. Plemmons, Least squares modifications with inverse factorizations: parallel implications, J. Comput. Appl. Math., 27 (1989), 109–127
12.
Beresford N. Parlett, The symmetric eigenvalue problem, Prentice-Hall Inc., Englewood Cliffs, N.J., 1980xix+348
13.
L. Reichel, W. B. Gragg, FORTRAN subroutines for updating the $QR$ decomposition, ACM Trans. Math. Software, 16 (1990), 369–377
14.
M. A. Saunders, Large-scale linear programming using the Cholesky factorization, Tech. Report, CS252, Computer Science Dept., Stanford University, Stanford, CA, 1972
15.
G. W. Stewart, The effects of rounding error on an algorithm for downdating a Cholesky factorization, J. Inst. Math. Appl., 23 (1979), 203–213
Information & Authors
Information
Published In
SIAM Journal on Matrix Analysis and Applications
Pages: 549 - 568
ISSN (online): 1095-7162
Copyright
Copyright © 1994 © Society for Industrial and Applied Mathematics.
History
Submitted: 7 April 1992
Accepted: 10 September 1992
Published online: 31 July 2006
MSC codes
Keywords
Authors
Metrics & Citations
Metrics
Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited By
- Algorithms for modifying recurrence relations of orthogonal polynomial and rational functions when changing the discrete inner productApplied Numerical Mathematics, Vol. 200 | 1 Jun 2024
- Sensitivity analysis for the generalized Cholesky block downdating problemLinear and Multilinear Algebra, Vol. 70, No. 6 | 9 April 2020
- Backward Stepwise Elimination: Approximation Guarantee, a Batched GPU Algorithm, and Empirical InvestigationSN Computer Science, Vol. 2, No. 5 | 31 July 2021
- ArchiText: Interactive Hierarchical Topic ModelingIEEE Transactions on Visualization and Computer Graphics, Vol. 27, No. 9 | 1 Sep 2021
- Multiplicative perturbation bounds for the block Cholesky downdating problemInternational Journal of Computer Mathematics, Vol. 97, No. 12 | 8 December 2019
- Sensitivity analysis for the block Cholesky downdating problemInternational Journal of Computer Mathematics, Vol. 97, No. 6 | 13 May 2019
- A Data-Driven Realization of the Control-Performance-Oriented Process Monitoring SystemIEEE Transactions on Industrial Electronics, Vol. 67, No. 1 | 1 Jan 2020
- Updating QR factorization procedure for solution of linear least squares problem with equality constraintsJournal of Inequalities and Applications, Vol. 2017, No. 1 | 13 November 2017
- Modified Gram–Schmidt-based methods for block downdating the Cholesky factorizationJournal of Computational and Applied Mathematics, Vol. 235, No. 8 | 1 Feb 2011
- Online Sparse Gaussian Process Regression and Its ApplicationsIEEE Transactions on Image Processing, Vol. 20, No. 2 | 1 Feb 2011
- Survey of Anomaly Detection AlgorithmsSecurity of Self-Organizing Networks | 29 September 2010
- An Exact Least Trimmed Squares Algorithm for a Range of Coverage ValuesJournal of Computational and Graphical Statistics, Vol. 19, No. 1 | 1 Jan 2010
- Adaptive Kernel Classifiers Via Matrix Decomposition Updating for Biological Data AnalysisMachine Learning in Bioinformatics | 21 April 2008
- Systolic Algorithms and Architectures for High-Throughput Processing ApplicationsJournal of Signal Processing Systems, Vol. 53, No. 1-2 | 16 August 2007
- Parallel algorithms for downdating the least squares estimator of the regression modelParallel Computing, Vol. 34, No. 6-8 | 1 Jul 2008
- Online Sparse Matrix Gaussian Process Regression and Vision ApplicationsComputer Vision – ECCV 2008 | 1 Jan 2008
- Adaptive nonlinear discriminant analysis by regularized minimum squared errorsIEEE Transactions on Knowledge and Data Engineering, Vol. 18, No. 5 | 1 May 2006
- A new stable bidiagonal reduction algorithmLinear Algebra and its Applications, Vol. 397 | 1 Mar 2005
- Some Implementation Aspects of Sliding Window Least Squares AlgorithmsIFAC Proceedings Volumes, Vol. 33, No. 15 | 1 Jun 2000
- On the Optimality of the Backward Greedy Algorithm for the Subset Selection ProblemSIAM Journal on Matrix Analysis and Applications, Vol. 21, No. 3 | 31 July 2006
- Schur-Type Methods for Solving Least Squares Problems with Toeplitz StructureSIAM Journal on Scientific Computing, Vol. 22, No. 2 | 25 July 2006
- Complex ISAR imaging of maneuvering targets via the Capon estimatorIEEE Transactions on Signal Processing, Vol. 47, No. 5 | 1 May 1999
- An efficient rank detection procedure for modifying the ULV decompositionBIT Numerical Mathematics, Vol. 38, No. 4 | 1 Dec 1998
- Perturbation Analyses for the Cholesky Downdating ProblemSIAM Journal on Matrix Analysis and Applications, Vol. 19, No. 2 | 31 July 2006
- Sliding window adaptive fast QR and QR-lattice algorithmsIEEE Transactions on Signal Processing, Vol. 46, No. 11 | 1 Jan 1998
- Fast optimal and suboptimal algorithms for sparse solutions to linear inverse problemsProceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181) | 1 Jan 1998
- Perturbation and error analyses for block downdating of a Cholesky decompositionBIT Numerical Mathematics, Vol. 36, No. 2 | 1 Jun 1996
- An algorithm and a stability theory for downdating the ULV decompositionBIT Numerical Mathematics, Vol. 36, No. 1 | 1 Mar 1996
- Accurate downdating of a modified Gram-Schmidt QR decompositionBIT Numerical Mathematics, Vol. 36, No. 1 | 1 Mar 1996
- Downdating the Singular Value DecompositionSIAM Journal on Matrix Analysis and Applications, Vol. 16, No. 3 | 17 July 2006
- Fast residual computation for sliding window recursive least squares methodsSignal Processing, Vol. 45, No. 1 | 1 Jul 1995
- Two-way bidiagonalization scheme for downdating the singular-value decompositionLinear Algebra and its Applications, Vol. 222 | 1 Jun 1995
- Downdating the Rank-Revealing URV DecompositionSIAM Journal on Matrix Analysis and Applications, Vol. 16, No. 1 | 17 July 2006
- Implementation of QR up- and downdating on a massively parallel computerParallel Computing, Vol. 21, No. 1 | 1 Jan 1995
- On the stability of sequential updates and downdatesIEEE Transactions on Signal Processing, Vol. 43, No. 11 | 1 Jan 1995
- An Attainable Lower Bound for the Best Normal ApproximationSIAM Journal on Matrix Analysis and Applications, Vol. 15, No. 3 | 31 July 2006
- Dual-state systolic architecture for recursive least-squares updating and downdatingProceedings of 1994 IEEE Workshop on VLSI Signal Processing | 1 Jan 1994
View Options
Get Access
- Access via your Institution
- Questions about how to access this content? Contact SIAM at [email protected].