Low Rank Approximation of a Sparse Matrix Based on LU Factorization with Column and Row Tournament Pivoting
Abstract
In this paper we present an algorithm for computing a low rank approximation of a sparse matrix based on a truncated LU factorization with column and row permutations. We present various approaches for determining the column and row permutations that show a trade-off between speed versus deterministic/probabilistic accuracy. We show that if the permutations are chosen by using tournament pivoting based on QR factorization, then the obtained truncated LU factorization with column/row tournament pivoting, LU_CRTP, satisfies bounds on the singular values which have similarities with the ones obtained by a communication avoiding rank revealing QR factorization. Experiments on challenging matrices show that LU_CRTP provides a good low rank approximation of the input matrix and it is less expensive than the rank revealing QR factorization in terms of computational and memory usage costs, while also minimizing the communication cost. We also compare the computational complexity of our algorithm with randomized algorithms and show that for sparse matrices and high enough but still modest accuracies, our approach is faster.
Keywords
MSC codes
Get full access to this article
View all available purchase options and get full access to this article.
References
1.
E. Anderson, Z. Bai, C. Bischof, S. Blackford, J. W. Demmel, J. Dongarra, J. D. Croz, A. Greenbaum, S. Hammarling, A. McKenney, and D. Sorensen, LAPACK Users' Guide, SIAM, Philadelphia, 1999.
2.
G. Ballard, J. Demmel, O. Holtz, and O. Schwartz, Minimizing communication in numerical linear algebra, SIAM J. Matrix Anal. Appl., 32 (2011), pp. 866--901.
3.
C. H. Bischof, A parallel QR factorization algorithm with controlled local pivoting, SIAM J. Sci. and Stat. Comput., 12 (1991), pp. 36--57.
4.
P. A. Businger and G. H. Golub, Linear least squares solutions by Householder transformations, Numer. Math., 7 (1965), pp. 269--276.
5.
S. Chandrasekaran and I. C. F. Ipsen, On rank-revealing factorisations, SIAM. J. Matrix Anal. Appl., 15 (1994), pp. 592--622.
6.
K. L. Clarkson and D. P. Woodruff, Low rank approximation and regression in input sparsity time, in Proceedings of the 45th Annual ACM Symposium on Theory of Computing, 2013, pp. 81--90.
7.
J. K. Cullum and R. A. Willoughby, Lanczos Algorithms for Large Symmetric Eigenvalue Computations, Vol. I: Theory, SIAM, Philadelphia, 2002.
8.
T. Davis, Algorithm 915, SuiteSparseQR: Multifrontal multithreaded rank-revealing sparse QR factorization, ACM Trans. Math. Software, 38 (2011), pp. 8:1--8:22.
9.
T. A. Davis, J. R. Gilbert, S. I. Larimore, and E. G. Ng, A column approximate minimum degree ordering algorithm, ACM Trans. Math. Software, 30 (2004), pp. 353--376.
10.
T. A. Davis and Y. Hu, The University of Florida Sparse Matrix Collection, ACM Trans. Math. Software, 38 (2011), pp. 1--25. .
11.
J. Demmel, L. Grigori, M. Gu, and H. Xiang, Communication-avoiding rank-revealing QR decomposition, SIAM J. Matrix Anal. Appl., 36 (2015), pp. 55--89.
12.
J. Demmel, L. Grigori, M. Gu, and H. Xiang, LU with Tournament Pivoting for Nearly Singular Matrices, Technical report, Inria and UC Berkeley, 2018, in preparation.
13.
J. W. Demmel, L. Grigori, M. Hoemmen, and J. Langou, Communication-optimal parallel and sequential QR and LU factorizations, SIAM J. Sci. Comput., 34 (2012), pp. 206--239.
14.
J. W. Demmel, N. J. Higham, and R. Schreiber, Block LU factorization, Numer. Linear Algebra Appl., 2 (1995), pp. 173--190.
15.
L. Foster, San Jose State University Singular Matrix Database, http://www.math.sjsu.edu/singular/matrices/.
16.
A. George, Nested Dissection of a Regular Finite Element Mesh, SIAM J. Numer. Anal., 10 (1973), pp. 345--363.
17.
A. George and J. W.-H. Liu, Computer Solution of Large Sparse Positive Definite Systems, Prentice-Hall, Englewood Cliffs, NJ, 1981.
18.
A. George, J. W.-H. Liu, and E. G.-Y. Ng, A data structure for sparse QR and LU factors, SIAM J. Sci. and Stat. Comput., 9 (1988), pp. 100--121.
19.
J. R. Gilbert, E. G. Ng, and B. W. Peyton, Separators and structure prediction in sparse orthogonal factorization, Linear Algebra Appl., 262 (1997).
20.
G. H. Golub, Numerical methods for solving linear least squares problems, Numer. Math., 7 (1965), pp. 206--216.
21.
G. H. Golub and C. F. Van Loan, Matrix Computations, 4th ed., The Johns Hopkins University Press, Baltimore, MD, 2013.
22.
Z. N. Goreinov SA, Tyrtyshnikov EE, A theory of pseudoskeleton approximations, Linear Algebra Appl., 261 (1997), pp. 1--21.
23.
L. Grigori, S. Cayrols, and J. Demmel, Low Rank Approximation of a Sparse Matrix Based on LU Factorization with Column and Row Tournament Pivoting, Technical report 8910, Inria, 2016.
24.
L. Grigori, J. W. Demmel, and H. Xiang, CALU: A communication optimal LU factorization algorithm, SIAM J. Matrix Anal. Appl., 32 (2011), pp. 1317--1350.
25.
M. Gu and S. C. Eisenstat, Efficient algorithms for computing a strong rank-revealing QR factorization, SIAM J. Sci. Comput., 17 (1996), pp. 848--869.
26.
N. Halko, P. G. Martinsson, and J. A. Tropp, Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions, SIAM Rev., 53 (2011), pp. 217--288.
27.
P. C. Hansen, Regularization tools version 4.1 for MATLAB 7.3.
28.
Y. P. Hong and C.-T. Pan, Rank-revealing QR factorizations and the singular value decomposition, Math. Comp., 58 (1992), pp. 213--232.
29.
G. Karypis and V. Kumar, A Software Package for Partitioning Unstructured Graphs, Partitioning Meshes and Computing Fill-Reducing Orderings of Sparse Matrices---Version 4.0, http://glaros.dtc.umn.edu/gkhome/views/metis, 1998.
30.
A. Khabou, J. W. Demmel, L. Grigori, and M. Gu, Communication avoiding LU factorization with panel rank revealing pivoting, SIAM J. Matrix Anal. Appl., 34 (2013), pp. 1401--1429.
31.
R. L. Lipton and R. E. Tarjan, A separator theorem for planar graphs, SIAM J. Appl. Math., 36 (1979), pp. 177--189.
32.
M. W. Mahoney, Randomized algorithms for matrices and data, Found. Trends Mach. Learn., 3 (2011), pp. 123--224.
33.
L. Miranian and M. Gu, Strong rank revealing LU factorizations, Linear Algebra Appl., (2003), pp. 1--16.
34.
C.-T. Pan, On the existence and computation of rank-revealing LU factorizations, Linear Algebra Appl., 316 (2000), pp. 199--222.
35.
Y. Saad, Numerical Methods for Large Eigenvalue Problems, 2nd ed., SIAM, Philadelphia, 2011.
36.
G. Stewart, Four algorithms for the efficient computation of truncated QR approximations to a sparse matrix, Numer. Math., 83 (1999), pp. 313--323.
37.
G. W. Stewart, The QLP approximation to the singular value decomposition, SIAM J. Sci. Comput., 20 (1999), pp. 1336--1348.
Information & Authors
Information
Published In
SIAM Journal on Scientific Computing
Pages: C181 - C209
DOI: 10.1137/16M1074527
ISSN (online): 1095-7197
Copyright
© 2018, Society for Industrial and Applied Mathematics.
History
Submitted: 9 May 2016
Accepted: 11 August 2016
Published online: 15 March 2018
Keywords
MSC codes
Authors
Funding Information
Defense Advanced Research Projects Agency https://doi.org/10.13039/100000185 : HR0011-12-2-0016
Funding Information
Office of Science https://doi.org/10.13039/100006132 : DE-SC0008700, DE-SC0010200, AC02-05CH11231
Funding Information
Horizon 2020 Framework Programme https://doi.org/10.13039/100010661 : 671633
Metrics & Citations
Metrics
Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited By
- Efficient algorithms for computing rank‐revealing factorizations on a GPUNumerical Linear Algebra with Applications, Vol. 30, No. 6 | 2 June 2023
- An Improved Analysis and Unified Perspective on Deterministic and Randomized Low-Rank Matrix ApproximationSIAM Journal on Matrix Analysis and Applications, Vol. 44, No. 2 | 11 May 2023
- Accuracy vs. Cost in Parallel Fixed-Precision Low-Rank Approximations of Sparse Matrices2022 IEEE International Parallel and Distributed Processing Symposium (IPDPS) | 1 May 2022
- Low-rank and sparse matrix factorization with prior relations for recommender systemsApplied Intelligence, Vol. 51, No. 6 | 13 November 2020
- Linear-time CUR approximation of BEM matricesJournal of Computational and Applied Mathematics, Vol. 368 | 1 Apr 2020
- Numerical algorithms for high-performance computational sciencePhilosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 378, No. 2166 | 20 January 2020
- Temporal‐aware and sparsity‐tolerant hybrid collaborative recommendation method with privacy preservationConcurrency and Computation: Practice and Experience, Vol. 32, No. 2 | 25 July 2019
- Two-Stage Column Block Parallel LU Factorization AlgorithmIEEE Access, Vol. 8 | 1 Jan 2020
- View's dependency and low‐rank background‐guided compressed sensing for multi‐view image joint reconstructionIET Image Processing, Vol. 13, No. 12 | 17 September 2019
- Block Randomized Optimization for Adaptive Hypergraph Learning2019 IEEE International Conference on Image Processing (ICIP) | 1 Sep 2019
- ALORA: Affine Low-Rank ApproximationsJournal of Scientific Computing, Vol. 79, No. 2 | 28 November 2018
- A Heterogeneous Parallel LU Factorization Algorithm Based on a Basic Column Block Uniform Allocation StrategyMathematical Problems in Engineering, Vol. 2019 | 25 Feb 2019
- Dynamic Scheduling Strategy for Block Parallel Cholesky Factorization Based on Activity on Edge NetworkIEEE Access, Vol. 7 | 1 Jan 2019
- Low-Rank and Sparse Matrix Factorization for Scientific Paper Recommendation in Heterogeneous NetworkIEEE Access, Vol. 6 | 1 Jan 2018