PRIMME_SVDS: A High-Performance Preconditioned SVD Solver for Accurate Large-Scale Computations
Abstract
The increasing number of applications requiring the solution of large-scale singular value problems has rekindled an interest in iterative methods for the SVD. Some promising recent advances in large-scale iterative methods are still plagued by slow convergence and accuracy limitations for computing smallest singular triplets. Furthermore, their current implementations in MATLAB cannot address the required large problems. Recently, we presented a preconditioned, two-stage method to effectively and accurately compute a small number of extreme singular triplets. In this research, we present a high-performance library, PRIMME_SVDS, that implements our hybrid method based on the state-of-the-art eigensolver package PRIMME for both largest and smallest singular values. PRIMME_SVDS fills a gap in production level software for computing the partial SVD, especially with preconditioning. The numerical experiments demonstrate its superior performance compared to other state-of-the-art software and its good parallel performance under strong and weak scaling.
Keywords
MSC codes
Get full access to this article
View all available purchase options and get full access to this article.
References
1.
H. Avron, A. Druinsky, and S. Toledo, Spectral Condition-Number Estimation of Large Sparse Matrices, preprint, ArXiv:1301.1107, 2013.
2.
J. Baglama and L. Reichel, Augmented implicitly restarted Lanczos bidiagonalization methods, SIAM J. Sci. Comput., 27 (2005), pp. 19--42.
3.
J. Baglama and L. Reichel, Restarted block Lanczos bidiagonalization methods, Numer. Algorithms, 43 (2006), pp. 251--272.
4.
J. Baglama and L. Reichel, An implicitly restarted block Lanczos bidiagonalization method using Leja shifts, BIT, 53 (2013), pp. 285--310.
5.
C. G. Baker and M. A. Heroux, Tpetra, and the use of generic programming in scientific computing, Sci. Program., 20 (2012), pp. 115--128.
6.
S. Balay, S. Abhyankar, M. Adams, J. Brown, P. Brune, K. Buschelman, V. Eijkhout, W. Gropp, D. Kaushik, M. Knepley, L. Curfman McInnes, K. Rupp, B. Smith, and H. Zhang, PETSc Users Manual Revision 3.5, Technical report, ANL-95/11 Rev 3.5, Argonne National Laboratory, Argonne, IL, 2014.
7.
M. W. Berry, Large-scale sparse singular value computations, Int. J. Supercomput. Appl., 6 (1992), pp. 13--49.
8.
J. Chen, L. Wu, K. Audhkhasi, B. Kingsbury, and B. Ramabhadran, Efficient one-vs-one kernel ridge regression for speech recognition, in 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, Piscataway, NJ, 2016, pp. 2454--2458.
9.
J. Cullum, R. A. Willoughby, and M. Lake, A Lanczos algorithm for computing singular values and vectors of large matrices, SIAM J. Sci. Stat. Comput., 4 (1983), pp. 197--215.
10.
S. Deerwester, S. T. Dumais, G. W. Furnas, T. K. Landauer, and R. Harshman, Indexing by latent semantic analysis, J. Amer. Soc. Inf. Sci., 41 (1990), pp. 391--407.
11.
J. J. Dongarra, Improving the accuracy of computed singular values, SIAM J. Sci. Stat. Comput., 4 (1983), pp. 712--719.
12.
G. Golub and W. Kahan, Calculating the singular values and pseudo-inverse of a matrix, J. SIAM Ser. B Numer. Anal., 2 (1965), pp. 205--224.
13.
G. H. Golub and C. F. Van Loan, An analysis of the total least squares problem, SIAM J. Numer. Anal., 17 (1980), pp. 883--893.
14.
G. H. Golub and C. F. Van Loan, Matrix Computations, 3rd ed., Johns Hopkins University Press, Baltimore, MD, 1996.
15.
G. H. Golub and Q. Ye, An inverse free preconditioned Krylov subspace method for symmetric generalized eigenvalue problems, SIAM J. Sci. Comput., 24 (2002), pp. 312--334.
16.
N. Halko, P. G. Martinsson, and J. A. Tropp, Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions, SIAM Rev., 53 (2011), pp. 217--288.
17.
V. E. Henson and U. M. Yang, BoomerAMG: A parallel algebraic multigrid solver and preconditioner, Appl. Numer. Math., 41 (2002), pp. 155--177.
18.
V. Hernandez and J. E. Roman, A robust and efficient parallel SVD solver based on restarted Lanczos bidiagonalization, Electron. Trans. Numer. Anal., 31 (2008), pp. 68--85.
19.
V. Hernandez, J. E. Roman, and V. Vidal, SLEPc: A scalable and flexible toolkit for the solution of eigenvalue problems, ACM Trans. Math. Software, 31 (2005), pp. 351--362.
20.
M. E. Hochstenbach, A Jacobi--Davidson type SVD method, SIAM J. Sci. Comput., 23 (2001), pp. 606--628.
21.
M. E. Hochstenbach, Harmonic and refined extraction methods for the singular value problem, with applications in least squares problems, BIT, 44 (2004), pp. 721--754.
22.
Z. Jia, Refined iterative algorithms based on Arnoldi's process for large unsymmetric eigenproblems, Linear Algebra Appl., 259 (1997), pp. 1--23.
23.
Z. Jia and D. Niu, An implicitly restarted refined bidiagonalization Lanczos method for computing a partial singular value decomposition, SIAM J. Matrix Anal. Appl., 25 (2003), pp. 246--265.
24.
Z. Jia and D. Niu, A refined harmonic Lanczos bidiagonalization method and an implicitly restarted algorithm for computing the smallest singular triplets of large matrices, SIAM J. Sci. Comput., 32 (2010), pp. 714--744.
25.
I. Jolliffe, Principal Component Analysis, Springer, New York, 2002.
26.
E. Kokiopoulou, C. Bekas, and E. Gallopoulos, Computing smallest singular triplets with implicitly restarted Lanczos bidiagonalization, Appl. Numer. Math., 49 (2004), pp. 39--61.
27.
R. M. Larsen, Lanczos Bidiagonalization with Partial Reorthogonalization, DAIMI Report Series 537, Aarhus University, Aarhus, Denmark, 1998.
28.
R. M. Larsen, Combining Implicit Restarts and Partial Reorthogonalization in Lanczos Bidiagonalization, presentation, SCCM, Stanford University, Palo Alto, CA, 2001.
29.
R. B. Lehoucq, D. C. Sorensen, and C. Yang, ARPACK Users' Guide: Solution of Large-Scale Eigenvalue Problems with Implicitly Restarted Arnoldi Methods, Software Environ. Tools, SIAM, Philadelphia, 1998.
30.
Q. Liang and Q. Ye, Computing singular values of large matrices with an inverse-free preconditioned Krylov subspace method, Electron. Trans. Numer. Anal., 42 (2014), pp. 197--221.
31.
F. Lin and W. W. Cohen, Power iteration clustering, in Proceedings of the 27th International Conference on Machine Learning (ICML-10), 2010, Haifa, Israel, Omnipress, Madison, 2010, pp. 655--662.
32.
M. Martone, Efficient multithreaded untransposed, transposed or symmetric sparse matrix--vector multiplication with the recursive sparse blocks format, Parallel Comput., 40 (2014), pp. 251--270.
33.
R. B. Morgan, Computing interior eigenvalues of large matrices, Linear Algebra Appl., 154 (1991), pp. 289--309.
34.
R. B. Morgan and M. Zeng, Harmonic projection methods for large non-symmetric eigenvalue problems, Numer. Linear Algebra Appl., 5 (1998), pp. 33--55.
35.
B. N. Parlett, The Symmetric Eigenvalue Problem, Prentice-Hall, Englewood Cliffs, NJ, 1980.
36.
B. Philippe and M. Sadkane, Computation of the fundamental singular subspace of a large matrix, Linear Algebra Appl., 257 (1997), pp. 77--104.
37.
B. Recht, M. Fazel, and P. A. Parrilo, Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization, SIAM Rev., 52 (2010), pp. 471--501.
38.
A. Stathopoulos, A case for a biorthogonal Jacobi--Davidson method: Restarting and correction equation, SIAM J. Matrix Anal. Appl., 24 (2002), pp. 238--259.
39.
A. Stathopoulos, Nearly optimal preconditioned methods for Hermitian eigenproblems under limited memory. Part I: Seeking one eigenvalue, SIAM J. Sci. Comput., 29 (2007), pp. 481--514.
40.
A. Stathopoulos and J. R. McCombs, PRIMME: Preconditioned iterative multimethod eigensolver: Methods and software description, ACM Trans. Math. Software, 37 (2010), 21.
41.
G. W. Stewart, Matrix Algorithms Volume 2: Eigensystems, Other Titles Appl. Math., SIAM, Philadelphia, 2001.
42.
L. N. Trefethen, Computation of pseudospectra, Acta Numer., 8 (1999), pp. 247--295.
43.
C. Williams and M. Seeger, Using the Nyström method to speed up kernel machines, in Proceedings of the 14th Annual Conference on Neural Information Processing Systems, MIT Press, Cambridge, MA, pp. 682--688.
44.
L. Wu, J. Laeuchli, V. Kalantzis, A. Stathopoulos, and E. Gallopoulos, Estimating the trace of the matrix inverse by interpolating from the diagonal of an approximate inverse, J. Comput. Phys., 326 (2016), pp. 828--844.
45.
L. Wu and A. Stathopoulos, A preconditioned hybrid SVD method for accurately computing singular triplets of large matrices, SIAM J. Sci. Comput., 37 (2015), pp. S365--S388.
46.
L. Wu, I. E. H. Yen, J. Chen, and R. Yan, Revisiting random binning feature: Fast convergence and strong parallelizability, in 22nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), ACM, New York, 2016, pp. 1265--1274.
47.
Z. Xianyi, W. Qian, and Z. Yunquan, Model-driven level 3 BLAS performance optimization on loongson 3a processor, in 2012 IEEE 18th International Conference on Parallel and Distributed Systems (ICPADS), IEEE, Piscataway, NJ, 2012, pp. 684--691.
Information & Authors
Information
Published In
SIAM Journal on Scientific Computing
Pages: S248 - S271
DOI: 10.1137/16M1082214
ISSN (online): 1095-7197
Copyright
© 2017, Society for Industrial and Applied Mathematics.
History
Submitted: 30 June 2016
Accepted: 23 January 2017
Published online: 26 October 2017
Keywords
MSC codes
Authors
Funding Information
Office of Science https://doi.org/10.13039/100006132 : DE-AC02-05CH11231
National Science Foundation https://doi.org/10.13039/100000001 : CCF 1218349, ACI S12-SSE 1440700
U.S. Department of Energy https://doi.org/10.13039/100000015 : DE-FC02-12ER41890
Metrics & Citations
Metrics
Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited By
- svds-C: A multi-thread C code for computing truncated singular value decompositionSoftwareX, Vol. 27 | 1 Sep 2024
- A hybrid algorithm for computing a partial singular value decomposition satisfying a given thresholdNumerical Algorithms, Vol. 20 | 6 August 2024
- Refined and refined harmonic Jacobi–Davidson methods for computing several GSVD components of a large regular matrix pairNumerical Algorithms, Vol. 23 | 31 July 2024
- Low-mode deflation for twisted-mass and RHMC reweighting in lattice QCDComputer Physics Communications, Vol. 300 | 1 Jul 2024
- Algorithm 1043: Faster Randomized SVD with Dynamic ShiftsACM Transactions on Mathematical Software, Vol. 50, No. 2 | 28 June 2024
- Observation of spin-splitting energies on sp–d exchange interactions tailored in colloidal CdSe/CdMnS core/shell nanoplatelets: an atomistic tight-binding modelPhysical Chemistry Chemical Physics, Vol. 26, No. 15 | 1 January 2024
- A compact Heart iteration for low-rank approximations of large matricesJournal of Computational and Applied Mathematics, Vol. 437 | 1 Feb 2024
- Optimizing shift selection in multilevel Monte Carlo for disconnected diagrams in lattice QCDComputer Physics Communications, Vol. 294 | 1 Jan 2024
- Strong Scaling of The SVD Algorithm For HPC Science: A Petsc-Based Approach2023 Winter Simulation Conference (WSC) | 10 Dec 2023
- Localization of Dirac modes in the SU(2) Higgs model at finite temperaturePhysical Review D, Vol. 108, No. 11 | 20 December 2023
- Electronic and optical properties of single substitutional Mn dopant in CdSe nanocrystals and CdSe/ZnSe core/shell nanocrystalsThe European Physical Journal Plus, Vol. 138, No. 8 | 26 August 2023
- Hybrid iterative refined restarted Lanczos bidiagonalization methodsNumerical Algorithms, Vol. 92, No. 2 | 6 July 2022
- Two Harmonic Jacobi–Davidson Methods for Computing a Partial Generalized Singular Value Decomposition of a Large Matrix PairJournal of Scientific Computing, Vol. 93, No. 2 | 16 September 2022
- A Robust Algebraic Multilevel Domain Decomposition Preconditioner for Sparse Symmetric Positive Definite MatricesSIAM Journal on Scientific Computing, Vol. 44, No. 4 | 18 August 2022
- Tunable electronic, optical and magnetic characteristics in Mn-doped inverted type-I ZnSe/CdSe core/shell nanocrystals: Atomistic tight-binding modelMaterials Science in Semiconductor Processing, Vol. 147 | 1 Aug 2022
- Periodically driven perturbed CFTs: The sine-Gordon modelNuclear Physics B, Vol. 976 | 1 Mar 2022
- Numerical Study of Low Rank Approximation Methods for Mechanics Data and Its AnalysisJournal of Scientific Computing, Vol. 87, No. 1 | 24 February 2021
- Approaching the self-dual point of the sinh-Gordon modelJournal of High Energy Physics, Vol. 2021, No. 1 | 5 January 2021
- A multigrid accelerated eigensolver for the Hermitian Wilson–Dirac operator in lattice QCDComputer Physics Communications, Vol. 258 | 1 Jan 2021
- Remarks on strange-quark simulations with Wilson fermionsPhysical Review D, Vol. 102, No. 7 | 27 October 2020
- Efficient toolkit implementing best practices for principal component analysis of population genetic dataBioinformatics, Vol. 36, No. 16 | 16 May 2020
- clMF: A fine-grained and portable alternating least squares algorithm for parallel matrix factorizationFuture Generation Computer Systems, Vol. 108 | 1 Jul 2020
- A cross-product approach for low-rank approximations of large matricesJournal of Computational and Applied Mathematics, Vol. 369 | 1 May 2020
- Native point defects in mono and bilayer phosphorenePhysical Review Materials, Vol. 4, No. 5 | 29 May 2020
- Basic SSASingular Spectrum Analysis for Time Series | 24 November 2020
- Classification of Occluded Images for Large-Scale Datasets With Numerous Occlusion PatternsIEEE Access, Vol. 8 | 1 Jan 2020
- Computing the smallest singular triplets of a large matrixResults in Applied Mathematics, Vol. 3 | 1 Oct 2019
- Scalable Global Alignment Graph Kernel Using Random FeaturesProceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining | 25 July 2019
- TRPL+K: Thick-Restart Preconditioned Lanczos+K Method for Large Symmetric Eigenvalue ProblemsSIAM Journal on Scientific Computing, Vol. 41, No. 2 | 9 April 2019
- Sketching for Principal Component RegressionSIAM Journal on Matrix Analysis and Applications, Vol. 40, No. 2 | 30 April 2019
- On Inner Iterations of Jacobi--Davidson Type Methods for Large SVD ComputationsSIAM Journal on Scientific Computing, Vol. 41, No. 3 | 16 May 2019
- Scalable Spectral Clustering Using Random Binning FeaturesProceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining | 19 July 2018
View Options
Get Access
- Access via your Institution
- Questions about how to access this content? Contact SIAM at [email protected].