Robust CUR Decomposition: Theory and Imaging Applications
Abstract
This paper considers the use of robust principal component analysis (RPCA) in a CUR decomposition framework and applications thereof. Our main algorithms produce a robust version of column-row factorizations of matrices $D=L+S$, where $L$ is low-rank and $S$ contains sparse outliers. These methods yield interpretable factorizations at low computational cost and provide new CUR decompositions that are robust to sparse outliers, in contrast to previous methods. We consider two key imaging applications of RPCA: video foreground-background separation and face modeling. This paper examines the qualitative behavior of our robust CUR decompositions on the benchmark videos and face datasets and finds that our method works as well as standard RPCA while being significantly faster. Additionally, we consider hybrid randomized and deterministic sampling methods which produce a compact CUR decomposition of a given matrix and apply this to video sequences to produce canonical frames thereof.
Keywords
MSC codes
Get full access to this article
View all available purchase options and get full access to this article.
References
1.
A. Aldroubi, K. Hamm, A. B. Koku, and A. Sekmen, CUR decompositions, similarity matrices, and subspace clustering, Front. Appl. Math. Stat., 4 (2019), p. 65.
2.
A. Aldroubi, A. Sekmen, A. B. Koku, and A. F. Cakmak, Similarity matrix framework for data from union of subspaces, Appl. Comput. Harmon. Anal., 45 (2018), pp. 425--435.
3.
H. Avron and C. Boutsidis, Faster subset selection for matrices and applications, SIAM J. Matrix Anal. Appl., 34 (2013), pp. 1464--1499.
4.
J. J. Bartholdi, III, A good submatrix is hard to find, Oper. Res. Lett., 1 (1982), pp. 190--193.
5.
A. Bhaskara, A. Rostamizadeh, J. Altschuler, M. Zadimoghaddam, T. Fu, and V. Mirrokni, Greedy column subset selection: New bounds and distributed algorithms, in Proceedings of the 33rd International Conference on Machine Learning, 2016, pp. 2539--2548.
6.
C. Boutsidis, P. Drineas, and M. Magdon-Ismail, Near-optimal column-based matrix reconstruction, SIAM J. Comput., 43 (2014), pp. 687--717.
7.
C. Boutsidis and D. P. Woodruff, Optimal CUR matrix decompositions, SIAM J. Comput., 46 (2017), pp. 543--589.
8.
T. Bouwmans, Subspace learning for background modeling: A survey, Recent Patents Comput. Sci., 2 (2009), pp. 223--234.
9.
T. Bouwmans, S. Javed, H. Zhang, Z. Lin, and R. Otazo, On the applications of robust PCA in image and video processing, Proc. IEEE, 106 (2018), pp. 1427--1457.
10.
H. Cai, Accelerating Truncated Singular-Value Decomposition: A Fast and Provable Method for Robust Principal Component Analysis, PhD thesis, University of Iowa, 2018.
11.
H. Cai, J.-F. Cai, T. Wang, and G. Yin, Accelerated structured alternating projections for robust spectrally sparse signal recovery, IEEE Trans. Signal Process., 69 (2021), pp. 809--821.
12.
H. Cai, J.-F. Cai, and K. Wei, Accelerated alternating projections for robust principal component analysis, J. Mach. Learn. Res., 20 (2019), pp. 685--717.
13.
H. Cai, K. Hamm, L. Huang, J. Li, and T. Wang, Rapid robust principal component analysis: CUR accelerated inexact low rank estimation, IEEE Signal Process. Lett., 28 (2021), pp. 116--120.
14.
E. J. Candès, X. Li, Y. Ma, and J. Wright, Robust principal component analysis?, J. ACM, 58 (2011), pp. 1--37.
15.
V. Chandrasekaran, S. Sanghavi, P. A. Parrilo, and A. S. Willsky, Rank-sparsity incoherence for matrix decomposition, SIAM J. Optim., 21 (2011), pp. 572--596.
16.
J. Chiu and L. Demanet, Sublinear randomized algorithms for skeleton decompositions, SIAM J. Matrix Anal. Appl., 34 (2013), pp. 1361--1383.
17.
A. Çivril, Column subset selection problem is UG-hard, J. Comput. System Sci., 80 (2014), pp. 849--859.
18.
M. Dereziński and M. K. Warmuth, Reverse iterative volume sampling for linear regression, J. Mach. Learn. Res., 19 (2018), pp. 853--891.
19.
A. Deshpande and L. Rademacher, Efficient volume sampling for row/column subset selection, in Proceedings of the 51st Annual IEEE Symposium on Foundations of Computer Science, IEEE, 2010, pp. 329--338.
20.
P. Drineas, R. Kannan, and M. W. Mahoney, Fast Monte Carlo algorithms for matrices III: Computing a compressed approximate matrix decomposition, SIAM J. Comput., 36 (2006), pp. 184--206.
21.
P. Drineas, M. Magdon-Ismail, M. W. Mahoney, and D. P. Woodruff, Fast approximation of matrix coherence and statistical leverage, J. Mach. Learn. Res., 13 (2012), pp. 3475--3506.
22.
P. Drineas, M. W. Mahoney, and S. Muthukrishnan, Relative-error CUR matrix decompositions, SIAM J. Matrix Anal. Appl., 30 (2008), pp. 844--881.
23.
E. Elhamifar and R. Vidal, Sparse subspace clustering: Algorithm, theory, and applications, IEEE Trans. Pattern Anal. Mach. Intell., 35 (2013), pp. 2765--2781.
24.
A. Georghiades, P. Belhumeur, and D. Kriegman, From few to many: Illumination cone models for face recognition under variable lighting and pose, IEEE Trans. Pattern Anal. Mach. Intelligence, 23 (2001), pp. 643--660.
25.
S. A. Goreinov, I. V. Oseledets, D. V. Savostyanov, E. E. Tyrtyshnikov, and N. L. Zamarashkin, How to find a good submatrix, in Matrix Methods: Theory, Algorithms and Applications: Dedicated to the Memory of Gene Golub, World Scientific, River Edge, NJ, 2010, pp. 247--256.
26.
S. A. Goreĭnov, E. E. Tyrtyshnikov, and N. L. Zamarashkin, A theory of pseudoskeleton approximations, Linear Algebra Appl., 261 (1997), pp. 1--21.
27.
S. A. Goreĭnov, N. L. Zamarashkin, and E. E. Tyrtyshnikov, Pseudo-skeleton approximations, Dokl. Akad. Nauk, 343 (1995), pp. 151--152.
28.
S. A. Goreĭnov, N. L. Zamarashkin, and E. E. Tyrtyshnikov, Pseudo-skeleton approximations by matrices of maximal volume, Math. Notes, 62 (1997), pp. 515--519.
29.
D. Gross and V. Nesme, Note on Sampling Without Replacing from a Finite Collection of Matrices, arXiv preprint, arXiv:1001.2738 [CS.IT], 2010.
30.
M. Gu and S. C. Eisenstat, Downdating the singular value decomposition, SIAM J. Matrix Anal. Appl., 16 (1995), pp. 793--810.
31.
B. D. Haeffele, C. You, and R. Vidal, A critique of self-expressive deep subspace clustering, International Conference on Learning Representations, 2021, https://openreview.net/pdf?id=FOyuZ26emy.
32.
K. Hamm and L. Huang, Perturbations of CUR decompositions, SIAM J. Matrix Anal. Appl., 42 (2021), pp. 351--375.
33.
K. Hamm and L. Huang, Perspectives on CUR decompositions, Appl. Comput. Harmon. Anal., 48 (2020), pp. 1088--1099.
34.
W.-D. Jang, C. Lee, and C.-S. Kim, Primary object segmentation in videos via alternate convex optimization of foreground and background distributions, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 696--704.
35.
X. Li and Y. Pang, Deterministic column-based matrix decomposition, IEEE Trans. Knowledge and Data Engrg., 22 (2010), pp. 145--149.
36.
G. Liu, Z. Lin, S. Yan, J. Sun, Y. Yu, and Y. Ma, Robust recovery of subspace structures by low-rank representation, IEEE Trans. Pattern Anal. Mach. Intell., 35 (2012), pp. 171--184.
37.
M. W. Mahoney and P. Drineas, CUR matrix decompositions for improved data analysis, Proc. Natl. Acad. Sci. USA, 106 (2009), pp. 697--702.
38.
A. Mikhalev and I. V. Oseledets, Rectangular maximum-volume submatrices and their applications, Linear Algebra Appl., 538 (2018), pp. 187--211.
39.
P. Netrapalli, U. Niranjan, S. Sanghavi, A. Anandkumar, and P. Jain, Non-convex robust PCA, Adv. Neural Inf. Process. Syst., 27 (2014), pp. 1107--1115.
40.
I. Oseledets and E. Tyrtyshnikov, Tt-cross approximation for multidimensional arrays, Linear Algebra Appl., 432 (2010), pp. 70--88.
41.
A. Osinsky, Rectangular Maximum Volume and Projective Volume Search Algorithms, arXiv preprint, arXiv:1809.02334 [math.NA], 2018.
42.
A. Osinsky and N. L. Zamarashkin, Pseudo-skeleton approximations with better accuracy estimates, Linear Algebra Appl., 537 (2018), pp. 221--249.
43.
Y. Shitov, Column subset selection is NP-complete, Linear Algebra Appl., 610 (2021), pp. 52--58.
44.
D. C. Sorensen and M. Embree, A DEIM induced CUR factorization, SIAM J. Sci. Comput., 38 (2016), pp. A1454--A1482.
45.
G. W. Stewart, On the perturbation of pseudo-inverses, projections and linear least squares problems, SIAM Rev., 19 (1977), pp. 634--662.
46.
S.Wang and Z.Zhang, Improving CUR matrix decomposition and the Nyström approximation via adaptive sampling, J. Mach. Learn. Res., 14 (2013), pp. 2729--2769.
47.
J. A. Tropp, Column subset selection, matrix factorization, and eigenvalue optimization, in Proceedings of the 20th Annual ACM-SIAM Symposium on Discrete Algorithms, SIAM, 2009, pp. 978--986.
48.
J. A. Tropp, Improved analysis of the subsampled randomized Hadamard transform, Adv. Adapt. Data Anal., 3 (2011), pp. 115--126.
49.
R. Vidal, Subspace clustering, IEEE Signal Process Mag., 28 (2011), pp. 52--68.
50.
S. Voronin and P.-G. Martinsson, Efficient algorithms for CUR and interpolative matrix decompositions, Adv. Comput. Math., 43 (2017), pp. 495--516.
51.
J. Wright, A. Y. Yang, A. Ganesh, S. S. Sastry, and Y. Ma, Robust face recognition via sparse representation, IEEE Trans. Pattern Anal. Mach. Intell. Trans. Inform., 31 (2008), pp. 210--227.
52.
H. Xu, C. Caramanis, and S. Sanghavi, Robust PCA via outlier pursuit, IEEE Trans. Inform. Theory, 58 (2012), pp. 3047--3064.
53.
X. Yi, D. Park, Y. Chen, and C. Caramanis, Fast algorithms for robust PCA via gradient descent, in Proceedings of the 30th International Conference on Neural Information Processing Systems, 2016, pp. 4152--4160.
Information & Authors
Information
Published In
Copyright
© 2021, Society for Industrial and Applied Mathematics.
History
Submitted: 28 December 2020
Accepted: 20 July 2021
Published online: 18 October 2021
Keywords
MSC codes
Authors
Funding Information
Army Research Office https://doi.org/10.13039/100000183 : W911NF-20-1-0076
Division of Mathematical Sciences https://doi.org/10.13039/100000121 : DMS-1740325, DMS-2011140
Metrics & Citations
Metrics
Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited By
- A Sublinear-Time Randomized Algorithm for Column and Row Subset Selection Based on Strong Rank-Revealing QR FactorizationsSIAM Journal on Matrix Analysis and Applications, Vol. 46, No. 1 | 3 January 2025
- Latent space active learning with message passing neural network: The case of HfO2Physical Review Materials, Vol. 8, No. 10 | 11 October 2024
- Randomized GCUR decompositionsAdvances in Computational Mathematics, Vol. 50, No. 4 | 18 July 2024
- Quantum State Tomography for Matrix Product Density OperatorsIEEE Transactions on Information Theory, Vol. 70, No. 7 | 1 Jul 2024
- CURSOR: Scalable Mixed-Order Hypergraph Matching with CUR Decomposition2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) | 16 Jun 2024
- The discrete empirical interpolation method in class identification and data summarizationWIREs Computational Statistics, Vol. 16, No. 3 | 5 May 2024
- CPQR-based randomized algorithms for generalized CUR decompositionsComputational and Applied Mathematics, Vol. 43, No. 3 | 28 March 2024
- Robust Tensor CUR Decompositions: Rapid Low-Tucker-Rank Tensor Recovery with Sparse CorruptionsSIAM Journal on Imaging Sciences, Vol. 17, No. 1 | 25 January 2024
- On the Robustness of Cross-Concentrated Sampling for Matrix Completion2024 58th Annual Conference on Information Sciences and Systems (CISS) | 13 Mar 2024
- CUR and Generalized CUR Decompositions of Quaternion Matrices and their ApplicationsNumerical Functional Analysis and Optimization, Vol. 45, No. 3 | 21 February 2024
- The Improved Wavelet Denoising Scheme Based on Robust Principal Component Analysis for Distributed Fiber Acoustic SensorIEEE Sensors Journal, Vol. 23, No. 19 | 1 Oct 2023
- Coseparable Nonnegative Matrix FactorizationSIAM Journal on Matrix Analysis and Applications, Vol. 44, No. 3 | 15 September 2023
- Matrix Completion With Cross-Concentrated Sampling: Bridging Uniform Sampling and CUR SamplingIEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 45, No. 8 | 1 Aug 2023
- Fast Dual-Graph Regularized Background Foreground Separation2023 International Conference on Sampling Theory and Applications (SampTA) | 10 Jul 2023
- Structured Gradient Descent for Fast Robust Low-Rank Hankel Matrix CompletionSIAM Journal on Scientific Computing, Vol. 45, No. 3 | 8 June 2023
- Efficient Robust Principal Component Analysis via Block Krylov Iteration and CUR Decomposition2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) | 1 Jun 2023
- Matrix Factorization Techniques in Machine Learning, Signal Processing, and StatisticsMathematics, Vol. 11, No. 12 | 12 June 2023
- Improving empirical efficiency of CUR decompositionThe Journal of Supercomputing, Vol. 79, No. 8 | 12 January 2023
- Generalized pseudoskeleton decompositionsLinear Algebra and its Applications, Vol. 664 | 1 May 2023
- Energy-Based Adaptive CUR Matrix DecompositionIEEE Access, Vol. 11 | 1 Jan 2023
- Tensor Robust CUR for Compression and Denoising of Hyperspectral DataIEEE Access, Vol. 11 | 1 Jan 2023
- A Novel False Data Injection Attack Formulation Based on CUR Low-Rank Decomposition MethodIEEE Transactions on Smart Grid, Vol. 13, No. 6 | 1 Nov 2022
- Fast Robust Tensor Principal Component Analysis via Fiber CUR Decomposition2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW) | 1 Oct 2021
- Cross Tensor Approximation Methods for Compression and Dimensionality ReductionIEEE Access, Vol. 9 | 1 Jan 2021
View Options
- Access via your Institution
- Questions about how to access this content? Contact SIAM at [email protected].