Abstract.

We study the tensor robust principal component analysis (TRPCA) problem, a tensorial extension of matrix robust principal component analysis, which aims to split the given tensor into an underlying low-rank component and a sparse outlier component. This work proposes a fast algorithm, called robust tensor CUR decompositions (RTCUR), for large-scale nonconvex TRPCA problems under the Tucker rank setting. RTCUR is developed within a framework of alternating projections that projects between the set of low-rank tensors and the set of sparse tensors. We utilize the recently developed tensor CUR decomposition to substantially reduce the computational complexity in each projection. In addition, we develop four variants of RTCUR for different application settings. We demonstrate the effectiveness and computational advantages of RTCUR against state-of-the-art methods on both synthetic and real-world datasets.

Keywords

  1. tensor CUR decomposition
  2. robust tensor principal component analysis
  3. low-rank tensor recovery
  4. outlier detection

MSC codes

  1. 68Q25
  2. 68W25
  3. 68W20
  4. 68P20

Get full access to this article

View all available purchase options and get full access to this article.

References

1.
H. Abdi and L. J. Williams, Principal component analysis, Wiley Interdiscip. Rev. Comput. Stat., 2 (2010), pp. 433–459.
2.
G. Bergqvist and E. G. Larsson, The higher-order singular value decomposition: Theory and an application [lecture notes], IEEE Signal Process. Mag., 27 (2010), pp. 151–154.
3.
T. Bouwmans, S. Javed, H. Zhang, Z. Lin, and R. Otazo, On the applications of robust PCA in image and video processing, Proc. IEEE, 106 (2018), pp. 1427–1457.
4.
T. Bouwmans, L. Maddalena, and A. Petrosino, Scene background initialization: A taxonomy, Pattern Recognit. Lett., 96 (2017), pp. 3–11.
5.
H. Cai, J.-F. Cai, T. Wang, and G. Yin, Accelerated structured alternating projections for robust spectrally sparse signal recovery, IEEE Trans. Signal Process., 69 (2021), pp. 809–821.
6.
H. Cai, J.-F. Cai, and K. Wei, Accelerated alternating projections for robust principal component analysis, J. Mach. Learn. Res., 20 (2019), pp. 685–717.
7.
H. Cai, J.-F. Cai, and J. You, Structured gradient descent for fast robust low-rank Hankel matrix completion, SIAM J. Sci. Comput., 45 (2023), pp. 1172–1198.
8.
H. Cai, Z. Chao, L. Huang, and D. Needell, Fast robust tensor principal component analysis via Fiber CUR decomposition, in Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, IEEE, New York, 2021, pp. 189–197.
9.
H. Cai, K. Hamm, L. Huang, J. Li, and T. Wang, Rapid robust principal component analysis: CUR accelerated inexact low rank estimation, IEEE Signal Process. Lett., 28 (2020), pp. 116–120.
10.
H. Cai, K. Hamm, L. Huang, and D. Needell, Mode-wise tensor decompositions: Multi-dimensional generalizations of CUR decompositions, J. Mach. Learn. Res., 22 (2021), pp. 1–36.
11.
H. Cai, K. Hamm, L. Huang, and D. Needell, Robust CUR decomposition: Theory and imaging applications, SIAM J. Imaging Sci., 14 (2021), pp. 1472–1503.
12.
H. Cai, J. Liu, and W. Yin, Learned Robust PCA: A Scalable Deep Unfolding Approach for High-Dimensional Outlier Detection, preprint, arXiv:2110.05649, 2021.
13.
J.-F. Cai, J. Li, and D. Xia, Generalized low-rank plus sparse tensor estimation by fast Riemannian optimization, J. Amer. Statist. Assoc., (2022), https://doi.org/10.1080/01621459.2022.2063131.
14.
C. F. Caiafa and A. Cichocki, Generalizing the column–row matrix decomposition to multi-way arrays, Linear Algebra Appl., 433 (2010), pp. 557–573.
15.
E. Candes and J. Romberg, Sparsity and incoherence in compressive sampling, Inverse Problems, 23 (2007), 969.
16.
E. J. Candès, X. Li, Y. Ma, and J. Wright, Robust principal component analysis?, J. ACM, 58 (2011), pp. 1–37.
17.
J. D. Carroll and J.-J. Chang, Analysis of individual differences in multidimensional scaling via an n-way generalization of “Eckart-Young” decomposition, Psychometrika, 35 (1970), pp. 283–319.
18.
J. Chambua, Z. Niu, A. Yousif, and J. Mbelwa, Tensor factorization method based on review text semantic similarity for rating prediction, Expert Syst. Appl., 114 (2018), pp. 629–638.
19.
V. Chandrasekaran, S. Sanghavi, P. A. Parrilo, and A. S. Willsky, Rank-sparsity incoherence for matrix decomposition, SIAM J. Optim., 21 (2011), pp. 572–596.
20.
Z. Chao, L. Huang, and D. Needell, HOSVD-based algorithm for weighted tensor completion, J. Imaging, 7 (2021), p. 110.
21.
J. Chen and Y. Saad, On the tensor SVD and the optimal low rank orthogonal approximation of tensors, SIAM J. Matrix Anal. Appl., 30 (2009), pp. 1709–1734.
22.
L. De Lathauwer, B. De Moor, and J. Vandewalle, On the best rank-1 and rank-(\(r_1,\ldots, r_n\)) approximation of higher-order tensors, SIAM J. Matrix Anal. Appl., 21 (2000), pp. 1324–1342.
23.
H. Dong, T. Tong, C. Ma, and Y. Chi, Fast and provable tensor robust principal component analysis via scaled gradient descent, Inf. Inference, 12 (2023), iaad019.
24.
S. A. Goreĭnov, N. L. Zamarashkin, and E. E. Tyrtyshnikov, Pseudo-skeleton approximations, Dokl. Akad. Nauk, 343 (1995), pp. 151–152.
25.
Q. Gu, H. Gui, and J. Han, Robust tensor decomposition with gross corruption, Adv. Neural Inform. Process. Syst., 27 (2014), pp. 1422–1430.
26.
K. Hamm and L. Huang, Perspectives on CUR decompositions, Appl. Comput. Harmon. Anal., 48 (2020), pp. 1088–1099.
27.
K. Hamm and L. Huang, Stability of sampling for CUR decompositions, Found. Data Sci., 2 (2020), 83.
28.
K. Hamm, M. Meskini, and H. Cai, Riemannian CUR decompositions for robust principal component analysis, in Topological, Algebraic and Geometric Learning Workshops 2022, PMLR, Cambridge, MA, 2022, pp. 152–160.
29.
F. L. Hitchcock, Multiple invariants and generalized rank of a \(p\)-way matrix or tensor, J. Math. Phys., 7 (1928), pp. 39–79.
30.
Y. Hu, J.-X. Liu, Y.-L. Gao, and J. Shang, DSTPCA: Double-sparse constrained tensor principal component analysis method for feature selection, IEEE/ACM Trans. Comput. Biol. Bioinform., 18 (2021), pp. 1481–1491.
31.
Y. Hu and D. B. Work, Robust tensor recovery with fiber outliers for traffic events, ACM Trans. Knowledge Discovery Data, 15 (2020), pp. 1–27.
32.
B. Huang, C. Mu, D. Goldfarb, and J. Wright, Provable low-rank tensor recovery, Optim. Online, 4252 (2014), pp. 455–500.
33.
P. Ji and J. Jin, Coauthorship and citation networks for statisticians, Ann. Appl. Stat., 10 (2016), pp. 1779–1812.
34.
E. Lopez-Rubio, Foreground detection in video sequences with probabilistic self-organizing maps, http://www.lcc.uma.es/∼ezeqlr/fsom/fsom.html.
35.
D. D. Lee and H. S. Seung, Learning the parts of objects by non-negative matrix factorization, Nature, 401 (1999), pp. 788–791.
36.
L. Li, W. Huang, I. Y.-H. Gu, and Q. Tian, Statistical modeling of complex backgrounds for foreground object detection, IEEE Trans. Image Process., 13 (2004), pp. 1459–1472.
37.
X. Li, D. Xu, H. Zhou, and L. Li, Tucker tensor regression and neuroimaging analysis, Statist. Biosci., 10 (2018), pp. 520–545.
38.
Z. Lin and H. Zhang, Low-Rank Models in Visual Analysis, Elsevier, New York, 2017, pp. 1–2.
39.
J. Liu, P. Musialski, P. Wonka, and J. Ye, Tensor completion for estimating missing values in visual data, IEEE Trans. Pattern Anal. Mach. Intell., 35 (2012), pp. 208–220.
40.
T. Liu, M. Yuan, and H. Zhao, Characterizing spatiotemporal transcriptome of the human brain via low-rank tensor decomposition, Statist. Biosci., (2022), pp. 1–29.
41.
Y. Liu, L. Chen, and C. Zhu, Improved robust tensor principal component analysis via low-rank core matrix, IEEE J. Sel. Top. Signal Process., 12 (2018), pp. 1378–1389.
42.
C. Lu, J. Feng, Y. Chen, W. Liu, Z. Lin, and S. Yan, Tensor robust principal component analysis: Exact recovery of corrupted low-rank tensors via convex optimization, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE, New York, 2016, pp. 5249–5257.
43.
C. Lu, J. Feng, Y. Chen, W. Liu, Z. Lin, and S. Yan, Tensor robust principal component analysis with a new tensor nuclear norm, IEEE Trans. Pattern Anal. Mach. Intell., 42 (2019), pp. 925–938.
44.
H. Lu, K. N. Plataniotis, and A. N. Venetsanopoulos, A survey of multilinear subspace learning for tensor data, Pattern Recognit., 44 (2011), pp. 1540–1551.
45.
Y. Luo, Y. Xin, E. Hochberg, R. Joshi, O. Uzuner, and P. Szolovits, Subgraph augmented non-negative tensor factorization (SANTF) for modeling clinical narrative text, J. Amer. Med. Inform. Assoc., 22 (2015), pp. 1009–1019.
46.
M. W. Mahoney, M. Maggioni, and P. Drineas, Tensor-CUR decompositions for tensor-based data, SIAM J. Matrix Anal. Appl., 30 (2008), pp. 957–987.
47.
M. M. Salut and D. Anderson, Randomized Tensor Robust Principal Component Analysis, TechRxiv, 2022.
48.
P. Netrapalli, N. U. N, S. Sanghavi, A. Anandkumar, and P. Jain, Non-convex robust PCA, in Advances in Neural Information Processing Systems, Vol. 27, MIT Press, Cambridge, MA, 2014.
49.
S. Oh, A. Hoogs, A. Perera, N. Cuntoor, C.-C. Chen, J. T. Lee, S. Mukherjee, J. Aggarwal, H. Lee, L. Davis, E. Swears, X. Wang, Q. Ji, K. Reddy, M. Shah, C. Vondrick, H. Pirsiavash, D. Ramanan, J. Yuen, A. Torralba, B. Song, A. Fong, A. Roy-Chowdhury, and M. Desai, A large-scale benchmark dataset for event recognition in surveillance video, in CVPR 2011, IEEE, New York, 2011, pp. 3153–3160.
50.
A. J. O’Toole, J. Harms, S. L. Snow, D. R. Hurst, M. R. Pappas, J. H. Ayyad, and H. Abdi, A video database of moving faces and people, IEEE Trans. Pattern Anal. Mach. Intell., 27 (2005), pp. 812–816.
51.
S. Rabanser, O. Shchur, and S. Günnemann, Introduction to Tensor Decompositions and Their Applications in Machine Learning, preprint, arXiv:1711.10781, 2017.
52.
A. Rajwade, A. Rangarajan, and A. Banerjee, Image denoising using the higher order singular value decomposition, IEEE Trans. Pattern Anal. Mach. Intell., 35 (2012), pp. 849–862.
53.
S. E. Sofuoglu and S. Aviyente, A two-stage approach to robust tensor decomposition, in 2018 IEEE Statistical Signal Processing Workshop (SSP), IEEE, New York, 2018, pp. 831–835.
54.
T. Song, Z. Peng, S. Wang, W. Fu, X. Hong, and P. S. Yu, Based cross-domain recommendation through joint tensor factorization, in International Conference on Database Systems for Advanced Applications, Springer-Verlag, Berlin, 2017, pp. 525–540.
55.
L. R. Tucker, Some mathematical notes on three-mode factor analysis, Psychometrika, 31 (1966), pp. 279–311.
56.
N. Vaswani and P. Narayanamurthy, Static and dynamic robust PCA and matrix completion: A review, Proc. IEEE, 106 (2018), pp. 1359–1379.
57.
J. Wright, A. Y. Yang, A. Ganesh, S. S. Sastry, and Y. Ma, Robust face recognition via sparse representation, IEEE Trans. Pattern Anal. Mach. Intell., 31 (2008), pp. 210–227.
58.
D. Xia, M. Yuan, and C.-H. Zhang, Statistically optimal and computationally efficient low rank tensor completion from noisy entries, Ann. Statist., 49 (2021), pp. 76–99.
59.
Z. Zhang, G. Ely, S. Aeron, N. Hao, and M. Kilmer, Novel methods for multilinear data completion and de-noising based on tensor-SVD, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE, New York, 2014, pp. 3842–3849.
60.
X. Zheng, W. Ding, Z. Lin, and C. Chen, Topic tensor factorization for recommender system, Inform. Sci., 372 (2016), pp. 276–293.
61.
H. Zhou, L. Li, and H. Zhu, Tensor regression with applications in neuroimaging data analysis, J. Amer. Statist. Assoc., 108 (2013), pp. 540–552.
62.
P. Zhou, C. Lu, Z. Lin, and C. Zhang, Tensor factorization for low-rank tensor completion, IEEE Trans. Image Process., 27 (2017), pp. 1152–1163.

Information & Authors

Information

Published In

cover image SIAM Journal on Imaging Sciences
SIAM Journal on Imaging Sciences
Pages: 225 - 247
ISSN (online): 1936-4954

History

Submitted: 22 May 2023
Accepted: 18 September 2023
Published online: 25 January 2024

Keywords

  1. tensor CUR decomposition
  2. robust tensor principal component analysis
  3. low-rank tensor recovery
  4. outlier detection

MSC codes

  1. 68Q25
  2. 68W25
  3. 68W20
  4. 68P20

Authors

Affiliations

Department of Statistics and Data Science and Department of Computer Science, University of Central Florida, Orlando, FL 32816 USA.
Zehan Chao
Department of Mathematics, University of California, Los Angeles, Los Angeles, CA 90095 USA.
Longxiu Huang Contact the author
Department of Computational Mathematics, Science, and Engineering and Department of Mathematics, Michigan State University, East Lansing, MI 48823 USA.
Department of Mathematics, University of California, Los Angeles, Los Angeles, CA 90095 USA.

Funding Information

National Science Foundation (NSF): DMS-2011140, DMS-2108479, DMS-2304489
Funding: The work of the authors was partially supported by National Science Foundation grants DMS-2011140, DMS-2108479, and DMS-2304489 and an AMS Simons Travel grant.

Metrics & Citations

Metrics

Citations

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

Cited By

View Options

View options

PDF

View PDF

Full Text

View Full Text

Figures

Tables

Media

Share

Share

Copy the content Link

Share with email

Email a colleague

Share on social media