Low-Rank Approximation of Generic $p \timesq \times2$ Arrays and Diverging Components in the Candecomp/Parafac Model
Abstract
We consider the low-rank approximation over the real field of generic $p \times q \times 2$ arrays. For all possible combinations of p, q, and R, we present conjectures on the existence of a best rank-R approximation. Our conjectures are motivated by a detailed analysis of the boundary of the set of arrays with at most rank R. We link these results to the Candecomp/Parafac (CP) model for three-way component analysis. Essentially, CP tries to find a best rank-R approximation to a given three-way array. In the case of $p \times q \times 2$ arrays, we show (under some regularity condition) that if a best rank-R approximation does not exist, then any sequence of CP updates will exhibit diverging CP components, which implies that several components are highly correlated in all three modes and their component weights become arbitrarily large. This extends Stegeman [Psychometrika, 71 (2006), pp. 483–501], who considers $p \times p \times 2$ arrays of rank $p+1$ or higher. We illustrate our results by means of simulations.
MSC codes
Keywords
Get full access to this article
View all available purchase options and get full access to this article.
References
1.
D. Bini, M. Capovani, F. Romani, and G. Lotti, $O(n^{2.7799})$ complexity for $n\times n$ approximate matrix multiplication, Inform. Process. Lett., 8 (1979), pp. 234–235.
2.
D. Bini, G. Lotti, and F. Romani, Approximate solutions for the bilinear form computational problem, SIAM J. Comput., 9 (1980), pp. 692–697.
3.
D. Bini, Relations between exact and approximate bilinear algorithms. Applications, Calcolo, 17 (1980), pp. 87–97.
4.
D. Bini, Border rank of a $p\times q\times 2$ tensor and the optimal approximation of a pair of bilinear forms, in Automata, Languages and Programming, J. W. de Bakker and J. van Leeuwen, eds., Lecture Notes in Comput. Sci. 85, Springer, New York, 1980, pp. 98–108.
5.
D. Bini, Border rank of $m\times n\times (mn-q)$ tensors, Linear Algebra Appl., 79 (1986), pp. 45–51.
6.
R. W. Brockett and D. Dobkin, On the optimal evaluation of a set of bilinear forms, Linear Algebra Appl., 19 (1978), pp. 207–235.
7.
P. Bürgisser, M. Clausen, and M. A. Shokrollahi, Algebraic Complexity Theory, Springer, Berlin, 1997.
8.
Y. Z. Cao, Z. P. Chen, C. Y. Mo, H. L. Wu, and R. Q. Yu, A Parafac algorithm using penalty diagonalization error (PDE) for three-way data array resolution, The Analyst, 125 (2000), pp. 2303–2310.
9.
J. D. Carroll and J. J. Chang, Analysis of individual differences in multidimensional scaling via an n-way generalization of Eckart–Young decomposition, Psychometrika, 35 (1970), pp. 283–319.
10.
V. De Silva and L.-H. Lim, Tensor Rank and the Ill-Posedness of the Best Low-Rank Approximation Problem, SCCM Technical report, 06-06, Stanford University, Palo Alto, CA, 2006; available online at http://www.sccm.stanford.edu/wrap/pubtech.html.
11.
C. Eckart and G. Young, The approximation of one matrix by another of lower rank, Psychometrika, 1 (1936), pp. 211–218.
12.
F. R. Gantmacher, The Theory of Matrices, Volumes 1 and 2, Chelsea, New York, 1959.
13.
R. A. Harshman, Foundations of the Parafac procedure: Models and conditions for an “explanatory" multimodal factor analysis, UCLA Working Papers in Phonetics, 16 (1970), pp. 1–84.
14.
R. A. Harshman and M. E. Lundy, Data preprocessing and the extended Parafac model, in Research Methods for Multimode Data Analysis, H. G. Law, C. W. Snyder Jr., J. A. Hattie, and R. P. McDonald, eds., Praeger, New York, 1984, pp. 216–284.
15.
F. L. Hitchcock, The expression of a tensor or a polyadic as a sum of products, J. Math. Phys., 6 (1927), pp. 164–189.
16.
F. L. Hitchcock, Multiple invariants and generalized rank of a p-way matrix or tensor, J. Math. Phys., 7 (1927), pp. 39–79.
17.
P. K. Hopke, P. Paatero, H. Jia, R. T. Ross, and R. A. Harshman, Three-way (Parafac) factor analysis: Examination and comparison of alternative computational methods as applied to ill-conditioned data, Chemom. Intel. Lab. Syst., 43 (1998), pp. 25–42.
18.
J. Ja'Ja', Optimal evaluation of pairs of bilinear forms, SIAM J. Comput., 8 (1979), pp. 443–462.
19.
J. Ja'Ja', An addendum to Kronecker's theory of pencils, SIAM J. Appl. Math., 37 (1979), pp. 700–712.
20.
L. Kronecker, Algebraische reduction der schaaren bilinearer formen, Sitzungber. Akad. Berlin (1890), pp. 763–776.
21.
P. M. Kroonenberg, Three-Mode Principal Component Analysis, DSWO, Leiden, The Netherlands, 1983.
22.
J. B. Kruskal, Three-way arrays: Rank and uniqueness of trilinear decompositions, with applications to arithmetic complexity and statistics, Linear Algebra Appl., 18 (1977), pp. 95–138.
23.
J. B. Kruskal, Rank, decomposition, and uniqueness for 3-way and N-way arrays, in Multiway Data Analysis, R. Coppi and S. Bolasco, eds., North-Holland, Amsterdam, 1989, pp. 7–18.
24.
J. B. Kruskal, R. A. Harshman, and M. E. Lundy, How 3-MFA data can cause degenerate Parafac solutions, among other relationships, in Multiway Data Analysis, R. Coppi and S. Bolasco, eds., North-Holland, Amsterdam, 1989, pp. 115–121 .
25.
J. M. Landsberg, The border rank of the multiplication of $2\times 2$ matrices in seven, J. Amer. Math. Soc., 19 (2006), pp. 447–459.
26.
L.-H. Lim, What's possible and what's impossible in tensor decompositions/approximation, Talk at the Tensor Decomposition Workshop, Palo Alto, CA, 2004; available online at http://csmr.ca.sandia.gov/$\!_{^{\sim}}\!$tgkolda/tdw2004/.
27.
L.-H. Lim, Optimal solutions to nonnegative Parafac/multilinear NMF always exist, Talk at the Workshop on Tensor Decompositions and Applications, CIRM, Luminy, Marseille, France, 2005; available online at http://www.etis.ensea.fr/$\!_{^{\sim}}\!$wtda/.
28.
B. C. Mitchell and D. S. Burdick, Slowly converging Parafac sequences: Swamps and two-factor degeneracies, J. Chemometrics, 8 (1994), pp. 155–168.
29.
P. Paatero, The Multilinear Engine—A table-driven least squares program for solving multilinear problems, including the n-way Parallel Factor Analysis model, J. Comput. Graph. Statist., 8 (1999), pp. 854–888.
30.
P. Paatero, Construction and analysis of degenerate Parafac models, J. Chemometrics, 14 (2000), pp. 285–299.
31.
W. S. Rayens and B. C. Mitchell, Two-factor degeneracies and a stabilization of Parafac, Chemometrics Intel. Lab. Syst., 38 (1997), pp. 173–181.
32.
N. D. Sidiropoulos, Low-rank decomposition of multi-way arrays: A signal processing perspective, Invited Plenary Lecture at the IEEE Sensor Array and Multichannel (SAM) Signal Processing Workshop, Sitges, Barcelona, Spain, 2004. Available online at http://www.telecom.tuc.gr/$\!_{^{\sim}}\!$nikos/.
33.
A. Smilde, R. Bro, and P. Geladi, Multi-way Analysis: Applications in the Chemical Sciences, John Wiley & Sons, New York, 2004.
34.
A. Stegeman, Degeneracy in Candecomp/Parafac explained for $p\times p\times 2$ arrays of rank $p+1$ or higher, Psychometrika, 71 (2006), pp. 483–501.
35.
A. Stegeman, Degeneracy in Candecomp/Parafac explained for several three-sliced arrays with a two-valued typical rank, Psychometrika, (2008), in press.
36.
V. Strassen, Vermeidung von divisionen, J. Reine Angew. Math., 246 (1973), pp. 184–202.
37.
J. M. F ten Berge, H. A. L. Kiers, and J. De Leeuw, Explicit Candecomp/Parafac solutions for a contrived $2\times 2\times 2$ array of rank three, Psychometrika, 53 (1988), pp. 579–584.
38.
J. M. F. ten Berge, Kruskal's polynomial for $2\times 2\times 2$ arrays and a generalization to $2\times n\times n$ arrays, Psychometrika, 56 (1991), pp. 631–636.
39.
J. M. F. ten Berge and H. A. L. Kiers, Simplicity of core arrays in three-way principal component analysis and the typical rank of $p\times q\times 2$ arrays, Linear Algebra Appl., 294 (1999), pp. 169–179.
40.
J. M. F. ten Berge, Partial uniqueness in Candecomp/Parafac, J. Chemometrics, 18 (2004), pp. 12–16.
41.
G. Tomasi and R. Bro, A Comparison of algorithms for fitting the Parafac model, Comput. Statist. Data Anal., 50 (2006), pp. 1700–1734.
42.
K. Weierstrass, Zur theorie der bilinearen und quadratischen formen, Monatsber. Akad. Wiss. Berlin, (1868), pp. 310–338.
43.
B. J. H. Zijlstra and H. A. L. Kiers, Degenerate solutions obtained from several variants of factor analysis, J. Chemometrics, 16 (2002), pp. 596–605.
Information & Authors
Information
Published In
SIAM Journal on Matrix Analysis and Applications
Pages: 988 - 1007
DOI: 10.1137/050644677
ISSN (online): 1095-7162
Copyright
Copyright © 2008 Society for Industrial and Applied Mathematics.
History
Submitted: 9 November 2005
Accepted: 6 March 2007
Published online: 25 September 2008
MSC codes
Keywords
Authors
Metrics & Citations
Metrics
Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited By
- Brief Review of Asymmetric MDSStructure and Dynamics of Asymmetric Interactions | 4 January 2025
- On Best Low Rank Approximation of Positive Definite TensorsSIAM Journal on Matrix Analysis and Applications, Vol. 44, No. 2 | 7 June 2023
- On the closedness and geometry of tensor network state setsLetters in Mathematical Physics, Vol. 112, No. 4 | 25 July 2022
- Guarantees for Existence of a Best Canonical Polyadic Approximation of a Noisy Low-Rank TensorSIAM Journal on Matrix Analysis and Applications, Vol. 43, No. 1 | 3 March 2022
- Unfoldings and the rank-one approximation of the tensorJournal of Computational and Applied Mathematics, Vol. 375 | 1 Sep 2020
- The Dynamics of Swamps in the Canonical Tensor Approximation ProblemSIAM Journal on Applied Dynamical Systems, Vol. 18, No. 3 | 23 July 2019
- Numerical Computation for Orthogonal Low-Rank Approximation of TensorsSIAM Journal on Matrix Analysis and Applications, Vol. 40, No. 3 | 10 September 2019
- The Optimization Landscape for Fitting a Rank-2 Tensor with a Rank-1 TensorSIAM Journal on Applied Dynamical Systems, Vol. 17, No. 2 | 17 May 2018
- On best rank-2 and rank-(2,2,2) approximations of order-3 tensorsLinear and Multilinear Algebra, Vol. 65, No. 7 | 27 September 2016
- Estimating latent trends in multivariate longitudinal data via Parafac2 with functional and structural constraintsBiometrical Journal, Vol. 59, No. 4 | 26 December 2016
- Some clarifications of remedies for Candecomp/Parafac degeneracy by means of an SVD-penalized approachChemometrics and Intelligent Laboratory Systems, Vol. 162 | 1 Mar 2017
- Remedies for Degeneracy in Candecomp/ParafacQuantitative Psychology Research | 5 August 2016
- A study on the differential strategy of some iterative trilinear decomposition algorithms: PARAFAC‐ALS, ATLD, SWATLD, and APTLDJournal of Chemometrics, Vol. 29, No. 3 | 25 November 2014
- Orthogonal Low Rank Tensor Approximation: Alternating Least Squares Method and Its Global ConvergenceSIAM Journal on Matrix Analysis and Applications, Vol. 36, No. 1 | 15 January 2015
- Successive Rank-One Approximations for Nearly Orthogonally Decomposable Symmetric TensorsSIAM Journal on Matrix Analysis and Applications, Vol. 36, No. 4 | 8 December 2015
- Three-Mode Factor Analysis by Means of Candecomp/ParafacPsychometrika, Vol. 79, No. 3 | 1 January 2025
- Finding the limit of diverging components in three-way Candecomp/Parafac—A demonstration of its practical meritsComputational Statistics & Data Analysis, Vol. 75 | 1 Jul 2014
- News Algorithms for tensor decomposition based on a reduced functionalNumerical Linear Algebra with Applications, Vol. 21, No. 3 | 14 March 2013
- On Generic Nonexistence of the Schmidt--Eckart--Young Decomposition for Complex TensorsSIAM Journal on Matrix Analysis and Applications, Vol. 35, No. 3 | 8 July 2014
- An Algebraic Solution for the Candecomp/PARAFAC Decomposition with Circulant FactorsSIAM Journal on Matrix Analysis and Applications, Vol. 35, No. 4 | 11 December 2014
- Candecomp/Parafac with ridge regularizationChemometrics and Intelligent Laboratory Systems, Vol. 129 | 1 Nov 2013
- Constrained Candecomp/Parafac via the LassoPsychometrika, Vol. 78, No. 4 | 1 January 2025
- The Special Sign Indeterminacy of the Direct-Fitting Parafac2 Model: Some Implications, Cautions, and Recommendations for Simultaneous Component AnalysisPsychometrika, Vol. 78, No. 4 | 1 January 2025
- Optimization-Based Algorithms for Tensor Decompositions: Canonical Polyadic Decomposition, Decomposition in Rank-$(L_r,L_r,1)$ Terms, and a New GeneralizationSIAM Journal on Optimization, Vol. 23, No. 2 | 18 April 2013
- A Three-Way Jordan Canonical Form as Limit of Low-Rank Tensor ApproximationsSIAM Journal on Matrix Analysis and Applications, Vol. 34, No. 2 | 30 May 2013
- Subtracting a best rank‐1 approximation from p × p × 2( p ≥2) tensorsNumerical Linear Algebra with Applications, Vol. 19, No. 3 | 7 April 2011
- Candecomp/Parafac: From Diverging Components to a Decomposition in Block TermsSIAM Journal on Matrix Analysis and Applications, Vol. 33, No. 2 | 17 April 2012
- Improved Uniqueness Conditions for Canonical Tensor Decompositions with Linearly Dependent LoadingsSIAM Journal on Matrix Analysis and Applications, Vol. 33, No. 4 | 29 November 2012
- A Brief Survey of Asymmetric MDS and Some Open ProblemsBehaviormetrika, Vol. 39, No. 1 | 1 January 2012
- On Uniqueness of the Canonical Tensor Decomposition with Some Form of SymmetrySIAM Journal on Matrix Analysis and Applications, Vol. 32, No. 2 | 29 June 2011
- Subtracting a best rank-1 approximation may increase tensor rankLinear Algebra and its Applications, Vol. 433, No. 7 | 1 Dec 2010
- A weak degeneracy revealing decomposition for the CANDECOMP/PARAFAC modelJournal of Chemometrics, Vol. 24, No. 2 | 18 January 2010
- Uniqueness Conditions for Constrained Three-Way Factor Decompositions with Linearly Dependent LoadingsSIAM Journal on Matrix Analysis and Applications, Vol. 31, No. 3 | 23 December 2009
- On Uniqueness of the nth Order Tensor Decomposition into Rank-1 Terms with Linear Independence in One ModeSIAM Journal on Matrix Analysis and Applications, Vol. 31, No. 5 | 17 August 2010
- Using the simultaneous generalized Schur decomposition as a Candecomp/Parafac algorithm for ill‐conditioned dataJournal of Chemometrics, Vol. 23, No. 7-8 | 16 March 2009
- Tensor decompositions, alternating least squares and other talesJournal of Chemometrics, Vol. 23, No. 7-8 | 20 April 2009
- Nonnegative approximations of nonnegative tensorsJournal of Chemometrics, Vol. 23, No. 7-8 | 23 June 2009
- On uniqueness conditions for Candecomp/Parafac and Indscal with full column rank in one modeLinear Algebra and its Applications, Vol. 431, No. 1-2 | 1 Jul 2009
- A survey of tensor methods2009 IEEE International Symposium on Circuits and Systems | 1 May 2009
- A Method to Avoid Diverging Components in the Candecomp/Parafac Model for Generic $I\timesJ\times2$ ArraysSIAM Journal on Matrix Analysis and Applications, Vol. 30, No. 4 | 16 January 2009
- On the Non-Existence of Optimal Solutions and the Occurrence of “Degeneracy” in the CANDECOMP/PARAFAC ModelPsychometrika, Vol. 73, No. 3 | 1 January 2025
- Tensor Rank and the Ill-Posedness of the Best Low-Rank Approximation ProblemSIAM Journal on Matrix Analysis and Applications, Vol. 30, No. 3 | 25 September 2008
- Decompositions of a Higher-Order Tensor in Block Terms—Part III: Alternating Least Squares AlgorithmsSIAM Journal on Matrix Analysis and Applications, Vol. 30, No. 3 | 25 September 2008
- Degeneracy in Candecomp/Parafac and Indscal Explained For Several Three-Sliced Arrays With A Two-Valued Typical RankPsychometrika, Vol. 72, No. 4 | 1 January 2025
View Options
- Access via your Institution
- Questions about how to access this content? Contact SIAM at [email protected].