Best Nonnegative Rank-One Approximations of Tensors
Abstract
In this paper, we study the polynomial optimization problem of a multiform over the intersection of the multisphere and the nonnegative orthants. This class of problems is NP-hard in general and includes the problem of finding the best nonnegative rank-one approximation of a given tensor. A Positivstellensatz is given for this class of polynomial optimization problems, based on which a globally convergent hierarchy of doubly nonnegative (DNN) relaxations is proposed. A (zeroth order) DNN relaxation method is applied to solve these problems, resulting in linear matrix optimization problems under both the positive semidefinite and nonnegative conic constraints. A worst case approximation bound is given for this relaxation method. The recent solver SDPNAL+ is adopted to solve this class of matrix optimization problems. Typically the DNN relaxations are tight, and hence the best nonnegative rank-one approximation of a tensor can be obtained frequently. Extensive numerical experiments show that this approach is quite promising.
Keywords
MSC codes
Get full access to this article
View all available purchase options and get full access to this article.
References
1.
E. Allman, J. Rhodes, B. Sturmfels, and P. Zwiernik, Tensors of nonnegative rank two, Linear Algebra Appl., 473 (2015), pp. 37--53.
2.
N. Asgarian and R. Greiner, Using Rank-1 Biclusters to Classify Microarray Data, Department of Computing Science and the Alberta Ingenuity Center for Machine Learning, University of Alberta, Edmonton, Canada, 2006.
3.
A. Ben-Tal and A. Nemirovskii, Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications, SIAM, Philadelphia, PA, 2001.
4.
S. Bergmann, J. Ihmels, and N. Barkai, Iterative signature algorithm for the analysis of large-scale gene expression data, Phys. Rev. E, 67 (2003), 031902.
5.
M. Biggs, A. Ghodsi, and S. Vavasis, Nonnegative matrix factorization via rank-one downdating, in Proceedings of the International Conference on Machine Learning, 2008.
6.
J. Bochnak, M. Coste, and M.-F. Roy, Real Algebraic Geometry, Springer, Berlin, 1998.
7.
R. Bro and S. De Jong, A fast non-negativity-constrained least squares algorithm, J. Chemometrics, 11 (1997), pp. 393--401.
8.
A. Cichocki, R. Zdunek, A. Phan, and S. Amari, Nonnegative Matrix and Tensor Factorizations: Application to Exploratory Multi-Way Data Analysis and Blind Separation, Wiley, Hoboken, NJ, 2009.
9.
J. Cohen and U. Rothblum, Nonnegative ranks, decompositions and factorizations of nonnegative matrices, Linear Algebra Appl., 190 (1993), pp. 149--168.
10.
R. Curto and L. Fialkow, Solution of the Truncated Complex Moment Problem for Flat Data, Mem. Amer. Math. Soc. 119, American Mathematical Society, Providence, RI, 1996.
11.
L. De Lathauwer, B. De Moor, and J. Vandewalle, A multilinear singular value decomposition, SIAM J. Matrix Anal. Appl., 21 (2000), pp. 1253--1278.
12.
L. De Lathauwer, B. De Moor, and J. Vandewalle, On the best rank-$1$ and rank-$(r_1, r_2,\dots,r_n)$ approximation of higher-order tensors, SIAM J. Matrix Anal. Appl., 21 (2000), pp. 1324--1342.
13.
O. Debals, M. Van Barel, and L. De Lathauwer, Nonnegative matrix factorization using nonnegative polynomial approximations, IEEE Signal Process. Lett., 24 (2017), pp. 948--952.
14.
P. Dickinson and L. Gijben, On the computational complexity of membership problems for the completely positive cone and its dual, Comput. Optim. Appl., 57 (2014), pp. 403--415.
15.
D. FitzGerald, M. Cranitch, and E. Coyle, Non-negative tensor factorisation for sound source separation, in Proceedings of the Irish Signals and Systems Conference, 2005.
16.
M. Friedlander and K. Hatz, Computing nonnegative tensor factorizations, Comput. Optim. Appl., 23 (2008), pp. 631--647.
17.
K. Fujisawa, Y. Futakata, M. Kojima, S. Matsuyama, S. Nakamura, K. Nakata, and M. Yamashita, SDPA-M (Semidefinite Programming Algorithm in MATLAB), http://homepage.mac.com/klabtitech/sdpa-homepage/download.html.
18.
M. Garey and D. Johnson, Computers and Intractability, A Guide to the Theory of NP-Completeness, Freeman, New York, NY, 1980.
19.
N. Gillis, Approximation et sous-approximation de matrices par factorisation positive: algorithmes, complexité et applications, Master's Thesis, Université Catholique de Louvain, Louvain-la-Neuve, Belgium, 2006.
20.
G. Hardy, J. Littlewood, and G. Pólya, Inequalities, 2nd ed., Cambridge University Press, Cambridge, UK, 1952.
21.
J. H\aastad, Tensor rank is NP-complete, J. Algorithms, 11 (1990), pp. 644--654.
22.
T. Hazan, S. Polak, and A. Shashua, Sparse image coding using a $3$D non-negative tensor factorization, in Proceedings of the 10th IEEE International Conference on Computer Vision, Vol. 1, IEEE Computer Society Press, 2005, pp. 50--57.
23.
S. Hu and G. Li, Convergence rate analysis for the higher order power method in best rank one approximations of tensors, Numer. Math., 140 (2018), pp. 993--1031.
24.
K. Huang, N. Sidiropoulos, and A. Liavas, A flexible and efficient algorithmic framework for constrained matrix and tensor factorization, IEEE Trans. Signal Process., 64 (2016), pp. 5052--5065.
25.
E. Kofidis and P. Regalia, On the best rank-1 approximation of higher-order supersymmetric tensors, SIAM J. Matrix Anal. Appl., 23 (2002), pp. 863--884.
26.
T. Kolda and B. Bader, Tensor decompositions and applications, SIAM Rev., 51 (2009), pp. 455--500.
27.
T. Kolda and J. Mayo, Shifted power method for computing tensor eigenpairs, SIAM J. Matrix Anal. Appl., 32 (2011), pp. 1095--1124.
28.
J. Lasserre, Global optimization with polynomials and the problem of moments, SIAM J. Optim., 11 (2001), pp. 796--817.
29.
C. Lawson and R. Hanson, Solving Least Squares Problems, Prentice-Hall, Englewood Cliffs, NJ, 1974.
30.
D. Lee and H. Seung, Learning the parts of objects by non-negative matrix factorization, Nature, 401 (1999), pp. 788--791.
31.
B. Levin, On Calculating Maximum Rank-One under Approximations for Positive Arrays, Technical report B-48, Division of Biostatistics, Columbia University, New York.
32.
L.-H. Lim, Tensors and hypermatrices, in Handbook of Linear Algebra, L. Hogben, ed., CRC Press, Boca Raton, FL 2013, Chapter 15.
33.
L.-H. Lim and P. Comon, Nonnegative approximations of nonnegative tensors, J. Chemometrics, 23 (2009), pp. 432--441.
34.
C. Ling, J. Nie, L. Qi, and Y. Ye, Biquadratic optimization over unit spheres and semidefinite programming relaxations, SIAM J. Optim., 20 (2009), pp. 1286--1310.
35.
T. Motzkin and E. Straus, Maxima for graphs and a new proof of a theorem of Túran, Canad. J. Math., 17 (1965), pp. 533--540.
36.
K. Murty and S. Kabadi, Some NP-complete problems in quadratic and nonlinear programming, Math. Program., 39 (1987), pp. 117--129.
37.
J. Nie, Sums of squares methods for minimizing polynomial forms over spheres and hypersurfaces, Front. Math. China, 7 (2012), pp. 321--346.
38.
J. Nie, Optimality conditions and finite convergence of Lasserre's hierarchy, Math. Program., 146 (2014), pp. 97--121.
39.
J. Nie, J. Demmel, and B. Sturmfels, Minimizing polynomials via sums of squares over the gradient ideal, Math. Program., 106 (2006), pp. 587--606.
40.
J. Nie and M. Schweighofer, On the complexity of Putinar's Positivstellensatz, J. Complexity, 23 (2007), pp. 135--150.
41.
J. Nie and L. Wang, Semidefinite relaxations for best rank-$1$ tensor approximations, SIAM J. Matrix Anal. Appl., 35 (2014), pp. 1155--1179.
42.
J. Nie, Z. Yang, and X. Zhang, A complete semidefinite algorithm for detecting copositive matrices and tensors, SIAM J. Optim., 28 (2018), pp. 2902--2921.
43.
J. Nie and X. Zhang, Positive maps and separable matrices, SIAM J. Optim., 26 (2016), pp. 1236--1256.
44.
P. Paatero, A weighted non-negative least squares algorithm for three-way “PARAFAC” factor analysis, Chemometrics. Intell. Lab. Syst., 38 (1997), pp. 223--242.
45.
P. Paatero and U. Tapper, Positive matrix factorization: a non-negative factor model with optimal utilization of error estimates of data values, Environmetrics, 5 (1994), pp. 111--126.
46.
P. Parrilo and B. Sturmfels, Minimizing polynomial functions, in Algorithmic and Quantitative Real Algebraic Geometry, DIMACS Series in Discrete Mathematics and Theoretical Computer Science 60 American Mathematical Society, Providence, RI 2003, pp. 83--99.
47.
J. Pena, J. Vera, and L. Zuluaga, Completely positive reformulations for polynomial optimization, Math. Program., 151 (2014), pp. 405--431.
48.
G. Pólya, Über positive Darstellung von Polynomen Vierteljschr, Naturforsch. Ges. Zürich., 73 (1928), pp. 141--145.
49.
V. Powers and B. Reznick, A new bound for Pólya's theorem with applications to polynomials positive on polyhedra, J. Pure Appl. Algebra, 164 (2001), pp. 221--229.
50.
M. Putinar, Positive polynomials on compact semi-algebraic sets, Indiana Univ. Math. J., 42 (1993), pp. 969--984.
51.
L. Qi, Symmetric nonnegative tensors and copositive tensors, Linear Algebra Appl., 439 (2013), pp. 228--238.
52.
Y. Qi, P. Comon, and L.-H. Lim, Uniqueness of nonnegative tensor approximations, IEEE Trans. Inform. Theory, 62 (2016), pp. 2170--2183.
53.
B. Reznick, Some concrete aspects of Hilbert's 17th problem, Contemp. Math., 253 (2000), pp. 251--272.
54.
J.-P. Royer, N. Thirion-Moreau, and P. Comon, Computing the polyadic decomposition of nonnegative third order tensors, Signal Process., 91 (2011), pp. 2159--2171.
55.
A. Shashua and T. Hazan, Non-negative tensor factorization with applications to statistics and computer vision, in Proceedings of the 22nd International Conference on Machine Learning, 2005, pp. 792--799.
56.
A. Shashua and A. Levin, Linear image coding for regression and classification using the tensor-rank principle, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2001.
57.
A. Shashua, R. Zass, and T. Hazan, Multi-way clustering using super-symmetric non-negative tensor factorization, in Proceedings of the 9th European Conference on Computer Vision, Part IV, 2006, pp. 595--608.
58.
N. Sidiropoulos, L. De Lathauwer, X. Fu, K. Huang, E. Papalexakis, and C. Faloutsos, Tensor decomposition for signal processing and machine learning, IEEE Trans. Signal Process., 65 (2017), pp. 3551--3582.
59.
J. Sturm, SeDuMi 1.02: A Matlab toolbox for optimization over symmetric cones, Optim. Methods Softw., 11 & 12 (1999), pp. 625--653.
60.
D. Sun, K.-C. Toh, and L. Yang, A convergent 3-block semi-proximal alternating direction method of multipliers for conic programming with 4-type constraints, SIAM J. Optim., 25 (2015), pp. 882--915.
61.
D. Sun, K.-C. Toh, Y. Yuan, and X.-Y. Zhao, SDPNAL+: A MATLAB software for semidefinite programming with bound constraints (version 1.0), Optim. Methods Softw., in press.
62.
K.-C. Toh, M. Todd, and R. Tutuncu, SDPT3: A Matlab software package for semidefinite programming, Optim. Methods Softw., 11 (1999), pp. 545--581.
63.
S. Vavasis, On the complexity of nonnegative matrix factorization, SIAM J. Optim., 20 (2009), pp. 1364--1377.
64.
M. Welling and M. Weber, Positive tensor factorization, Pattern Recogn. Lett., 22 (2001), pp. 1255--1261.
65.
L. Yang, D. Sun, and K.-C. Toh, SDPNAL+: A majorized semismooth Newton-CG augmented Lagrangian method for semidefinite programming with nonnegative constraints, Math. Program. Comput., 7 (2015), pp. 331--366.
66.
X.-Y. Zhao, D. Sun, and K.-C. Toh, A Newton-CG augmented Lagrangian method for semidefinite programming, SIAM J. Optim., 20 (2010), pp. 1737--1765.
67.
G. Zhou, A. Cichocki, Q. Zhao, and S. Xie, Efficient nonnegative Tucker decompositions: Algorithms and uniqueness, IEEE Trans. Image Process., 24 (2015), pp. 4990--5003.
Information & Authors
Information
Published In
SIAM Journal on Matrix Analysis and Applications
Pages: 1527 - 1554
DOI: 10.1137/18M1224064
ISSN (online): 1095-7162
Copyright
© 2019, Society for Industrial and Applied Mathematics.
History
Submitted: 1 November 2018
Accepted: 7 October 2019
Published online: 3 December 2019
Keywords
MSC codes
Authors
Funding Information
Hong Kong Research Grant Council : PolyU153014/18p
Ministry of Education, Singapore : R-146-000-257-112
National Natural Science Foundation of China https://doi.org/10.13039/501100001809 : 11771328
Tianjin University https://doi.org/10.13039/501100004517 : 2017XZC-0084, 2017XRG-0015
Metrics & Citations
Metrics
Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited By
- Quantifying low rank approximations of third order symmetric tensorsMathematical Programming, Vol. 4 | 28 November 2024
- A DCA-Newton method for quartic minimization over the sphereAdvances in Computational Mathematics, Vol. 49, No. 4 | 10 July 2023
- Certifying the Global Optimality of Quartic Minimization over the SphereJournal of the Operations Research Society of China, Vol. 10, No. 2 | 3 May 2021
View Options
- Access via your Institution
- Questions about how to access this content? Contact SIAM at [email protected].