Abstract.

We study the problem of decomposing a polynomial \(p\) into a sum of \(r\) squares by minimizing a quadratically penalized objective \(f_p(\boldsymbol{\mathbf{u}}) = \left \lVert \sum_{i=1}^r u_i^2 - p \right \lVert^2\). This objective is nonconvex and is equivalent to the rank-\(r\) Burer–Monteiro factorization of a semidefinite program (SDP) encoding the sum of squares decomposition. We show that for all univariate polynomials \(p\), if \(r \ge 2\), then \(f_p(\boldsymbol{\mathbf{u}})\) has no spurious second-order critical points, showing that all local optima are also global optima. This is in contrast to previous work showing that for general SDPs, in addition to genericity conditions, \(r\) has to be roughly the square root of the number of constraints (the degree of \(p\)) for there to be no spurious second-order critical points. Our proof uses tools from computational algebraic geometry and can be interpreted as constructing a certificate using the first- and second-order necessary conditions. We also show that by choosing a norm based on sampling equally spaced points on the circle, the gradient \(\nabla f_p\) can be computed in nearly linear time using fast Fourier transforms. Experimentally we demonstrate that this method has very fast convergence using first-order optimization algorithms such as L-BFGS, with near-linear scaling to million-degree polynomials.

Keywords

  1. nonconvex optimization
  2. sum of squares
  3. trigonometric polynomials
  4. Burer–Monteiro method
  5. global landscape
  6. semidefinite programming

MSC codes

  1. 90C23
  2. 90C26
  3. 90C22

Get full access to this article

View all available purchase options and get full access to this article.

Acknowledgment.

We thank the two anonymous referees for their valuable feedback and comments.

References

1.
A. S. Bandeira, N. Boumal, and V. Voroninski, On the low-rank approach for semidefinite programs arising in synchronization and community detection, in Proceedings of the 29th Conference on Learning Theory, Proc. Mach. Learn. Res. 49, 2016, pp. 361–382.
2.
A. Barvinok, A remark on the rank of positive semidefinite matrices subject to affine constraints, Discrete Comput. Geom., 25 (2001), pp. 23–31, https://doi.org/10.1007/s004540010074.
3.
S. Bhojanapalli, N. Boumal, P. Jain, and P. Netrapalli, Smoothed analysis for low-rank solutions to semidefinite programs in quadratic penalty form, in Proceedings of the 31st Conference On Learning Theory, Proc. Mach. Learn. Res. 75, S. Bubeck, V. Perchet, and P. Rigollet, eds., 2018, pp. 3243–3270.
4.
S. Bhojanapalli, B. Neyshabur, and N. Srebro, Global optimality of local search for low rank matrix recovery, in Proceedings of the 30th International Conference on Neural Information Processing Systems, NIPS’16, 2016, pp. 3880–3888.
5.
G. Blekherman, P. A. Parrilo, and R. R. Thomas, eds., Semidefinite Optimization and Convex Algebraic Geometry, MOS-SIAM Ser. Optim. 13, SIAM, Philadelphia, 2013.
6.
G. Blekherman, D. Plaumann, R. Sinn, and C. Vinzant, Low-rank sum-of-squares representations on varieties of minimal degree, Int. Math. Res. Not. IMRN, 2019, pp. 33–54.
7.
N. Boumal, V. Voroninski, and A. Bandeira, The non-convex Burer-Monteiro approach works on smooth semidefinite programs, in Proceedings of Advances in Neural Information Processing Systems, D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett, eds., 2016; also available online from https://proceedings.neurips.cc/paper/2016/file/3de2334a314a7a72721f1f74a6cb4cee-Paper.pdf.
8.
N. Boumal, V. Voroninski, and A. S. Bandeira, Deterministic guarantees for Burer-Monteiro factorizations of smooth semidefinite programs, Comm. Pure Appl. Math., 73 (2020), pp. 581–608.
9.
S. Burer and R. D. Monteiro, A nonlinear programming algorithm for solving semidefinite programs via low-rank factorization, Math. Program., 95 (2003), pp. 329–357, https://doi.org/10.1007/s10107-002-0352-8.
10.
S. Burer and R. D. Monteiro, Local minima and convergence in low-rank semidefinite programming, Math. Program., 103 (2005), pp. 427–444, https://doi.org/10.1007/s10107-004-0564-1.
11.
Y. Chi, Y. M. Lu, and Y. Chen, Nonconvex optimization meets low-rank matrix factorization: An Overview, IEEE Trans. Signal Process., 67 (2019), pp. 5239–5269, https://doi.org/10.1109/TSP.2019.2937282.
12.
M.-D. Choi, T. Y. Lam, and B. Reznick, Sums of squares of real polynomials, in K-Theory and Algebraic Geometry: Connections with Quadratic Forms and Division Algebras, Part 2, Proc. Sympos. Pure Math. 58, American Mathematical Society, Providence, RI, 1995, pp. 103–126, https://doi.org/10.1090/pspum/058.2/1327293.
13.
D. Cifuentes, On the Burer-Monteiro method for general semidefinite programs, Optim. Lett., 15 (2021), pp. 2299–2309.
14.
D. Cifuentes and A. Moitra, Polynomial Time Guarantees for the Burer-Monteiro Method, https://arxiv.org/abs/1912.01745, 2019.
15.
D. Cifuentes and P. A. Parrilo, Sampling algebraic varieties for sum of squares programs, SIAM J. Optim., 27 (2017), pp. 2381–2404, https://doi.org/10.1137/15M1052548.
16.
D. A. Cox, J. Little, and D. O’Shea, Ideals, Varieties, and Algorithms, Springer, New York, 2015.
17.
B. Dumitrescu, Positive Trigonometric Polynomials and Signal Processing Applications, Signals Commun. Technol. 103, Springer, New York, 2007.
18.
D. Dummit and R. Foote, Abstract Algebra, Wiley, New York, 2003.
19.
M. Frigo and S. G. Johnson, The design and implementation of FFTW3, Proc. IEEE, 93 (2005), pp. 216–231.
20.
R. Ge, F. Huang, C. Jin, and Y. Yuan, Escaping from saddle points—Online stochastic gradient for tensor decomposition, in Proceedings of the Conference on Learning Theory, 2015, pp. 797–842.
21.
R. Ge, C. Jin, and Y. Zheng, No spurious local minima in nonconvex low rank problems: A unified geometric analysis, in Proceedings of the 34th International Conference on Machine Learning, J. Mach. Learn. Res. 70, 2017, pp. 1233–1242.
22.
R. Ge, J. D. Lee, and T. Ma, Matrix completion has no spurious local minimum, in Proceedings of the 30th International Conference on Neural Information Processing Systems, NIPS’16, 2016, pp. 2981–2989.
23.
R. W. Hamming, Numerical Methods for Scientists and Engineers, McGraw-Hill, New York, 1973.
24.
C. Jin, R. Ge, P. Netrapalli, S. M. Kakade, and M. I. Jordan, How to escape saddle points efficiently, in Proceedings of the International Conference on Machine Learning, Proc. Mach. Learn. Res. 70, 2017, pp. 1724–1732.
25.
S. G. Johnson, The NLopt Nonlinear-Optimization Package, 2014.
26.
L. Kapelevich, C. Coey, and J. P. Vielma, Sum of squares generalizations for conic sets, Math. Program., 199 (2023), pp. 1417–1429, https://doi.org/10.1007/s10107-022-01831-6.
27.
M. Laurent, Sums of squares, moment matrices and optimization over polynomials, in Emerging Applications of Algebraic Geometry, Springer, New York, 2009, pp. 157–270.
28.
B. Legat, C. Yuan, and P. A. Parrilo, Low-rank univariate sum of squares has no spurious local minima, 2023, https://doi.org/10.24433/CO.7364738.v1.
29.
Q. Li, Z. Zhu, and G. Tang, The non-convex geometry of low-rank matrix optimization, Inf. Inference, 8 (2019), pp. 51–96.
30.
J. Lofberg and P. Parrilo, From coefficients to samples: A new approach to SOS optimization, in Proceedings of the 43rd IEEE Conference on Decision and Control (CDC), IEEE, 2004, pp. 3154–3159, https://doi.org/10.1109/CDC.2004.1428957.
31.
D. Papp, Univariate polynomial optimization with sum-of-squares interpolants, in Modeling and Optimization: Theory and Applications, Springer, New York, 2017, pp. 143–162.
32.
P. A. Parrilo, Structured Semidefinite Programs and Semialgebraic Geometry Methods in Robustness and Optimization, Ph.D. thesis, California Institute of Technology, 2000.
33.
P. A. Parrilo, Polynomial games and sum of squares optimization, in Proceedings of the 45th IEEE Conference on Decision and Control, IEEE, 2006, pp. 2855–2860.
34.
G. Pataki, On the rank of extreme matrices in semidefinite programs and the multiplicity of optimal eigenvalues, Math. Oper. Res., 23 (1998), pp. 257–511 https://doi.org/10.1287/moor.23.2.339.
35.
T. Roh and L. Vandenberghe, Discrete transforms, semidefinite programming, and sum-of-squares representations of nonnegative polynomials, SIAM J. Optim., 16 (2006), pp. 939–964.
36.
C. Scheiderer, Extreme points of Gram spectrahedra of binary forms, Discrete Comput. Geom., 67 (2022), pp. 1174–1190.
37.
K. Schmüdgen, The K-moment problem for compact semi-algebraic sets, Math. Ann., 289 (1991), pp. 203–206.
38.
B. Sturmfels, Solving Systems of Polynomial Equations, CBMS Reg. Conf. Ser. Math. 97, American Mathematical Society, Providence, RI, 2002.
39.
J. Sun, Q. Qu, and J. Wright, A geometric analysis of phase retrieval, Found. Comput. Math., 18 (2018), pp. 1131–1198.
40.
I. Waldspurger and A. Waters, Rank optimality for the Burer-Monteiro factorization, SIAM J. Optim., 30 (2020), pp. 2577–2602.
41.
S.-P. Wu, S. Boyd, and L. Vandenberghe, FIR filter design via spectral factorization and convex optimization, in Applied and Computational Control, Signals, and Circuits, Springer, New York, 1999, pp. 215–245.

Information & Authors

Information

Published In

cover image SIAM Journal on Optimization
SIAM Journal on Optimization
Pages: 2041 - 2061
ISSN (online): 1095-7189

History

Submitted: 16 August 2022
Accepted: 18 April 2023
Published online: 8 August 2023

Keywords

  1. nonconvex optimization
  2. sum of squares
  3. trigonometric polynomials
  4. Burer–Monteiro method
  5. global landscape
  6. semidefinite programming

MSC codes

  1. 90C23
  2. 90C26
  3. 90C22

Authors

Affiliations

Benoît Legat
Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, MA 02139 USA.
Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, MA 02139 USA.
Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, MA 02139 USA.

Funding Information

Funding: The first author was supported by a BAEF Postdoctoral Fellowship and the NSF grant OAC-1835443. The second author was supported by the NSF grant CCF-1565235 and AFOSR grant FA8750-19-2-1000.

Metrics & Citations

Metrics

Citations

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

View Options

View options

PDF

View PDF

Full Text

View Full Text

Figures

Tables

Media

Share

Share

Copy the content Link

Share with email

Email a colleague

Share on social media