On the Estimation Performance and Convergence Rate of the Generalized Power Method for Phase Synchronization

Abstract

An estimation problem of fundamental interest is that of phase (or angular) synchronization, in which the goal is to recover a collection of phases (or angles) using noisy measurements of relative phases (or angle offsets). It is known that in the Gaussian noise setting, the maximum likelihood estimator (MLE) is an optimal solution to a nonconvex quadratic optimization problem and can be found with high probability using semidefinite programming (SDP), provided that the noise power is not too large. In this paper, we study the estimation and convergence performance of a recently proposed low-complexity alternative to the SDP-based approach, namely, the generalized power method (GPM). Our contribution is twofold. First, we show that the sequence of estimation errors associated with the GPM iterates is bounded above by a decreasing sequence. As a corollary, we show that all iterates achieve an estimation error that is on the same order as that of an MLE. Our result holds under the least restrictive assumption on the noise power and gives the best provable bound on the estimation error known to date. It also implies that one can terminate the GPM at any iteration and still obtain an estimator that has a theoretical guarantee on its estimation error. Second, we show that under the same assumption on the noise power as that in [A. S. Bandeira, N. Boumal, and A. Singer, Math. Program. Ser. A, 163 (2017), pp. 145--167] for the SDP-based method, the GPM will converge to an MLE at a linear rate with high probability. This answers a question raised in [N. Boumal, SIAM J. Optim., 26 (2016), pp. 2355--2377] and shows that the GPM is competitive in terms of both theoretical guarantees and numerical efficiency with the SDP-based method. At the heart of our convergence rate analysis is a new error bound for the nonconvex quadratic optimization formulation of the phase synchronization problem, which could be of independent interest. As a by-product, we give an alternative proof of a result in [N. Boumal, SIAM J. Optim., 26 (2016), pp. 2355--2377], which asserts that every second-order critical point of the aforementioned nonconvex quadratic optimization formulation is globally optimal in a certain noise regime.

Keywords

  1. phase synchronization
  2. generalized power method
  3. convergence rate analysis
  4. maximum likelihood estimation
  5. error bounds

MSC codes

  1. 90C26
  2. 90C30
  3. 90C46

Get full access to this article

View all available purchase options and get full access to this article.

References

1.
B. Alexeev, A. S. Bandeira, M. Fickus, and D. G. Mixon, Phase retrieval with polarization, SIAM J. Imaging Sci., 7 (2014), pp. 35--66, https://doi.org/10.1137/12089939X.
2.
A. S. Bandeira, N. Boumal, and A. Singer, Tightness of the maximum likelihood semidefinite relaxation for angular synchronization, Math. Program. Ser. A, 163 (2017), pp. 145--167, https://doi.org/10.1007/s10107-016-1059-6.
3.
N. Boumal, On intrinsic Cramér--Rao bounds for Riemannian submanifolds and quotient manifolds, IEEE Trans. Signal Process., 61 (2013), pp. 1809--1821, https://doi.org/10.1109/TSP.2013.2242068.
4.
N. Boumal, Nonconvex phase synchronization, SIAM J. Optim., 26 (2016), pp. 2355--2377, https://doi.org/10.1137/16M105808X.
5.
N. Boumal, V. Voroninski, and A. S. Bandeira, The non--convex Burer--Monteiro approach works on smooth semidefinite programs, in Proceedings of the 2016 Conference on Advances in Neural Information Processing Systems 29, D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett, eds., Curran, Red Hook, NY, 2017, pp. 2757--2765.
6.
M. Cucuringu, Sync--rank: Robust ranking, constrained ranking and rank aggregation via eigenvector and SDP synchronization, IEEE Trans. Network Sci. Eng., 3 (2016), pp. 58--79, https://doi.org/10.1109/TNSE.2016.2523761.
7.
A. Giridhar and P. R. Kumar, Distributed clock synchronization over wireless networks: Algorithms and analysis, in Proceedings of the 45th IEEE Conference on Decision and Control, IEEE, Piscataway, NJ, 2006, pp. 4915--4920, https://doi.org/10.1109/CDC.2006.377325.
8.
G. H. Golub and C. F. Van Loan, Matrix Computations, 3rd ed., The Johns Hopkins University Press, Baltimore, MD, 1996.
9.
K. Hou, Z. Zhou, A. M.-C. So, and Z.-Q. Luo, On the linear convergence of the proximal gradient method for trace norm regularization, in Advances in Proceedings of the 2013 Conference on Neural Information Processing Systems 26, C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Weinberger, eds., Curran, Red Hook, NY, 2014, pp. 710--718.
10.
M. Journée, Yu. Nesterov, P. Richtárik, and R. Sepulchre, Generalized power method for sparse principal component analysis, J. Mach. Learn. Res., 11 (2010), pp. 517--553.
11.
G. Li and T. K. Pong, Calculus of the exponent of Kurdyka--Łojasiewicz inequality and its applications to linear convergence of first--order methods, Found. Comput. Math., to appear, https://doi.org/10.1007/s10208-017-9366-8.
12.
H. Liu, W. Wu, and A. M.-C. So, Quadratic optimization with orthogonality constraints: Explicit Łojasiewicz exponent and linear convergence of line--search methods, in Proceedings of the 33rd International Conference on Machine Learning (ICML 2016), Curran, Red Hook, NY, 2016, pp. 1158--1167.
13.
H. Liu, M.-C. Yue, A. M.-C. So, and W.-K. Ma, A discrete first-order method for large-scale MIMO detection with provable guarantees, in Proceedings of the 18th IEEE International Workshop on Signal Processing Advances in Wireless Communications (SPAWC 2017), IEEE, Piscataway, NJ, 2017, pp. 669--673.
14.
Z.-Q. Luo, W.-K. Ma, A. M.-C. So, Y. Ye, and S. Zhang, Semidefinite relaxation of quadratic optimization problems, IEEE Signal Processing Mag., 27 (2010), pp. 20--34, https://doi.org/10.1109/MSP.2010.936019.
15.
R. Luss and M. Teboulle, Conditional gradient algorithms for rank--one matrix approximations with a sparsity constraint, SIAM Rev., 55 (2013), pp. 65--98, https://doi.org/10.1137/110839072.
16.
A. Singer, Angular synchronization by eigenvectors and semidefinite programming, Appl. Comput. Harmon. Anal., 30 (2011), pp. 20--36, https://doi.org/10.1016/j.acha.2010.02.001.
17.
A. M.-C. So, Probabilistic analysis of the semidefinite relaxation detector in digital communications, in Proceedings of the 21st Annual ACM--SIAM Symposium on Discrete Algorithms (SODA 2010), SIAM, Philadelphia, 2010, pp. 698--711, https://doi.org/10.1137/1.9781611973075.57.
18.
A. M.-C. So, J. Zhang, and Y. Ye, On approximating complex quadratic optimization problems via semidefinite programming relaxations, Math. Program. Ser. B, 110 (2007), pp. 93--110, https://doi.org/10.1007/s10107-006-0064-6.
19.
A. M.-C. So and Z. Zhou, Non--asymptotic convergence analysis of inexact gradient methods for machine learning without strong convexity, Optim. Methods Softw., 32 (2017), pp. 963--992, https://doi.org/10.1080/10556788.2017.1296439.
20.
J. Sun, Q. Qu, and J. Wright, When Are Nonconvex Problems Not Scary?, 2015, https://arxiv.org/abs/1510.06096.
21.
O. Toker and H. Özbay, On the complexity of purely complex $\mu$ computation and related problems in multidimensional systems, IEEE Trans. Automat. Control, 43 (1998), pp. 409--414, https://doi.org/10.1109/9.661609.
22.
A. Viswanathan and M. Iwen, Fast angular synchronization for phase retrieval via incomplete information, in Wavelets and Sparsity XVI, Proc. SPIE 9597, Bellingham, WA, 2015, 959718, https://doi.org/10.1117/12.2186336.
23.
H.-T. Wai, W.-K. Ma, and A. M.-C. So, Cheap semidefinite relaxation MIMO detection using row--by--row block coordinate descent, in Proceedings of the 2011 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2011), IEEE, Piscataway, NJ, 2011, pp. 3256--3259, https://doi.org/10.1109/ICASSP.2011.5946716.
24.
Z. Wen, D. Goldfarb, and K. Scheinberg, Block coordinate descent methods for semidefinite programming, in Handbook on Semidefinite, Conic and Polynomial Optimization, M. F. Anjos and J. B. Lasserre, eds., Internat. Ser. Oper. Res. Manag. Sci. 166, Springer, New York, 2012, pp. 533--564.
25.
Z. Wen, D. Goldfarb, and W. Yin, Alternating direction augmented Lagrangian methods for semidefinite programming, Math. Program. Comput., 2 (2010), pp. 203--230, https://doi.org/10.1007/s12532-010-0017-1.
26.
H. Zhang, New Analysis of Linear Convergence of Gradient--Type Methods via Unifying Error Bound Conditions, 2016, https://arxiv.org/abs/1606.00269.
27.
Y. Zhong and N. Boumal, Near--Optimal Bounds for Phase Synchronization, 2017, https://arxiv.org/abs/1703.06605.
28.
Z. Zhou and A. M.-C. So, A unified approach to error bounds for structured convex optimization problems, Math. Program. Ser. A, 165 (2017), pp. 689--728, https://doi.org/10.1007/s10107-016-1100-9.
29.
Z. Zhou, Q. Zhang, and A. M.-C. So, $\ell_{1,p}$--norm regularization: Error bounds and convergence rate analysis of first--order methods, in Proceedings of the 32nd International Conference on Machine Learning (ICML 2015), Curran, Red Hook, NY, 2015, pp. 1501--1510.

Information & Authors

Information

Published In

cover image SIAM Journal on Optimization
SIAM Journal on Optimization
Pages: 2426 - 2446
ISSN (online): 1095-7189

History

Submitted: 28 October 2016
Accepted: 31 August 2017
Published online: 12 December 2017

Keywords

  1. phase synchronization
  2. generalized power method
  3. convergence rate analysis
  4. maximum likelihood estimation
  5. error bounds

MSC codes

  1. 90C26
  2. 90C30
  3. 90C46

Authors

Affiliations

Metrics & Citations

Metrics

Citations

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

Cited By

Media

Figures

Other

Tables

Share

Share

Copy the content Link

Share with email

Email a colleague

Share on social media