SIAM Digital Library
 
 
 

SIAM J. on Optimization

BROWSE VOLUMES

Year Range: 

All Journal content published prior to 1997 is part of LOCUS.

Search Issue | RSS Feeds RSS
Previous Issue

2012

Volume 22, Issue 2 (partial)


Some Regularity Results for the Pseudospectral Abscissa and Pseudospectral Radius of a Matrix

Mert Gürbüzbalaban and Michael L. Overton

SIAM J. Optim. 22, pp. 281-285 (5 pages)

Online Publication Date: April 04, 2012

Full Text: | Download PDF

Show Abstract
The $\varepsilon$-pseudospectral abscissa $\alpha_\varepsilon$ and radius $\rho_\varepsilon$ of an $n\times n$ matrix are, respectively, the maximal real part and the maximal modulus of points in its $\varepsilon$-pseudospectrum, defined using the spectral norm. It was proved in [A.S. Lewis and C.H.J. Pang, SIAM J. Optim., 19 (2008), pp. 1048–1072] that for fixed $\varepsilon>0$, $\alpha_\varepsilon$ and $\rho_\varepsilon$ are Lipschitz continuous at a matrix $A$ except when $\alpha_\varepsilon$ and $\rho_\varepsilon$ are attained at a critical point of the norm of the resolvent (in the nonsmooth sense), and it was conjectured that the points where $\alpha_\varepsilon$ and $\rho_\varepsilon$ are attained are not resolvent-critical. We prove this conjecture, which leads to the new result that $\alpha_\varepsilon$ and $\rho_\varepsilon$ are Lipschitz continuous, and also establishes the Aubin property with respect to both $\varepsilon$ and $A$ of the $\varepsilon$-pseudospectrum for the points $z \in {\mathbb C}$ where $\alpha_\varepsilon$ and $\rho_\varepsilon$ are attained. Finally, we give a proof showing that the pseudospectrum can never be “pointed outwards.”

Sampling-Based Decomposition Methods for Multistage Stochastic Programs Based on Extended Polyhedral Risk Measures

Vincent Guigues and Werner Römisch

SIAM J. Optim. 22, pp. 286-312 (27 pages)

Online Publication Date: April 04, 2012

Full Text: | Download PDF

Show Abstract
We define a risk-averse nonanticipative feasible policy for multistage stochastic programs and propose a methodology to implement it. The approach is based on dynamic programming equations written for a risk-averse formulation of the problem. This formulation relies on a new class of multiperiod risk functionals called extended polyhedral risk measures. Dual representations of such risk functionals are given and used to derive conditions of coherence. In the one-period case, conditions for convexity and consistency with second order stochastic dominance are also provided. The risk-averse dynamic programming equations are specialized considering convex combinations of one-period extended polyhedral risk measures such as spectral risk measures. To implement the proposed policy, the approximation of the risk-averse recourse functions for stochastic linear programs is discussed. In this context, we detail a stochastic dual dynamic programming algorithm which converges to the optimal value of the risk-averse problem. (A corrected PDF is attached to the original pdf.)

Alternating Direction Method with Gaussian Back Substitution for Separable Convex Programming

Bingsheng He, Min Tao, and Xiaoming Yuan

SIAM J. Optim. 22, pp. 313-340 (28 pages)

Online Publication Date: April 12, 2012

Full Text: | Download PDF

Show Abstract
We consider the linearly constrained separable convex minimization problem whose objective function is separable into $m$ individual convex functions with nonoverlapping variables. A Douglas–Rachford alternating direction method of multipliers (ADM) has been well studied in the literature for the special case of $m=2$. But the convergence of extending ADM to the general case of $m\ge 3$ is still open. In this paper, we show that the straightforward extension of ADM is valid for the general case of $m\ge 3$ if it is combined with a Gaussian back substitution procedure. The resulting ADM with Gaussian back substitution is a novel approach towards the extension of ADM from $m=2$ to $m\ge 3$, and its algorithmic framework is new in the literature. For the ADM with Gaussian back substitution, we prove its convergence via the analytic framework of contractive-type methods, and we show its numerical efficiency by some application problems.

Efficiency of Coordinate Descent Methods on Huge-Scale Optimization Problems

Yu. Nesterov

SIAM J. Optim. 22, pp. 341-362 (22 pages)

Online Publication Date: April 24, 2012

Full Text: | Download PDF

Show Abstract
In this paper we propose new methods for solving huge-scale optimization problems. For problems of this size, even the simplest full-dimensional vector operations are very expensive. Hence, we propose to apply an optimization technique based on random partial update of decision variables. For these methods, we prove the global estimates for the rate of convergence. Surprisingly, for certain classes of objective functions, our results are better than the standard worst-case bounds for deterministic algorithms. We present constrained and unconstrained versions of the method and its accelerated variant. Our numerical test confirms a high efficiency of this technique on problems of very big size.

Constrained Polynomial Optimization Problems with Noncommuting Variables

Kristijan Cafuta, Igor Klep, and Janez Povh

SIAM J. Optim. 22, pp. 363-383 (21 pages)

Online Publication Date: April 24, 2012

Full Text: | Download PDF

Show Abstract
In this paper we study constrained eigenvalue optimization of noncommutative (nc) polynomials, focusing on the polydisc and the ball. Our three main results are as follows: (1) an nc polynomial is nonnegative if and only if it admits a weighted sum of hermitian squares decomposition; (2) (eigenvalue) optima for nc polynomials can be computed using a single semidefinite program (SDP)—this sharply contrasts with the commutative case where sequences of SDPs are needed; (3) the dual solution to this “single” SDP can be exploited to extract eigenvalue optimizers with an algorithm based on two ingredients: solution to a truncated nc moment problem via flat extensions, and Gelfand–Naimark–Segal construction. The implementation of these procedures in our computer algebra system NCSOStools is presented, and several examples pertaining to matrix inequalities are given to illustrate our results.

Local Convergence of Exact and Inexact Augmented Lagrangian Methods under the Second-Order Sufficient Optimality Condition

D. Fernández and M. V. Solodov

SIAM J. Optim. 22, pp. 384-407 (24 pages)

Online Publication Date: April 26, 2012

Full Text: | Download PDF

Show Abstract
We establish local convergence and rate of convergence of the classical augmented Lagrangian algorithm under the sole assumption that the dual starting point is close to a multiplier satisfying the second-order sufficient optimality condition. In particular, no constraint qualifications of any kind are needed. Previous literature on the subject required, in addition, the linear independence constraint qualification and either the strict complementarity assumption or a stronger version of the second-order sufficient condition. That said, the classical results allow the initial multiplier estimate to be far from the optimal one, at the expense of proportionally increasing the threshold value for the penalty parameters. Although our primary goal is to avoid constraint qualifications, if the stronger assumptions are introduced, then starting points far from the optimal multiplier are allowed within our analysis as well. Using only the second-order sufficient optimality condition, for penalty parameters large enough we prove primal-dual $Q$-linear convergence rate, which becomes superlinear if the parameters are allowed to go to infinity. Both exact and inexact solutions of subproblems are considered. In the exact case, we further show that the primal convergence rate is of the same $Q$-order as the primal-dual rate. Previous assertions for the primal sequence all had to do with the weaker $R$-rate of convergence and required the stronger assumptions cited above. Finally, we show that under our assumptions one of the popular rules of controlling the penalty parameters ensures their boundedness.

Regularization Methods for SDP Relaxations in Large-Scale Polynomial Optimization

Jiawang Nie and Li Wang

SIAM J. Optim. 22, pp. 408-428 (21 pages)

Online Publication Date: May 03, 2012

Full Text: | Download PDF

Show Abstract
We study how to solve semidefinite programming (SDP) relaxations for large-scale polynomial optimization. When interior-point methods are used, typically only small or moderately large problems could be solved. This paper studies regularization methods for solving polynomial optimization problems. We describe these methods for semidefinite optimization with block structures and then apply them to solve large-scale polynomial optimization problems. The performance is tested on various numerical examples. With regularization methods, significantly bigger problems could be solved on a regular computer, which is almost impossible with interior point methods.

A First-Order Augmented Lagrangian Method for Compressed Sensing

N. S. Aybat and G. Iyengar

SIAM J. Optim. 22, pp. 429-459 (31 pages)

Online Publication Date: May 03, 2012

Full Text: | Download PDF

Show Abstract
We propose a first-order augmented Lagrangian (FAL) algorithm for solving the basis pursuit problem. FAL computes a solution to this problem by inexactly solving a sequence of $\ell_1$-regularized least squares subproblems. These subproblems are solved using an infinite memory proximal gradient algorithm wherein each update reduces to “shrinkage” or constrained “shrinkage.” We show that FAL converges to an optimal solution of the basis pursuit problem whenever the solution is unique, which is the case with very high probability for compressed sensing problems. We construct a parameter sequence such that the corresponding FAL iterates are $\epsilon$-feasible and $\epsilon$-optimal for all $\epsilon>0$ within $\mathcal{O}\left(\log\left(\epsilon^{-1}\right)\right)$ FAL iterations. Moreover, FAL requires at most $\mathcal{O}(\epsilon^{-1})$ matrix-vector multiplications of the form $Ax$ or $A^Ty$ to compute an $\epsilon$-feasible, $\epsilon$-optimal solution. We show that FAL can be easily extended to solve the basis pursuit denoising problem when there is a nontrivial level of noise on the measurements. We report the results of numerical experiments comparing FAL with the state-of-the-art solvers for both noisy and noiseless compressed sensing problems. A striking property of FAL that we observed in the numerical experiments with randomly generated instances when there is no measurement noise was that FAL always correctly identifies the support of the target signal without any thresholding or postprocessing, for moderately small error tolerance values.

Lower Bounds for Polynomials Using Geometric Programming

Mehdi Ghasemi and Murray Marshall

SIAM J. Optim. 22, pp. 460-473 (14 pages)

Online Publication Date: May 15, 2012

Full Text: | Download PDF

Show Abstract
We make use of a result of Hurwitz [J. Reine Angew. Math., 108 (1891), pp. 266–268] and Reznick [Math. Ann., 283 (1989), pp. 431–464], and a consequence of this result due to Fidalgo and Kovacec [Math. Z., 269 (2011), pp. 629–645], to establish, in Theorem 2.3, a new sufficient condition for a polynomial $f\in\mathbb{R}[X_1,\dots,X_n]$ of even degree to be a sum of squares. Theorem 2.3 yields as special cases the results of Ghasemi and Marshall in [Arch. Math. (Basel), 95 (2010), pp. 343–353] and, consequently, also those of Fidalgo and Kovacec and Lasserre [Arch. Math. (Basel), 89 (2007), pp. 390–398]. We apply Theorem 2.3 to obtain a new lower bound $f_{\textrm{gp}}$ for $f$, and we explain how $f_{\textrm{gp}}$ can be computed using geometric programming. The lower bound $f_{\textrm{gp}}$ is generally not as good as the lower bound $f_{\textrm{sos}}$ introduced by Lasserre [SIAM J. Optim., 11 (2001), pp. 796–817] and Parrilo and Sturmfels [Algorithmic and Quantitative Real Algebraic Geometry, AMS, Providence, RI, 2003, pp. 88–99], which can be computed using semidefinite programming, but a run time comparison shows that, in practice, the computation of $f_{\textrm{gp}}$ is faster, and larger problems can be handled. The computation is simplest when the highest degree component of $f$ has the form $\sum_{i=1}^n a_iX_i^{2d}$, $a_i>0$, $i=1,\dots,n$. The lower bounds for $f$ established by Ghasemi and Marshall are obtained by evaluating the objective function of the geometric program at appropriate feasible points.

A Sequential Quadratic Programming Algorithm for Nonconvex, Nonsmooth Constrained Optimization

Frank E. Curtis and Michael L. Overton

SIAM J. Optim. 22, pp. 474-500 (27 pages)

Online Publication Date: May 15, 2012

Full Text: | Download PDF

Show Abstract
We consider optimization problems with objective and constraint functions that may be nonconvex and nonsmooth. Problems of this type arise in important applications, many having solutions at points of nondifferentiability of the problem functions. We present a line search algorithm for situations when the objective and constraint functions are locally Lipschitz and continuously differentiable on open dense subsets of $\mathbb{R}^{n}$. Our method is based on a sequential quadratic programming (SQP) algorithm that uses an $\ell_1$ penalty to regularize the constraints. A process of gradient sampling (GS) is employed to make the search direction computation effective in nonsmooth regions. We prove that our SQP-GS method is globally convergent to stationary points with probability one and illustrate its performance with a MATLAB implementation.
Close

Your Recent History Recent History Help

Recently Viewed

  1. Sparse Optimization with Least-Squares Constraints
  2. Positive Polynomials and Projections of Spectrahedra
  3. A Moving Balls Approximation Method for a Class of Smooth Constrained Minimization Problems
  4. Exploiting Sparsity in SDP Relaxation for Sensor Network Localization
  5. Benchmarking Derivative-Free Optimization Algorithms
© SIAM By using SIAM Publications Online you agree to abide by the Terms and Conditions of Use. Banner art adapted from a figure by Hinke M. Osinga and Bernd Krauskopf (University of Bristol, UK).
close