SIAM Digital Library
 
 
 

SIAM J. on Optimization

All Journal content published prior to 1997 is part of LOCUS.

Search Issue | RSS Feeds RSS
Previous Issue

2011

Volume 21, Issue 4, pp. 1201-1739


Sparse Optimization with Least-Squares Constraints

Ewout van den Berg and Michael P. Friedlander

SIAM J. Optim. 21, pp. 1201-1229 (29 pages)

Online Publication Date: October 04, 2011

Full Text: | Download PDF

Show Abstract
The use of convex optimization for the recovery of sparse signals from incomplete or compressed data is now common practice. Motivated by the success of basis pursuit in recovering sparse vectors, new formulations have been proposed that take advantage of different types of sparsity. In this paper we propose an efficient algorithm for solving a general class of sparsifying formulations. For several common types of sparsity we provide applications, along with details on how to apply the algorithm, and experimental results.

A Monotone+Skew Splitting Model for Composite Monotone Inclusions in Duality

Luis M. Briceño-Arias and Patrick L. Combettes

SIAM J. Optim. 21, pp. 1230-1250 (21 pages)

Online Publication Date: October 11, 2011

Full Text: | Download PDF

Show Abstract
The principle underlying this paper is the basic observation that the problem of simultaneously solving a large class of composite monotone inclusions and their duals can be reduced to that of finding a zero of the sum of a maximally monotone operator and a linear skew-adjoint operator. An algorithmic framework is developed for solving this generic problem in a Hilbert space setting. New primal-dual splitting algorithms are derived from this framework for inclusions involving composite monotone operators, and convergence results are established. These algorithms draw their simplicity and efficacy from the fact that they operate in a fully decomposed fashion in the sense that the monotone operators and the linear transformations involved are activated separately at each iteration. Comparisons with existing methods are made and applications to composite variational problems are demonstrated.

Coupling Forward-Backward with Penalty Schemes and Parallel Splitting for Constrained Variational Inequalities

Hédy Attouch, Marc-Olivier Czarnecki, and Juan Peypouquet

SIAM J. Optim. 21, pp. 1251-1274 (24 pages)

Online Publication Date: November 01, 2011

Full Text: | Download PDF

Show Abstract
We are concerned with the study of a class of forward-backward penalty schemes for solving variational inequalities $0\in Ax + N_C (x)$ where $\mathcal{H}$ is a real Hilbert space, $A: \mathcal{H}\rightrightarrows \mathcal{H}$ is a maximal monotone operator, and $N_C$ is the outward normal cone to a closed convex set $C\subset\mathcal{H}$. Let $\Psi: \mathcal{H} \to \mathbb R$ be a convex differentiable function whose gradient is Lipschitz continuous and which acts as a penalization function with respect to the constraint $x\in C.$ Given a sequence $(\beta_n)$ of penalization parameters which tends to infinity, and a sequence of positive time steps $(\lambda_n) \in\ell^2\setminus\ell^1$, we consider the diagonal forward-backward algorithm $x_{n+1}=(I+\lambda_nA)^{-1}(x_n-\lambda_n\beta_n \nabla \Psi (x_n)).$ Assuming that $(\beta_n)$ satisfies the growth condition $\limsup_{n\to\infty}\lambda_n\beta_n<2/\theta$ (where $\theta$ is the Lipschitz constant of $\nabla \Psi$), we obtain weak ergodic convergence of the sequence $(x_n)$ to an equilibrium for a general maximal monotone operator $A$. We also obtain weak convergence of the whole sequence $(x_n)$ when $A$ is the subdifferential of a proper lower-semicontinuous convex function. As a key ingredient of our analysis, we use the cocoerciveness of the operator $\nabla \Psi$. When specializing our results to coupled systems, we bring new light to Passty's theorem and obtain convergence results of new parallel splitting algorithms for variational inequalities involving coupling in the constraint. We also establish robustness and stability results that account for numerical approximation errors. An illustration of compressive sensing is given.

Maximum-Volume Symmetric Gauge Ball Problem on the Convex Cone of Positive Definite Matrices and Convexity of Optimal Sets

Yongdo Lim

SIAM J. Optim. 21, pp. 1275-1288 (14 pages)

Online Publication Date: November 01, 2011

Full Text: | Download PDF

Show Abstract
In this paper we show that the radius of the largest symmetric gauge ball inscribed in the convex cone ${\mathrm{PD}}(n)$ of $n\times n$ positive definite matrices equipped with the Finsler metric inherited from a unitarily invariant norm is precisely the minimal eigenvalue of its center. We then solve Todd's largest dual ellipsoids problem [Math. Program. Ser. B, 117 (2009), pp. 425–434] for unitarily invariant norms, i.e., the problem of maximizing the product of the unitarily invariant norm distances to boundaries of the cone and its dual cone. We further show that the optimal set of maximizers forms a convex cone and is a closed geodesically convex subset of the Riemannian manifold ${\mathrm{PD}}(n)$. This in particular provides a one-parameter family of strictly increasing solid convex (in both a Euclidean and a Riemannian sense) cones of positive definite matrices which starts from the optimal set and covers ${\mathrm{PD}}(n)$.

Products of Finitely Many Resolvents of Maximal Monotone Mappings in Reflexive Banach Spaces

Shoham Sabach

SIAM J. Optim. 21, pp. 1289-1308 (20 pages)

Online Publication Date: November 08, 2011

Full Text: | Download PDF

Show Abstract
We propose two algorithms for finding (common) zeros of finitely many maximal monotone mappings in reflexive Banach spaces. These algorithms are based on the Bregman distance related to a well-chosen convex function and improve previous results. Finally, we mention two applications of our algorithms for solving equilibrium problems and convex feasibility problems.

An LQP-Based Decomposition Method for Solving a Class of Variational Inequalities

Xiaoming Yuan and Min Li

SIAM J. Optim. 21, pp. 1309-1318 (10 pages)

Online Publication Date: November 22, 2011

Full Text: | Download PDF

Show Abstract
The alternating direction method (ADM) is an influential decomposition method for solving a class of variational inequalities with block-separable structures. In the literature, the subproblems of the ADM are usually regularized by quadratic proximal terms to ensure a more stable and attractive numerical performance. In this paper, we propose to apply the logarithmic-quadratic proximal (LQP) terms to regularize the ADM subproblems, and thus develop an LQP-based decomposition method for solving a class of variational inequalities. Global convergence of the new method is proved under standard assumptions.

Iterative Methods for Solving Systems of Variational Inequalities in Reflexive Banach Spaces

Gábor Kassay, Simeon Reich, and Shoham Sabach

SIAM J. Optim. 21, pp. 1319-1344 (26 pages)

Online Publication Date: November 22, 2011

Full Text: | Download PDF

Show Abstract
We prove strong convergence theorems for three iterative algorithms which approximate solutions to systems of variational inequalities for mappings of monotone type. All the theorems are set in reflexive Banach spaces and take into account possible computational errors.

Regularity on a Fixed Set

A. D. Ioffe

SIAM J. Optim. 21, pp. 1345-1370 (26 pages)

Online Publication Date: December 01, 2011

Full Text: | Download PDF

Show Abstract
This paper is devoted to noninfinitesimal methods in nonlocal regularity theory for set-valued mappings between metric spaces and concentrates on studying two main interconnected topics: noninfinitesimal regularity criteria and fixed-points of set-valued mappings. A number of new results are proved, in particular those which cover and extend to a general metric setting some theorems viewed specifically as Banach-space results. In addition, a special technical interest in studying these two topics together is determined by the fact that each of them exploits a certain sequential iteration scheme (connected with Ekeland's principle in the first and Newton-type iterations in the second) and the extent to which each of the schemes can be effectively applied to the study of the other topic is at least unclear.

Comparison and Automated Selection of Local Optimization Solvers for Interval Global Optimization Methods

Mihály Csaba Markót and Hermann Schichl

SIAM J. Optim. 21, pp. 1371-1391 (21 pages)

Online Publication Date: December 01, 2011

Full Text: | Download PDF

Show Abstract
We compare six state-of-the-art local optimization solvers, with a focus on their efficiency when invoked within an interval-based global optimization algorithm. For comparison purposes we design three special performance indicators: a solution check indicator (measuring whether the local minimizers found are good candidates for near-optimal verified feasible points), a function value indicator (measuring the contribution to the progress of the global search), and a running time indicator (estimating the computational cost of the local search within the global search). The solvers are compared on the COCONUT Environment test set consisting of 1307 problems. Our main goal is to predict the behavior of the solvers in terms of the three performance indicators on a new problem. For this we introduce a $k$-nearest neighbor method applied over a feature space consisting of several categorical and numerical features of the optimization problems. The quality and robustness of the prediction is demonstrated by various quality measurements with detailed comparative tests. In particular, we found that on the test set we are able to pick a “best” solver in 66–89% of the cases and avoid picking all “useless” solvers in 95–99% of the cases (when a useful alternative exists). The resulting automated solver selection method is implemented as an inference engine of the COCONUT Environment.

Nonsingularity Conditions for the Fischer–Burmeister System of Nonlinear SDPs

Shujun Bi, Shaohua Pan, and Jein-Shan Chen

SIAM J. Optim. 21, pp. 1392-1417 (26 pages)

Online Publication Date: December 06, 2011

Full Text: | Download PDF

Show Abstract
For a locally optimal solution to the nonlinear semidefinite programming problem, under Robinson's constraint qualification, we show that the nonsingularity of Clarke's Jacobian of the Fischer–Burmeister (FB) nonsmooth system is equivalent to the strong regularity of the Karush–Kuhn–Tucker point. Consequently, from Sun's paper [Math. Oper. Res., 31 (2006), pp. 761–776] the semismooth Newton method applied to the FB system may attain the locally quadratic convergence under the strong second order sufficient condition and constraint nondegeneracy.

Piecewise-quadratic Approximations in Convex Numerical Optimization

A. Astorino, A. Frangioni, M. Gaudioso, and E. Gorgone

SIAM J. Optim. 21, pp. 1418-1438 (21 pages)

Online Publication Date: December 08, 2011

Full Text: | Download PDF

Show Abstract
We present a bundle method for convex nondifferentiable minimization where the model is a piecewise-quadratic convex approximation of the objective function. Unlike standard bundle approaches, the model only needs to support the objective function from below at a properly chosen (small) subset of points, as opposed to everywhere. We provide the convergence analysis for the algorithm, with a general form of master problem which combines features of trust region stabilization and proximal stabilization, taking care of all the important practical aspects such as proper handling of the proximity parameters and the bundle of information. Numerical results are also reported.

First Order and Second Order Characterizations of Metric Subregularity and Calmness of Constraint Set Mappings

Helmut Gfrerer

SIAM J. Optim. 21, pp. 1439-1474 (36 pages)

Online Publication Date: December 13, 2011

Full Text: | Download PDF

Show Abstract
A condition ensuring metric subregularity (respectively, calmness) of general multifunctions between Banach spaces is derived. This condition is expressed solely in terms of the given data at the reference point and does not involve any information concerning the solution set of the corresponding inclusion given by the multifunction. In finite dimensions this condition can be expressed in terms of a derivative which appears to be a combination of the coderivative and the contingent derivative. It is further shown that this sufficient condition is in some sense the weakest possible first order condition sufficient for subregularity. We extend this condition under the additional assumption that one part of the multifunction is known to be subregular in advance. We also derive second order conditions for metric subregularity, both sufficient and necessary, for multifunctions associated with constraint systems as they occur in optimization. We show that the main difference between the necessary and sufficient conditions is the replacement of an inequality by a strict inequality, just as in the case of “no gap” second order optimality conditions in optimization.

KKT Solution and Conic Relaxation for Solving Quadratically Constrained Quadratic Programming Problems

Cheng Lu, Shu-Cherng Fang, Qingwei Jin, Zhenbo Wang, and Wenxun Xing

SIAM J. Optim. 21, pp. 1475-1490 (16 pages)

Online Publication Date: December 15, 2011

Full Text: | Download PDF

Show Abstract
To find a global optimal solution to the quadratically constrained quadratic programming problem, we explore the relationship between its Lagrangian multipliers and related linear conic programming problems. This study leads to a global optimality condition that is more general than the known positive semidefiniteness condition in the literature. Moreover, we propose a computational scheme that provides clues of designing effective algorithms for more solvable quadratically constrained quadratic programming problems.

Nonconvex Games with Side Constraints

Jong-Shi Pang and Gesualdo Scutari

SIAM J. Optim. 21, pp. 1491-1522 (32 pages)

Online Publication Date: December 15, 2011

Full Text: | Download PDF

Show Abstract
This paper develops an optimization-based theory for the existence and uniqueness of equilibria of a noncooperative game wherein the selfish players' optimization problems are nonconvex and there are side constraints and an associated price clearance to be satisfied by the equilibria. A new concept of equilibrium for such a nonconvex game, which we term a “quasi-Nash equilibrium” (QNE), is introduced as a solution of the variational inequality (VI) obtained by aggregating the first-order optimality conditions of the players' problems while retaining the convex constraints (if any) in the defining set of the VI. Under a second-order sufficiency condition from nonlinear programming, a QNE becomes a local Nash equilibrium of the game. Uniqueness of a QNE is established using a degree-theoretic proof. Under a key boundedness property of the Karush–Kuhn–Tucker multipliers of the nonconvex constraints and the positive definiteness of the Hessians of the players' Lagrangian functions, we establish the single-valuedness of the players' best-response maps, from which the existence of a Nash equilibrium (NE) of the nonconvex game follows. We also present a distributed algorithm for computing an NE of such a game and provide a matrix-theoretic condition for the convergence of the algorithm. An application is presented that pertains to a special multi-leader-follower game wherein the nonconvexity is due to the followers' equilibrium conditions in the leaders' optimization problems. Another application to a cognitive radio paradigm in a signal processing game is described in detail in [G. Scutari and J.S. Pang, IEEE Trans. Inform. Theory, submitted; J.S. Pang and G. Scutari, Joint IEEE Trans. Signal Process, submitted].

Weak Sharp Minima on Riemannian Manifolds

Chong Li, Boris S. Mordukhovich, Jinhua Wang, and Jen-Chih Yao

SIAM J. Optim. 21, pp. 1523-1560 (38 pages)

Online Publication Date: December 15, 2011

Full Text: | Download PDF

Show Abstract
This is the first paper dealing with the study of weak sharp minima for constrained optimization problems on Riemannian manifolds, which are important in many applications. We consider the notions of local weak sharp minima, boundedly weak sharp minima, and global weak sharp minima for such problems and establish their complete characterizations in the case of convex problems on finite-dimensional Riemannian manifolds and Hadamard manifolds. A number of the results obtained in this paper are also new for the case of conventional problems in finite-dimensional Euclidean spaces. Our methods involve appropriate tools of variational analysis and generalized differentiation on Riemannian and Hadamard manifolds developed and efficiently implemented in this paper.

First-Order Optimality Conditions for Elliptic Mathematical Programs with Equilibrium Constraints via Variational Analysis

M. Hintermüller and T. Surowiec

SIAM J. Optim. 21, pp. 1561-1593 (33 pages)

Online Publication Date: December 15, 2011

Full Text: | Download PDF

Show Abstract
Mathematical programs in which the constraint set is partially defined by the solutions of an elliptic variational inequality, so-called “elliptic MPECs,” are formulated in reflexive Banach spaces. With the goal of deriving explicit first-order optimality conditions amenable to the development of numerical procedures, variational analytic concepts are both applied and further developed. The paper is split into two main parts. The first part concerns the derivation of conditions in which the (lower-level) state constraints are assumed to be polyhedric sets. This part is then completed by two examples, the latter of which involves pointwise bilateral bounds on the gradient of the state. The second part focuses on an important nonpolyhedric example, namely, when the lower-level state constraints are presented by pointwise bounds on the Euclidean norm of the gradient of the state. A formula for the second-order (Mosco) epiderivative of the indicator function for this convex set is derived. This result is then used to demonstrate the (Hadamard) directional differentiability of the solution mapping of the variational inequality, which then leads to the derivation of explicit strong stationarity conditions for this problem.

Mixing Sets Linked by Bidirected Paths

Marco Di Summa and Laurence A. Wolsey

SIAM J. Optim. 21, pp. 1594-1613 (20 pages)

Online Publication Date: December 15, 2011

Full Text: | Download PDF

Show Abstract
Recently there has been considerable research on simple mixed-integer sets, called mixing sets, and closely related sets arising in uncapacitated and constant capacity lot-sizing. This in turn has led to study of more general sets, called network-dual sets, for which it is possible to derive extended formulations whose projection gives the convex hull of the network-dual set. Unfortunately this formulation cannot be used (in general) to optimize in polynomial time. Furthermore the inequalities defining the convex hull of a network-dual set in the original space of variables are known only for some special cases. Here we study two new cases, in which the continuous variables of the network-dual set are linked by a bidirected path. In the first case, which is motivated by lot-sizing problems with (lost) sales, we provide a description of the convex hull as the intersection of the convex hulls of $2^n$ mixing sets, where $n$ is the number of continuous variables of the set. However optimization is polynomial as only $n+1$ of the sets are required for any given objective function. In the second case, generalizing single arc flow sets, we describe again the convex hull as the intersection of an exponential number of mixing sets and also give a combinatorial polynomial-time separation algorithm.

Low-rank Matrix Recovery via Iteratively Reweighted Least Squares Minimization

Massimo Fornasier, Holger Rauhut, and Rachel Ward

SIAM J. Optim. 21, pp. 1614-1640 (27 pages)

Online Publication Date: December 15, 2011

Full Text: | Download PDF

Show Abstract
We present and analyze an efficient implementation of an iteratively reweighted least squares algorithm for recovering a matrix from a small number of linear measurements. The algorithm is designed for the simultaneous promotion of both a minimal nuclear norm and an approximately low-rank solution. Under the assumption that the linear measurements fulfill a suitable generalization of the null space property known in the context of compressed sensing, the algorithm is guaranteed to recover iteratively any matrix with an error of the order of the best $k$-rank approximation. In certain relevant cases, for instance, for the matrix completion problem, our version of this algorithm can take advantage of the Woodbury matrix identity, which allows us to expedite the solution of the least squares problems required at each iteration. We present numerical experiments which confirm the robustness of the algorithm for the solution of matrix completion problems, and we demonstrate its competitiveness with respect to other techniques proposed recently in the literature.

A Sequential Semismooth Newton Method for the Nearest Low-rank Correlation Matrix Problem

Qingna Li and Hou-duo Qi

SIAM J. Optim. 21, pp. 1641-1666 (26 pages)

Online Publication Date: December 15, 2011

Full Text: | Download PDF

Show Abstract
Based on the well-known result that the sum of the largest eigenvalues of a symmetric matrix can be represented as a semidefinite programming problem (SDP), we formulate the nearest low-rank correlation matrix problem as a nonconvex SDP and propose a numerical method that solves a sequence of least-square problems. Each of the least-square problems can be solved by a specifically designed semismooth Newton method, which is shown to be quadratically convergent. The sequential method is guaranteed to produce a stationary point of the nonconvex SDP. Our numerical results demonstrate the high efficiency of the proposed method on large scale problems.

Self-concordance and Decomposition-based Interior Point Methods for the Two-stage Stochastic Convex Optimization Problem

Michael Chen and Sanjay Mehrotra

SIAM J. Optim. 21, pp. 1667-1687 (21 pages)

Online Publication Date: December 20, 2011

Full Text: | Download PDF

Show Abstract
We study the two-stage stochastic convex optimization problem whose first- and second-stage feasible regions admit a self-concordant barrier. We show that the barrier recourse functions and the composite barrier functions for this problem form self-concordant families. These results are used to develop prototype primal interior point decomposition algorithms that are more suitable for a heterogeneous distributed computing environment. We show that the worst case iteration complexity of the proposed algorithms is the same as that for the short- and long-step primal interior algorithms applied to the extensive formulation of this problem. The generality of our results allows the possibility of using barriers other than the standard log-barrier in an algorithmic framework.

Complexity of Variants of Tseng's Modified F-B Splitting and Korpelevich's Methods for Hemivariational Inequalities with Applications to Saddle-point and Convex Optimization Problems

Renato D. C. Monteiro and B. F. Svaiter

SIAM J. Optim. 21, pp. 1688-1720 (33 pages)

Online Publication Date: December 22, 2011

Full Text: | Download PDF

Show Abstract
In this paper, we consider both a variant of Tseng's modified forward-backward splitting method and an extension of Korpelevich's method for solving hemivariational inequalities with Lipschitz continuous operators. By showing that these methods are special cases of the hybrid proximal extragradient method introduced by Solodov and Svaiter, we derive iteration-complexity bounds for them to obtain different types of approximate solutions. In the context of saddle-point problems, we also derive complexity bounds for these methods to obtain another type of an approximate solution, namely, that of an approximate saddle point. Finally, we illustrate the usefulness of the above results by applying them to a large class of linearly constrained convex programming problems, including, for example, cone programming and problems whose objective functions converge to infinity as the boundaries of their effective domains are approached.

On the Evaluation Complexity of Composite Function Minimization with Applications to Nonconvex Nonlinear Programming

Coralia Cartis, Nicholas I. M. Gould, and Philippe L. Toint

SIAM J. Optim. 21, pp. 1721-1739 (19 pages)

Online Publication Date: December 22, 2011

Full Text: | Download PDF

Show Abstract
We estimate the worst-case complexity of minimizing an unconstrained, nonconvex composite objective with a structured nonsmooth term by means of some first-order methods. We find that it is unaffected by the nonsmoothness of the objective in that a first-order trust-region or quadratic regularization method applied to it takes at most $\mathcal{O}(\epsilon^{-2})$ function evaluations to reduce the size of a first-order criticality measure below $\epsilon$. Specializing this result to the case when the composite objective is an exact penalty function allows us to consider the objective- and constraint-evaluation worst-case complexity of nonconvex equality-constrained optimization when the solution is computed using a first-order exact penalty method. We obtain that in the reasonable case when the penalty parameters are bounded, the complexity of reaching within $\epsilon$ of a KKT point is at most $\mathcal{O}(\epsilon^{-2})$ problem evaluations, which is the same in order as the function-evaluation complexity of steepest-descent methods applied to unconstrained, nonconvex smooth optimization.
Close

Your Recent History Recent History Help

Recently Viewed

  1. A Moving Balls Approximation Method for a Class of Smooth Constrained Minimization Problems
  2. Exploiting Sparsity in SDP Relaxation for Sensor Network Localization
  3. Benchmarking Derivative-Free Optimization Algorithms
  4. On the Solution of the Tikhonov Regularization of the Total Least Squares Problem
  5. On the Stability of Convex-valued Mappings and Their Relative Boundary and Extreme Points Set Mappin...
© SIAM By using SIAM Publications Online you agree to abide by the Terms and Conditions of Use. Banner art adapted from a figure by Hinke M. Osinga and Bernd Krauskopf (University of Bristol, UK).
close