SIAM Digital Library
 
 
 

SIAM J. on Optimization

All Journal content published prior to 1997 is part of LOCUS.

Search Issue | RSS Feeds RSS
Previous Issue

2010

Volume 20, Issue 6, pp. 2755-3576


On the Complexity of the Hybrid Proximal Extragradient Method for the Iterates and the Ergodic Mean

Renato D. C. Monteiro and B. F. Svaiter

SIAM J. Optim. 20, pp. 2755-2787 (33 pages) | Cited 1 time

Online Publication Date: August 17, 2010

Full Text: | Download PDF

Show Abstract
In this paper we analyze the iteration complexity of the hybrid proximal extragradient (HPE) method for finding a zero of a maximal monotone operator recently proposed by Solodov and Svaiter. One of the key points of our analysis is the use of new termination criteria based on the $\varepsilon$-enlargement of a maximal monotone operator. The advantage of using these termination criteria is that their definition do not depend on the boundedness of the domain of the operator. We then show that Korpelevich's extragradient method for solving monotone variational inequalities falls in the framework of the HPE method. As a consequence, using the complexity analysis of the HPE method, we obtain new complexity bounds for Korpelevich's extragradient method which do not require the feasible set to be bounded, as assumed in a recent paper by Nemirovski. Another feature of our analysis is that the derived iteration-complexity bounds are proportional to the distance of the initial point to the solution set. The HPE framework is also used to obtain the first iteration-complexity result for Tseng's modified forward-backward splitting method for finding a zero of the sum of a monotone Lipschitz continuous map with an arbitrary maximal monotone operator whose resolvent is assumed to be easily computable. Also using the framework of the HPE method, we study the complexity of a variant of a Newton-type extragradient algorithm proposed by Solodov and Svaiter for finding a zero of a smooth monotone function with Lipschitz continuous Jacobian.

Variational Analysis in Semi-Infinite and Infinite Programming, II: Necessary Optimality Conditions

M. J. Cánovas, M. A. López, B. S. Mordukhovich, and J. Parra

SIAM J. Optim. 20, pp. 2788-2806 (19 pages)

Online Publication Date: August 19, 2010

Full Text: | Download PDF

Show Abstract
This paper concerns applications of advanced techniques of variational analysis and generalized differentiation to problems of semi-infinite and infinite programming with feasible solution sets defined by parameterized systems of infinitely many linear inequalities of the type intensively studied in the preceding development [Cánovas et al., SIAM J. Optim., 20 (2009), pp. 1504–1526] from the viewpoint of robust Lipschitzian stability. The main results establish necessary optimality conditions for broad classes of semi-infinite and infinite programs, where objectives are generally described by nonsmooth and nonconvex functions on Banach spaces and where infinite constraint inequality systems are indexed by arbitrary sets. The results obtained are new in both smooth and nonsmooth settings of semi-infinite and infinite programming. We illustrate our model and results by considering a practically meaningful model of water resource optimization via systems of reservoirs.

Trading Accuracy for Sparsity in Optimization Problems with Sparsity Constraints

Shai Shalev-Shwartz, Nathan Srebro, and Tong Zhang

SIAM J. Optim. 20, pp. 2807-2832 (26 pages)

Online Publication Date: August 19, 2010

Full Text: | Download PDF

Show Abstract
We study the problem of minimizing the expected loss of a linear predictor while constraining its sparsity, i.e., bounding the number of features used by the predictor. While the resulting optimization problem is generally NP-hard, several approximation algorithms are considered. We analyze the performance of these algorithms, focusing on the characterization of the trade-off between accuracy and sparsity of the learned predictor in different scenarios.

On the Complexity of Steepest Descent, Newton's and Regularized Newton's Methods for Nonconvex Unconstrained Optimization Problems

C. Cartis, N. I. M. Gould, and Ph. L. Toint

SIAM J. Optim. 20, pp. 2833-2852 (20 pages) | Cited 2 times

Online Publication Date: September 09, 2010

Full Text: | Download PDF

Show Abstract
It is shown that the steepest-descent and Newton's methods for unconstrained nonconvex optimization under standard assumptions may both require a number of iterations and function evaluations arbitrarily close to $O(\epsilon^{-2})$ to drive the norm of the gradient below $\epsilon$. This shows that the upper bound of $O(\epsilon^{-2})$ evaluations known for the steepest descent is tight and that Newton's method may be as slow as the steepest-descent method in the worst case. The improved evaluation complexity bound of $O(\epsilon^{-3/2})$ evaluations known for cubically regularized Newton's methods is also shown to be tight.

A New Class of Large Neighborhood Path-Following Interior Point Algorithms for Semidefinite Optimization with $O(\sqrt{n}\log\frac{\mathrm{Tr}(X^0S^0)}{\epsilon})$ Iteration Complexity

Yang Li and Tamás Terlaky

SIAM J. Optim. 20, pp. 2853-2875 (23 pages)

Online Publication Date: September 23, 2010

Full Text: | Download PDF

Show Abstract
In this paper, we extend the Ai–Zhang direction to the class of semidefinite optimization problems. We define a new wide neighborhood $\mathcal{N}(\tau_1,\tau_2,\eta)$, and, as usual but with a small change, we make use of the scaled Newton equations for symmetric search directions. After defining the “positive part” and the “negative part” of a symmetric matrix, we recommend solving the Newton equation with its right-hand side replaced first by its positive part and then by its negative part, respectively. In such a way, we obtain a decomposition of the classical Newton direction and use different step lengths for each of them. Starting with a feasible point $(X^0,y^0,S^0)$ in $\mathcal{N}(\tau_1,\tau_2,\eta)$, the algorithm terminates in at most $O(\eta\sqrt{\kappa_{\infty}n}\log({Tr}(X^0S^0)/\epsilon))$ iterations, where $\kappa_{\infty}$ is a parameter associated with the scaling matrix $P$ and $\epsilon$ is the required precision. To our best knowledge, when the parameter $\eta$ is a constant, this is the first large neighborhood path-following interior point method (IPM) with the same complexity as small neighborhood path-following IPMs for semidefinite optimization that use the Nesterov–Todd direction. In the case where $\eta$ is chosen to be in the order of $\sqrt{n}$, our result coincides with the results for classical large neighborhood IPMs. Some preliminary numerical results also confirm the efficiency of the algorithm.

Computing Rational Points in Convex Semialgebraic Sets and Sum of Squares Decompositions

Mohab Safey El Din and Lihong Zhi

SIAM J. Optim. 20, pp. 2876-2889 (14 pages)

Online Publication Date: September 23, 2010

Full Text: | Download PDF

Show Abstract
Let $\mathcal{P}=\{h_1,\dots,h_s\}\subset\mathbb{Z}[Y_1,\dots,Y_k]$, $D\geq\deg(h_i)$ for $1\leq i\leq s$, $\sigma$ bounding the bit length of the coefficients of the $h_i$'s, and let $\Phi$ be a quantifier-free $\mathcal{P}$-formula defining a convex semialgebraic set. We design an algorithm returning a rational point in $\mathcal{S}$ if and only if $\mathcal{S}\cap\mathbb{Q}\neq\emptyset$. It requires $\sigma^{\mathrm{O}(1)}D^{\mathrm{O}(k^3)}$ bit operations. If a rational point is outputted, its coordinates have bit length dominated by $\sigma D^{\mathrm{O}(k^3)}$. Using this result, we obtain a procedure for deciding whether a polynomial $f\in\mathbb{Z}[X_1,\dots,X_n]$ is a sum of squares of polynomials in $\mathbb{Q}[X_1,\dots,X_n]$. Denote by $d$ the degree of $f$, $\tau$ the maximum bit length of the coefficients in $f$, $D={n+d\choose n}$, and $k\leq D(D+1)-{n+2d\choose n}$. This procedure requires $\tau^{\mathrm{O}(1)}D^{\mathrm{O}(k^3)}$ bit operations, and the coefficients of the outputted polynomials have bit length dominated by $\tau D^{\mathrm{O}(k^3)}$.

Constrained Infinite Group Relaxations of MIPs

Santanu S. Dey and Laurence A. Wolsey

SIAM J. Optim. 20, pp. 2890-2912 (23 pages) | Cited 1 time

Online Publication Date: September 29, 2010

Full Text: | Download PDF

Show Abstract
Recently minimal and extreme inequalities for continuous group relaxations of general mixed integer sets have been characterized. In this paper, we consider a stronger relaxation of general mixed integer sets by allowing constraints, such as bounds, on the free integer variables in the continuous group relaxation. We generalize a number of results for the continuous infinite group relaxation to this stronger relaxation and characterize the extreme inequalities when there are two integer variables.

An Optimal Algorithm for Minimization of Quadratic Functions with Bounded Spectrum Subject to Separable Convex Inequality and Linear Equality Constraints

Zdeněk Dostál and Radek Kučera

SIAM J. Optim. 20, pp. 2913-2938 (26 pages)

Online Publication Date: October 07, 2010

Full Text: | Download PDF

Show Abstract
An, in a sense, optimal algorithm for minimization of quadratic functions subject to separable convex inequality and linear equality constraints is presented. Its unique feature is an error bound in terms of bounds on the spectrum of the Hessian of the cost function. If applied to a class of problems with the spectrum of the Hessians in a given positive interval, the algorithm can find approximate solutions in a uniformly bounded number of simple iterations, such as matrix-vector multiplications. Moreover, if the class of problems admits a sparse representation of the Hessian, it simply follows that the cost of the solution is proportional to the number of unknowns. Theoretical results are illustrated by numerical experiments.

Finding Efficient Solutions by Free Disposal Outer Approximation

Daniel Gourion and Dinh The Luc

SIAM J. Optim. 20, pp. 2939-2958 (20 pages)

Online Publication Date: October 07, 2010

Full Text: | Download PDF

Show Abstract
The main purpose of this paper is to present a method for generating a representation of efficient solutions and efficient values for nonconvex multiobjective optimization problems. The method is based on a particular outer approximation of nonconvex sets by so-called free disposal nonconvex polyhedra. The convergence of the method is proven, and a discussion on the rate of convergence is addressed with emphasis on the case of two objectives. Examples of up to four objectives are tested, some of which are taken from the recent literature. The numerical experience shows that the optimal value representation obtained by the algorithm is nicely distributed among the set of efficient values of the problem.

A New Exchange Method for Convex Semi-Infinite Programming

Liping Zhang, Soon-Yi Wu, and Marco A. López

SIAM J. Optim. 20, pp. 2959-2977 (19 pages)

Online Publication Date: October 12, 2010

Full Text: | Download PDF

Show Abstract
In this paper we propose a new exchange method for solving convex semi-infinite programming (CSIP) problems. We introduce a new dropping-rule in the proposed exchange algorithm, which only keeps those active constraints with positive Lagrange multipliers. Moreover, we exploit the idea of looking for $\eta$-infeasible indices of the lower level problem as the adding-rule in our algorithm. Hence the algorithm does not require to solve a maximization problem over the index set at each iteration; it only needs to find some points such that a certain computationally-easy criterion is satisfied. Under some reasonable conditions, the new adding-dropping rule guarantees that our algorithm provides an approximate optimal solution for the CSIP problem in a finite number of iterations. In the numerical experiments, we apply the proposed algorithm to solve some test problems from the literature, including some medium-sized problems from complex approximation theory and FIR filter design. We compare our algorithm with an existing central cutting plane algorithm and with the semi-infinite solver fseminf in MATLAB toolbox, and we find that our algorithm solves the CSIP problem much faster. For the FIR filter design problem, we show that our algorithm solves the problem better than some algorithms that were technically established for the problem.

Feasibility of Integer Knapsacks

Iskander Aliev and Martin Henk

SIAM J. Optim. 20, pp. 2978-2993 (16 pages)

Online Publication Date: October 12, 2010

Full Text: | Download PDF

Show Abstract
Given a matrix $A\in\mathbb{Z}^{m\times n}$ satisfying certain regularity assumptions, we consider the set $\mathcal{F}(A)$ of all vectors $\boldsymbol{b}\in\mathbb{Z}^m$ such that the associated knapsack polytope $P(A,\boldsymbol{b})=\{\boldsymbol{x}\in\mathbb{R}^n_{\geq0}:A\boldsymbol{x}=\boldsymbol{b}\}$ contains an integer point. When $m=1$ the set $\mathcal{F}(A)$ is known to contain all consecutive integers greater than the Frobenius number associated with $A$. In this paper we introduce the diagonal Frobenius number $\mathrm{g}(A)$ which reflects in an analogous way feasibility properties of the problem and the structure of $\mathcal{F}(A)$ in the general case. We give an optimal upper bound for $\mathrm{g}(A)$ and also estimate the asymptotic growth of the diagonal Frobenius number on average.

Solving Log-Determinant Optimization Problems by a Newton-CG Primal Proximal Point Algorithm

Chengjing Wang, Defeng Sun, and Kim-Chuan Toh

SIAM J. Optim. 20, pp. 2994-3013 (20 pages)

Online Publication Date: October 14, 2010

Full Text: | Download PDF

Show Abstract
We propose a Newton-CG primal proximal point algorithm (PPA) for solving large scale log-determinant optimization problems. Our algorithm employs the essential ideas of PPA, the Newton method, and the preconditioned CG solver. When applying the Newton method to solve the inner subproblem, we find that the log-determinant term plays the role of a smoothing term as in the traditional smoothing Newton technique. Focusing on the problem of maximum likelihood sparse estimation of a Gaussian graphical model, we demonstrate that our algorithm performs favorably compared to existing state-of-the-art algorithms and is much preferred when a high quality solution is required for problems with many equality constraints.

Unified Analysis of Kernel-Based Interior-Point Methods for $P_*(\kappa)$-Linear Complementarity Problems

G. Lesaja and C. Roos

SIAM J. Optim. 20, pp. 3014-3039 (26 pages)

Online Publication Date: October 19, 2010

Full Text: | Download PDF

Show Abstract
We present an interior-point method for the $P_*(\kappa)$-linear complementarity problem (LCP) that is based on barrier functions which are defined by a large class of univariate functions called eligible kernel functions. This class is fairly general and includes the classical logarithmic function and the self-regular functions, as well as many non-self-regular functions as special cases. We provide a unified analysis of the method and give a general scheme on how to calculate the iteration bounds for the entire class. We also calculate the iteration bounds of both long-step and short-step versions of the method for several specific eligible kernel functions. For some of them we match the best known iteration bounds for the long-step method, while for the short-step method the iteration bounds are of the same order of magnitude. As far as we know, this is the first paper that provides a unified approach and comprehensive treatment of interior-point methods for $P_*(\kappa)$-LCPs based on the entire class of eligible kernel functions. (The title of this article has been corrected.)

Generalized Hessians of $C^{1,1}$-Functions and Second-Order Viscosity Subjets

Luc Barbet, Aris Daniilidis, and Pierpaolo Soravia

SIAM J. Optim. 20, pp. 3040-3058 (19 pages)

Online Publication Date: October 19, 2010

Full Text: | Download PDF

Show Abstract
Given a $C^{1,1}$–function $f:U\rightarrow R$ (where $U\subset R^{n}$ open), we deal with the question of whether or not at a given point $x_{0}\in U$ there exists a local minorant $\varphi$ of $f$ of class $C^{2}$ that satisfies $\varphi(x_{0})=f(x_{0})$, $D\varphi(x_{0})=Df(x_{0})$, and $D^{2}\varphi(x_{0})\in\mathcal{H}f(x_{0})$ (the generalized Hessian of $f$ at $x_{0}$). This question is motivated by the second-order viscosity theory of the PDEs, since for nonsmooth functions, an analogous result between subgradients and first-order viscosity subjets is known to hold in every separable Asplund space. In this work we show that the aforementioned second-order result holds true whenever $\mathcal{H}f(x_{0})$ has a minimum with respect to the positive semidefinite cone (thus, in particular, in one dimension), but it fails in two dimensions even for piecewise polynomial functions. We extend this result by introducing a new notion of directional minimum of $\mathcal{H}f(x_{0})$.

Universal Rigidity and Edge Sparsification for Sensor Network Localization

Zhisu Zhu, Anthony Man-Cho So, and Yinyu Ye

SIAM J. Optim. 20, pp. 3059-3081 (23 pages)

Online Publication Date: October 19, 2010

Full Text: | Download PDF

Show Abstract
Owing to their high accuracy and ease of formulation, there has been great interest in applying convex optimization techniques, particularly that of semidefinite programming (SDP) relaxation, to tackle the sensor network localization problem in recent years. However, a drawback of such techniques is that the resulting convex program is often expensive to solve. In order to speed up computation, various edge sparsification heuristics have been proposed, whose aim is to reduce the number of edges in the input graph. Although these heuristics do reduce the size of the convex program and hence make it faster to solve, they are often ad hoc in nature and do not preserve the localization properties of the input. As such, one often has to face a tradeoff between solution accuracy and computational effort. In this paper, we propose a novel edge sparsification heuristic that can provably preserve the localization properties of the original input. At the heart of our heuristic is a graph decomposition procedure, which allows us to identify certain sparse generically universally rigid subgraphs of the input graph. Our computational results show that the proposed approach can significantly reduce the computational and memory complexities of SDP-based algorithms for solving the sensor network localization problem. Moreover, it compares favorably with existing speedup approaches, both in terms of accuracy and solution time.

Representations of Positive Polynomials and Optimization on Noncompact Semialgebraic Sets

Hà Huy Vui and Pham Tien So'n

SIAM J. Optim. 20, pp. 3082-3103 (22 pages)

Online Publication Date: October 19, 2010

Full Text: | Download PDF

Show Abstract
This paper studies the representation of a positive polynomial $f$ on a closed semialgebraic set $S:=\{x\in\mathbb{R}^n\mid g_i(x)=0, i=1,\dots,l, h_j(x)\geq0, j=1,\dots,m\}$ modulo the so-called critical ideal $I(f,S)$ of $f$ on $S$. Under a constraint qualification condition, it is demonstrated that, if either $f>0$ on $S$ or $f\geq0$ on $S$ and the critical ideal $I(f,S)$ is radical, then $f$ belongs to the preordering generated by the polynomials $h_1,\dots,h_m$ modulo the critical ideal $I(f,S)$. These facts imply that we can find a natural sequence of semidefinite programs whose optimal values converge monotonically, increasing to the infimum value $f^*:=\inf_{x\in S}f(x)$ of $f$ on $S$, provided that the infimum value is attained at some point. Besides, we shall construct a finite set in $\mathbb{R}$ containing the infimum value $f^*$. Moreover, some relations between the Fedoryuk [Soviet Math. Dokl., 17 (1976), pp. 486–490] and Malgrange [Complex Analysis, Microlocal Calculus and Relativistic Quantum Theory, Lecture Notes in Phys. 126, Springer, Berlin, 1980, pp. 170–177] conditions and coercivity for polynomials, which are bounded from below on $S$, are also established. In particular, a sufficient condition for $f$ to attain its infimum on $S$ is derived from these facts. We also show that every polynomial $f$, which is bounded from below on $S$, can be approximated in the $l_1$-norm of coefficients by a sequence of polynomials $f_\epsilon$ that are coercive. Finally, it is shown that almost every linear polynomial function, which is bounded from below on $S$, attains its infimum on $S$ and has the same asymptotic growth at infinity.

Error Bounds for Some Semidefinite Programming Approaches to Polynomial Minimization on the Hypercube

Etienne de Klerk and Monique Laurent

SIAM J. Optim. 20, pp. 3104-3120 (17 pages)

Online Publication Date: October 19, 2010

Full Text: | Download PDF

Show Abstract
We consider the problem of minimizing a polynomial on the hypercube $[0,1]^n$ and derive new error bounds for the hierarchy of semidefinite programming approximations to this problem corresponding to the Positivstellensatz of Schmüdgen [Math. Ann., 289 (1991), pp. 203–206]. The main tool we employ is Bernstein approximations of polynomials, which also gives constructive proofs and degree bounds for positivity certificates on the hypercube.

AFPTAS Results for Common Variants of Bin Packing: A New Method for Handling the Small Items

Leah Epstein and Asaf Levin

SIAM J. Optim. 20, pp. 3121-3145 (25 pages)

Online Publication Date: October 19, 2010

Full Text: | Download PDF

Show Abstract
We consider two well-known natural variants of bin packing and show that these packing problems admit asymptotic fully polynomial time approximation schemes (AFPTASs). In bin packing problems, a set of one-dimensional items of size at most 1 is to be assigned (packed) to subsets of sum at most 1 (bins). It has been known for a while that the most basic problem admits an AFPTAS. In this paper, we develop new methods that allow us to extend this result to other variants of bin packing consisting of a family of two-parameter bin packing problems. We demonstrate the use of our methods by designing AFPTASs for the following problems. The first problem is bin packing with cardinality constraints, where a parameter $k$ is given such that a bin may contain up to $k$ items. The goal is to minimize the number of bins used. The second problem is bin packing with rejection, where every item has a rejection penalty associated with it. An item needs to be either packed to a bin or rejected, and the goal is to minimize the number of bins used and the total rejection penalty of unpacked items. This resolves the complexity of two important variants of the bin packing problem. Our approximation schemes use a novel method for packing the small items. This new method is the core of the improved running times of our schemes over the running times of the previous results, which are only asymptotic polynomial time approximationschemes.

The Geometry of Strict Maximality

E. Casini and E. Miglierina

SIAM J. Optim. 20, pp. 3146-3160 (15 pages)

Online Publication Date: October 21, 2010

Full Text: | Download PDF

Show Abstract
The notion of a strictly maximal point is a concept of proper maximality that plays an important role in the study of the stability of vector optimization problems. The aim of this paper is to study some properties of this notion with particular attention to geometrical aspects. More precisely, we individuate some relationships between strict maximality and the properties of the bases of the ordering cone. In order to prove this result, a new characterization of the existence of a bounded base for a closed convex cone is given. Moreover, we link strict maximality to the geometrical notion of strongly exposed points of a given set. Finally, we deal with the linear scalarization for the strictly maximal points.

Global Descent Method for Global Optimization

Chi-Kong Ng, Duan Li, and Lian-Sheng Zhang

SIAM J. Optim. 20, pp. 3161-3184 (24 pages)

Online Publication Date: October 21, 2010

Full Text: | Download PDF

Show Abstract
This paper develops a novel method—the global descent method—for solving a general class of global optimization problems. This method moves from one local minimizer of the objective function $f$ to a better one at each iteration with the help of an auxiliary function termed the global descent function. The global descent function is not only guaranteed to have a local minimizer $x'$ over the problem domain in $\mathbb{R}^n$ but also ensures that each of its local minimizers is located in some neighborhoods of a better minimizer of $f$ with $f(x')<f(x^*)$. These features of the global descent function enable a global descent to be achieved at each iteration using only local descent methods. Computational experiments conducted on several test problems with up to 1000 variables demonstrate the applicability of the proposed method. Furthermore, numerical comparison experiments carried out with GAMS/BARON on several test problems also justify the efficiency and effectiveness of the proposed method.

A Randomized Cutting Plane Method with Probabilistic Geometric Convergence

F. Dabbene, P. S. Shcherbakov, and B. T. Polyak

SIAM J. Optim. 20, pp. 3185-3207 (23 pages)

Online Publication Date: October 21, 2010

Full Text: | Download PDF

Show Abstract
We propose a randomized method for general convex optimization problems; namely, the minimization of a linear function over a convex body. The idea is to generate $N$ random points inside the body, choose the best one, and cut the part of the body defined by the linear constraint. We analyze the convergence properties of the algorithm from a theoretical viewpoint, i.e., under the rather standard assumption that an algorithm for uniform generation of random points in the convex body is available. Under this assumption, the expected rate of convergence for such method is proved to be geometric. This analysis is based on new results on the statistical properties of the empirical minimum over a convex body that we obtained in this paper. Moreover, explicit sample size results on convergence are derived. In particular, we compute the minimum number of random points that should be generated at each step in order to guarantee that, in a probabilistic sense, the method performs better than the deterministic center-of-gravity algorithm. From a practical viewpoint, we show how the method can be implemented using hit-and-run versions of Markov-chain Monte Carlo algorithms and exemplify the performance of this implementable modification via a number of illustrative problems. A crucial notion for the hit-and-run implementation is that of boundary oracle, which is available for most optimization problems including linear matrix inequalities and many other kinds of constraints. Preliminary numerical results for semidefinite programs are presented confirming that the randomized approach might be competitive to modern deterministic convex optimization methods.

Optimality Conditions via Exact Penalty Functions

K. W. Meng and X. Q. Yang

SIAM J. Optim. 20, pp. 3208-3231 (24 pages)

Online Publication Date: October 26, 2010

Full Text: | Download PDF

Show Abstract
In this paper, we study KKT optimality conditions for constrained nonlinear programming problems and strong and Mordukhovich stationarities for mathematical programs with complementarity constraints using $l_p$ penalty functions, with $0\leq p\leq1$. We introduce some optimality indication sets by using contingent derivatives of penalty function terms. Some characterizations of optimality indication sets are obtained by virtue of the original problem data. We show that the KKT optimality condition holds at a feasible point if this point is a local minimizer of some $l_p$ penalty function with $p$ belonging to the optimality indication set. Our result on constrained nonlinear programming includes some existing results from the literature as special cases.

A Moving Balls Approximation Method for a Class of Smooth Constrained Minimization Problems

Alfred Auslender, Ron Shefi, and Marc Teboulle

SIAM J. Optim. 20, pp. 3232-3259 (28 pages)

Online Publication Date: October 26, 2010

Full Text: | Download PDF

Show Abstract
We introduce a new algorithm for a class of smooth constrained minimization problems which is an iterative scheme that generates a sequence of feasible points that approximates the constraints set by a sequence of balls and is accordingly called the Moving Balls Approximation algorithm (MBA). The computational simplicity of MBA, which uses first order data information, makes it suitable for large scale problems. Theoretical and computational properties of MBA in its primal and dual forms are studied, and convergence and global rate of convergence results are established for nonconvex and convex problems. We then introduce a variant of MBA that includes an active set technique and is particularly suitable for problems with a large number of constraints. This variant is as simple as MBA and is proven to preserve the convergence properties of MBA. Extension of MBA is also developed for a class of variational inequalities. Initial numerical experiments on quadratically constrained problems demonstrate the viability and performance of our methods when compared to some existing state-of-the-art optimization methods/software such as a sequential quadratic programming solver from the IMSL Library and the CVXOPT software package for convex optimization.

Fully Distributed Algorithms for Convex Optimization Problems

Damon Mosk-Aoyama, Tim Roughgarden, and Devavrat Shah

SIAM J. Optim. 20, pp. 3260-3279 (20 pages)

Online Publication Date: October 28, 2010

Full Text: | Download PDF

Show Abstract
We design and analyze a fully distributed algorithm for convex constrained optimization in networks without any consistent naming infrastructure. The algorithm produces an approximately feasible and near-optimal solution in time polynomial in the network size, the inverse of the permitted error, and a measure of curvature variation in the dual optimization problem. It blends, in a novel way, gossip-based information spreading, iterative gradient ascent, and the barrier method from the design of interior-point algorithms.

Stability of Error Bounds for Convex Constraint Systems in Banach Spaces

Alexander Kruger, Huynh Van Ngai, and Michel Théra

SIAM J. Optim. 20, pp. 3280-3296 (17 pages) | Cited 1 time

Online Publication Date: October 28, 2010

Full Text: | Download PDF

Show Abstract
This paper studies stability of error bounds for convex constraint systems in Banach spaces. We show that certain known sufficient conditions for local and global error bounds actually ensure error bounds for the family of functions being in a sense small perturbations of the given one. A single inequality as well as semi-infinite constraint systems are considered.

Integer Programming Subject to Monomial Constraints

Christoph Buchheim, Dennis Michaels, and Robert Weismantel

SIAM J. Optim. 20, pp. 3297-3311 (15 pages)

Online Publication Date: November 09, 2010

Full Text: | Download PDF

Show Abstract
We investigate integer programs containing constraints of the type $\prod_{i\in I}x_i^{\alpha_i}=b$. Due to the number-theoretic nature of these constraints, standard methods based on linear algebra cannot be applied directly. Instead, we present a reformulation resulting in integer programs with linear constraints and polynomial objective functions, using prime decompositions of the right-hand sides $b$. Moreover, we show that minimizing a linear objective function with nonnegative coefficients over bivariate constraints is possible in polynomial time.

Sharp Primal Superlinear Convergence Results for Some Newtonian Methods for Constrained Optimization

D. Fernández, A. F. Izmailov, and M. V. Solodov

SIAM J. Optim. 20, pp. 3312-3334 (23 pages) | Cited 1 time

Online Publication Date: November 09, 2010

Full Text: | Download PDF

Show Abstract
As is well known, $Q$-superlinear or $Q$-quadratic convergence of the primal-dual sequence generated by an optimization algorithm does not, in general, imply $Q$-superlinear convergence of the primal part. Primal convergence, however, is often of particular interest. For the sequential quadratic programming (SQP) algorithm, local primal-dual quadratic convergence can be established under the assumptions of uniqueness of the Lagrange multiplier associated to the solution and the second-order sufficient condition. At the same time, previous primal $Q$-superlinear convergence results for SQP required strengthening of the first assumption to the linear independence constraint qualification. In this paper, we show that this strengthening of assumptions is actually not necessary. Specifically, we show that once primal-dual convergence is assumed or already established, for primal superlinear rate one needs only a certain error bound estimate. This error bound holds, for example, under the second-order sufficient condition, which is needed for primal-dual local analysis in any case. Moreover, in some situations even second-order sufficiency can be relaxed to the weaker assumption that the multiplier in question is noncritical. Our study is performed for a rather general perturbed SQP framework which covers, in addition to SQP and quasi-Newton SQP, some other algorithms as well. For example, as a byproduct, we obtain primal $Q$-superlinear convergence results for the linearly constrained (augmented) Lagrangian methods for which no primal $Q$-superlinear rate of convergence results were previously available. Another application of the general framework is sequential quadratically constrained quadratic programming methods. Finally, we discuss some difficulties with proving primal superlinear convergence for the stabilized version of SQP.

A Primal-Dual Exterior Point Method for Nonlinear Optimization

Hiroshi Yamashita and Takahito Tanabe

SIAM J. Optim. 20, pp. 3335-3363 (29 pages)

Online Publication Date: November 09, 2010

Full Text: | Download PDF

Show Abstract
In this paper, primal-dual methods for general nonconvex nonlinear optimization problems are considered. The proposed methods are exterior point type methods that permit primal variables to violate inequality constraints during the iterations. The methods are based on the exact penalty type transformation of inequality constraints and use a smooth approximation of the problem to form primal-dual iteration based on Newton's method as in usual primal-dual interior point methods. Global convergence and local superlinear/quadratic convergence of the proposed methods are proved. For global convergence, methods using line searches and trust region type searches are proposed. The trust region type method is tested with CUTEr problems and is shown to have similar efficiency to the primal-dual interior point method code IPOPT. It is also shown that the methods can be warm started easily, unlike interior point methods, and that the methods can be efficiently used in parametric programming problems.

Positivity and Optimization for Semi-Algebraic Functions

Jean B. Lasserre and Mihai Putinar

SIAM J. Optim. 20, pp. 3364-3383 (20 pages) | Cited 1 time

Online Publication Date: December 02, 2010

Full Text: | Download PDF

Show Abstract
We describe algebraic certificates of positivity for functions belonging to a finitely generated algebra of Borel measurable functions, with particular emphasis on algebras generated by semi-algebraic functions. In this case the standard global optimization problem, with constraints given by elements of the same algebra, is reduced via a natural change of variables to the better-understood case of polynomial optimization. A collection of simple examples and numerical experiments complement the theoretical parts of the article.

Strong Duality in Robust Convex Programming: Complete Characterizations

V. Jeyakumar and G. Y. Li

SIAM J. Optim. 20, pp. 3384-3407 (24 pages)

Online Publication Date: December 02, 2010

Full Text: | Download PDF

Show Abstract
Duality theory has played a key role in convex programming in the absence of data uncertainty. In this paper, we present a duality theory for convex programming problems in the face of data uncertainty via robust optimization. We characterize strong duality between the robust counterpart of an uncertain convex program and the optimistic counterpart of its uncertain Lagrangian dual. We provide a new robust characteristic cone constraint qualification which is necessary and sufficient for strong duality in the sense that the constraint qualification holds if and only if strong duality holds for every convex objective function of the program. We further show that this strong duality always holds for uncertain polyhedral convex programming problems by verifying our constraint qualification, where the uncertainty set is a polytope. We derive these results by way of first establishing a robust theorem of the alternative for parameterized convex inequality systems using conjugate analysis. We also give a convex characteristic cone constraint qualification that is necessary and sufficient for strong duality between the deterministic dual pair: the robust counterpart and its Lagrangian dual. Through simple numerical examples we also provide an insightful account of the development of our duality theory.

Estimating Bounds for Quadratic Assignment Problems Associated with Hamming and Manhattan Distance Matrices Based on Semidefinite Programming

Hans Mittelmann and Jiming Peng

SIAM J. Optim. 20, pp. 3408-3426 (19 pages)

Online Publication Date: December 02, 2010

Full Text: | Download PDF

Show Abstract
Quadratic assignment problems (QAPs) with a Hamming distance matrix for a hypercube or a Manhattan distance matrix for a rectangular grid arise frequently from communications and facility locations and are known to be among the hardest discrete optimization problems. In this paper we consider the issue of how to obtain lower bounds for those two classes of QAPs based on semidefinite programming (SDP). By exploiting the data structure of the distance matrix $B$, we first show that for any permutation matrix $X$, the matrix $Y=\alpha E-XBX^T$ is positive semidefinite, where $\alpha$ is a properly chosen parameter depending only on the associated graph in the underlying QAP and $E=ee^T$ is a rank-1 matrix whose elements have value 1. This results in a natural way to approximate the original QAPs via SDP relaxation based on the matrix-splitting technique. Our new SDP relaxations have a smaller size compared with other SDP relaxations in the literature and can be solved efficiently by most open source SDP solvers. Experimental results show that for the underlying QAPs of size up to $n=200$, strong bounds can be obtained effectively.

Random Convex Programs

Giuseppe Carlo Calafiore

SIAM J. Optim. 20, pp. 3427-3464 (38 pages)

Online Publication Date: December 02, 2010

Full Text: | Download PDF

Show Abstract
Random convex programs (RCPs) are convex optimization problems subject to a finite number $N$ of random constraints. The optimal objective value $J^*$ of an RCP is thus a random variable. We study the probability with which $J^*$ is no longer optimal if a further random constraint is added to the problem (violation probability, $V^*$). It turns out that this probability rapidly concentrates near zero as $N$ increases. We first develop a theory for RCPs, leading to explicit bounds on the upper tail probability of $V^*$. Then we extend the setup to the case of RCPs with $r$ a posteriori violated constraints (RCPVs): a paradigm that permits us to improve the optimal objective value while maintaining the violation probability under control. Explicit and nonasymptotic bounds are derived also in this case: the upper tail probability of $V^*$ is upper bounded by a multiple of a beta distribution, irrespective of the distribution on the random constraints. All results are derived under no feasibility assumptions on the problem. Further, the relation between RCPVs and chance-constrained problems (CCP) is explored, showing that the optimal objective $J^*$ of an RCPV with the generic constraint removal rule provides, with arbitrarily high probability, an upper bound on the optimal objective of a corresponding CCP. Moreover, whenever an optimal constraint removal rule is used in the RCPVs, then appropriate choices of $N$ and $r$ exist such that $J^*$ approximates arbitrarily well the objective of the CCP.

Trace Norm Regularization: Reformulations, Algorithms, and Multi-Task Learning

Ting Kei Pong, Paul Tseng, Shuiwang Ji, and Jieping Ye

SIAM J. Optim. 20, pp. 3465-3489 (25 pages)

Online Publication Date: December 02, 2010

Full Text: | Download PDF

Show Abstract
We consider a recently proposed optimization formulation of multi-task learning based on trace norm regularized least squares. While this problem may be formulated as a semidefinite program (SDP), its size is beyond general SDP solvers. Previous solution approaches apply proximal gradient methods to solve the primal problem. We derive new primal and dual reformulations of this problem, including a reduced dual formulation that involves minimizing a convex quadratic function over an operator-norm ball in matrix space. This reduced dual problem may be solved by gradient-projection methods, with each projection involving a singular value decomposition. The dual approach is compared with existing approaches and its practical effectiveness is illustrated on simulations and an application to gene expression pattern analysis.

Applying Metric Regularity to Compute a Condition Measure of a Smoothing Algorithm for Matrix Games

Boris S. Mordukhovich, Javier F. Peña, and Vera Roshchina

SIAM J. Optim. 20, pp. 3490-3511 (22 pages)

Online Publication Date: December 08, 2010

Full Text: | Download PDF

Show Abstract
We develop an approach of variational analysis and generalized differentiation to conditioning issues for two-person zero-sum matrix games. Our major results establish precise relationships between a certain condition measure of the smoothing first-order algorithm proposed by Gilpin, Peña, and Sandholm [Proceedings of the 23rd Conference on Artificial Intelligence, 2008, pp. 75–82] and the exact bound of metric regularity for an associated set-valued mapping. In this way we compute the aforementioned condition measure in terms of the initial matrix game data.

Self-Correcting Geometry in Model-Based Algorithms for Derivative-Free Unconstrained Optimization

K. Scheinberg and Ph. L. Toint

SIAM J. Optim. 20, pp. 3512-3532 (21 pages) | Cited 1 time

Online Publication Date: December 16, 2010

Full Text: | Download PDF

Show Abstract
Several efficient methods for derivative-free optimization are based on the construction and maintenance of an interpolation model for the objective function. Most of these algorithms use special “geometry-improving” iterations, where the geometry (poisedness) of the underlying interpolation set is made better at the cost of one or more function evaluations. We show that such geometry improvements cannot be completely eliminated if one wishes to ensure global convergence, but we also provide an algorithm where such steps occur only in the final stage of the algorithm, where criticality of a putative stationary point is verified. Global convergence for this method is proved by making use of a self-correction mechanism inherent to the combination of trust regions and interpolation models. This mechanism also throws some light on the surprisingly good numerical results reported by Fasano, Morales, and Nocedal [Optim. Methods Softw., 24 (2009), pp. 145–154] for a method where no care is ever taken to guarantee poisedness of the interpolation set.

A New Sequential Optimality Condition for Constrained Optimization and Algorithmic Consequences

Roberto Andreani, J. M. Martínez, and B. F. Svaiter

SIAM J. Optim. 20, pp. 3533-3554 (22 pages)

Online Publication Date: December 16, 2010

Full Text: | Download PDF

Show Abstract
Necessary first-order sequential optimality conditions provide adequate theoretical tools to justify stopping criteria for nonlinear programming solvers. Sequential optimality conditions are satisfied by local minimizers of optimization problems independently of the fulfillment of constraint qualifications. A new condition of this type is introduced in the present paper. It is proved that a well-established augmented Lagrangian algorithm produces sequences whose limits satisfy the new condition. Practical consequences are discussed.

A Derivative-Free Algorithm for Least-Squares Minimization

Hongchao Zhang, Andrew R. Conn, and Katya Scheinberg

SIAM J. Optim. 20, pp. 3555-3576 (22 pages)

Online Publication Date: December 16, 2010

Full Text: | Download PDF

Show Abstract
We develop a framework for a class of derivative-free algorithms for the least-squares minimization problem. These algorithms are designed to take advantage of the problem structure by building polynomial interpolation models for each function in the least-squares minimization. Under suitable conditions, global convergence of the algorithm is established within a trust region framework. Promising numerical results indicate the algorithm is both efficient and robust. Numerical comparisons are made with standard derivative-free software packages that do not exploit the special structure of the least-squares problem or that use finite differences to approximate the gradients.
Close

Your Recent History Recent History Help

Recently Viewed

  1. On the Solution of the Tikhonov Regularization of the Total Least Squares Problem
  2. On the Stability of Convex-valued Mappings and Their Relative Boundary and Extreme Points Set Mappin...
  3. Constraint Reduction for Linear Programs with Many Inequality Constraints
  4. A Note on Trust-Region Radius Update
  5. Convergence of the Iterates of Descent Methods for Analytic Cost Functions
© SIAM By using SIAM Publications Online you agree to abide by the Terms and Conditions of Use. Banner art adapted from a figure by Hinke M. Osinga and Bernd Krauskopf (University of Bristol, UK).
close