SIAM Digital Library
 
 
 

SIAM J. on Optimization

All Journal content published prior to 1997 is part of LOCUS.

Search Issue | RSS Feeds RSS
Next Issue

2009

Volume 20, Issue 1, pp. 1-571


A Derivative-Free Algorithm for Inequality Constrained Nonlinear Programming via Smoothing of an $\ell_\infty$ Penalty Function

G. Liuzzi and S. Lucidi

SIAM J. Optim. 20, pp. 1-29 (29 pages) | Cited 1 time

Online Publication Date: March 13, 2009

Full Text: | Download PDF

Show Abstract
In this paper we consider inequality constrained nonlinear optimization problems where the first order derivatives of the objective function and the constraints cannot be used. Our starting point is the possibility to transform the original constrained problem into an unconstrained or linearly constrained minimization of a nonsmooth exact penalty function. This approach shows two main difficulties: the first one is the nonsmoothness of this class of exact penalty functions which may cause derivative-free codes to converge to nonstationary points of the problem; the second one is the fact that the equivalence between stationary points of the constrained problem and those of the exact penalty function can only be stated when the penalty parameter is smaller than a threshold value which is not known a priori. In this paper we propose a derivative-free algorithm which overcomes the preceding difficulties and produces a sequence of points that admits a subsequence converging to a Karush–Kuhn–Tucker point of the constrained problem. In particular the proposed algorithm is based on a smoothing of the nondifferentiable exact penalty function and includes an updating rule which, after at most a finite number of updates, is able to determine a “right value” for the penalty parameter. Furthermore we present the results obtained on a real world problem concerning the estimation of parameters in an insulin-glucose model of the human body.

An Adaptive Linear Approximation Algorithm for Copositive Programs

Stefan Bundfuss and Mirjam Dür

SIAM J. Optim. 20, pp. 30-53 (24 pages)

Online Publication Date: March 13, 2009

Full Text: | Download PDF

Show Abstract
We study linear optimization problems over the cone of copositive matrices. These problems appear in nonconvex quadratic and binary optimization; for instance, the maximum clique problem and other combinatorial problems can be reformulated as such problems. We present new polyhedral inner and outer approximations of the copositive cone which we show to be exact in the limit. In contrast to previous approximation schemes, our approximation is not necessarily uniform for the whole cone but can be guided adaptively through the objective function, yielding a good approximation in those parts of the cone that are relevant for the optimization and only a coarse approximation in those parts that are not. Using these approximations, we derive an adaptive linear approximation algorithm for copositive programs. Numerical experiments show that our algorithm gives very good results for certain nonconvex quadratic problems.

Convergent Network Approximation for the Continuous Euclidean Length Constrained Minimum Cost Path Problem

Ranga Muhandiramge, Natashia Boland, and Song Wang

SIAM J. Optim. 20, pp. 54-77 (24 pages)

Online Publication Date: March 13, 2009

Full Text: | Download PDF

Show Abstract
In many path-planning situations we would like to find a path of constrained Euclidean length in $\mathbb{R}^2$ that minimizes a line integral. We call this the Continuous Length-Constrained Minimum Cost Path Problem (C-LCMCPP). Generally, this will be a nonconvex optimization problem, for which continuous approaches ensure only locally optimal solutions. However, network discretizations yield weight constrained network shortest path problems (WCSPPs), which can in practice be solved to global optimality, even for large networks; we can readily find a globally optimal solution to an approximation of the C-LCMCPP. Solutions to these WCSPPs yield feasible solutions and hence upper bounds. We show how networks can be constructed, and a WCSPP in these networks formulated, so that the solutions provide lower bounds on the global optimum of the continuous problem. We give a general convergence scheme for our network discretizations and use it to prove that both the upper and lower bounds so generated converge to the global optimum of the C-LCMCPP, as the network discretization is refined. Our approach provides a computable lower bound formula (of course, the upper bounds are readily computable). We give computational results showing the lower bound formula in practice, and compare the effectiveness of our network construction technique with that of standard grid-based approaches in generating good quality solutions. We find that for the same computational effort, we are able to find better quality solutions, particularly when the length constraint is tighter.

A New Regularization Scheme for Mathematical Programs with Complementarity Constraints

Abdeslam Kadrani, Jean-Pierre Dussault, and Abdelhamid Benchakroun

SIAM J. Optim. 20, pp. 78-103 (26 pages)

Online Publication Date: March 13, 2009

Full Text: | Download PDF

Show Abstract
We propose a new regularization scheme for mathematical programs with complementarity constraints (MPCC) by relaxing all the constraints of the complementarity system. We show that, under the MPCC-linear independence constraint qualifications (MPCC-LICQ), the Lagrange multipliers exist for this regularization. Our method has strong convergence properties under MPCC-linear independence constraint qualifications and some weak conditions of the strict complementarity. In particular, under MPCC-LICQ, it is shown that any accumulation point of the regularized stationary points is M-stationary for the MPCC problem, and if the asymptotically weak nondegeneracy condition holds at a stationary point of the regularized problem, then it is strongly stationary. An algorithm for solving the proposed regularization is presented and numerical experiments are reported. Some comparisons with other methods are discussed with illustrative examples.

Stability Analysis of Optimal Control Problems with a Second-Order State Constraint

Audrey Hermant

SIAM J. Optim. 20, pp. 104-129 (26 pages)

Online Publication Date: March 25, 2009

Full Text: | Download PDF

Show Abstract
This paper gives stability results for nonlinear optimal control problems subject to a regular state constraint of second-order. The strengthened Legendre–Clebsch condition is assumed to hold, and no assumption on the structure of the contact set is made. Under a weak second-order sufficient condition (taking into account the active constraints), we show that the solutions are Lipschitz continuous w.r.t. the perturbation parameter in the $L^2$ norm, and Hölder continuous in the $L^\infty$ norm. We use a generalized implicit function theorem in metric spaces by Dontchev and Hager [SIAM J. Control Optim., 36 (1998), pp. 698–718]. The difficulty is that multipliers associated with second-order state constraints have a low regularity (they are only bounded measures). We obtain Lipschitz stability of a “primitive” of the state constraint multiplier.

A Sequential Convex Semidefinite Programming Algorithm with an Application to Multiple-Load Free Material Optimization

M. Stingl, M. Kočvara, and G. Leugering

SIAM J. Optim. 20, pp. 130-155 (26 pages) | Cited 1 time

Online Publication Date: March 25, 2009

Full Text: | Download PDF

Show Abstract
A new method for the efficient solution of a class of convex semidefinite programming (SDP) problems is introduced. The method extends the sequential convex programming (SCP) concept to optimization problems with matrix variables. The basic idea of the new method is to approximate the original optimization problem by a sequence of subproblems, in which nonlinear functions (defined in matrix variables) are approximated by block separable convex functions. The subproblems are semidefinite programs with a favorable structure which can be efficiently solved by existing SDP software. The new method is shown to be globally convergent. The article is concluded by a series of numerical experiments with free material optimization problems demonstrating the effectiveness of the generalized SCP approach.

The Smoothed Spectral Abscissa for Robust Stability Optimization

Joris Vanbiervliet, Bart Vandereycken, Wim Michiels, Stefan Vandewalle, and Moritz Diehl

SIAM J. Optim. 20, pp. 156-171 (16 pages)

Online Publication Date: March 25, 2009

Full Text: | Download PDF

Show Abstract
This paper concerns the stability optimization of (parameterized) matrices $A(x)$, a problem typically arising in the design of fixed-order or fixed-structured feedback controllers. It is well known that the minimization of the spectral abscissa function $\alpha(A)$ gives rise to very difficult optimization problems, since $\alpha(A)$ is not everywhere differentiable and even not everywhere Lipschitz. We therefore propose a new stability measure, namely, the smoothed spectral abscissa $\tilde\alpha_{\epsilon}(A)$, which is based on the inversion of a relaxed $H_2$-type cost function. The regularization parameter $\epsilon$ allows tuning the degree of smoothness. For $\epsilon$ approaching zero, the smoothed spectral abscissa converges towards the nonsmooth spectral abscissa from above so that $\tilde\alpha_{\epsilon}(A)\leq0$ guarantees asymptotic stability. Evaluation of the smoothed spectral abscissa and its derivatives w.r.t. matrix parameters $x$ can be performed at the cost of solving a primal-dual Lyapunov equation pair, allowing for an efficient integration into a derivative-based optimization framework. Two optimization problems are considered: On the one hand, the minimization of the smoothed spectral abscissa $\tilde\alpha_{\epsilon}(A(x))$ as a function of the matrix parameters for a fixed value of $\epsilon$, and, on the other hand, the maximization of $\epsilon$ such that the stability requirement $\tilde\alpha_{\epsilon}(A(x))\leq0$ is still satisfied. The latter problem can be interpreted as an $H_2$-norm minimization problem, and its solution additionally implies an upper bound on the corresponding $H_\infty$-norm or a lower bound on the distance to instability. In both cases, additional equality and inequality constraints on the variables can be naturally taken into account in the optimization problem.

Benchmarking Derivative-Free Optimization Algorithms

Jorge J. Moré and Stefan M. Wild

SIAM J. Optim. 20, pp. 172-191 (20 pages) | Cited 3 times

Online Publication Date: March 25, 2009

Full Text: | Download PDF

Show Abstract
We propose data profiles as a tool for analyzing the performance of derivative-free optimization solvers when there are constraints on the computational budget. We use performance and data profiles, together with a convergence test that measures the decrease in function value, to analyze the performance of three solvers on sets of smooth, noisy, and piecewise-smooth problems. Our results provide estimates for the performance difference between these solvers, and show that on these problems, the model-based solver tested performs better than the two direct search solvers tested.

Exploiting Sparsity in SDP Relaxation for Sensor Network Localization

Sunyoung Kim, Masakazu Kojima, and Hayato Waki

SIAM J. Optim. 20, pp. 192-215 (24 pages) | Cited 3 times

Online Publication Date: April 01, 2009

Full Text: | Download PDF

Show Abstract
A sensor network localization problem can be formulated as a quadratic optimization problem (QOP). For QOPs, semidefinite programming (SDP) relaxation by Lasserre with relaxation order 1 for general polynomial optimization problems (POPs) is known to be equivalent to the sparse SDP relaxation by Waki et al. with relaxation order 1, except for the size and sparsity of the resulting SDP relaxation problems. We show that the sparse SDP relaxation applied to the QOP is at least as strong as the Biswas–Ye SDP relaxation for the sensor network localization problem. A sparse variant of the Biswas–Ye SDP relaxation, which is equivalent to the original Biswas–Ye SDP relaxation, is also derived. We compare numerically the sparse SDP relaxation applied to the QOP, the Biswas–Ye SDP relaxation, its sparse variant, and the edge-based SDP relaxation by Wang et al. to confirm the effectiveness of the proposed techniques for exploiting the sparsity in SDP relaxation for sensor network localization problems. The sparse SDP relaxation applied to the QOP is much faster than the Biswas–Ye SDP relaxation, and the sparse variant of the Biswas–Ye SDP relaxation outperforms all other SDP relaxations in speed.

What Shape Is Your Conjugate? A Survey of Computational Convex Analysis and Its Applications

Yves Lucet

SIAM J. Optim. 20, pp. 216-250 (35 pages) | Cited 1 time

Online Publication Date: April 01, 2009

Full Text: | Download PDF

Show Abstract
Computational convex analysis algorithms have been rediscovered several times in the past by researchers from different fields. To further communications between practitioners, we review the field of computational convex analysis, which focuses on the numerical computation of fundamental transforms arising from convex analysis. Current models use symbolic, numeric, and hybrid symbolic-numeric algorithms. Our objective is to disseminate widely the most efficient numerical algorithms useful for applications in image processing (computing the distance transform, the generalized distance transform, and mathematical morphology operators), partial differential equations (solving Hamilton–Jacobi equations and using differential equations numerical schemes to compute the convex envelope), max-plus algebra (computing the equivalent of the fast Fourier transform), multifractal analysis, etc. The fields of applications include, among others, computer vision, robot navigation, thermodynamics, electrical networks, medical imaging, and network communication.

Some Sufficient Optimality Conditions in Nonsmooth Analysis

A. Eberhard and R. Wenczel

SIAM J. Optim. 20, pp. 251-296 (46 pages)

Online Publication Date: April 03, 2009

Full Text: | Download PDF

Show Abstract
In this paper we study some second order conditions that may be added to the first order nessessary optimality condition $0\in\partial_{p}f(\bar{x})$ (with $\partial_{p}f$ denoting the proximal subdifferential) in order to obtain a sufficient condition for a strict local minimum for extended–real-valued, nonsmooth functions. Three different types of second order conditions are investigated, all based on a different second order subdifferential. Namely, the subhessian, the graphical derivative, and the (contingent) coderivative to the proximal subdifferential.

On the Local Convergence of Semismooth Newton Methods for Linear and Nonlinear Second-Order Cone Programs Without Strict Complementarity

Christian Kanzow, Izabella Ferenczi, and Masao Fukushima

SIAM J. Optim. 20, pp. 297-320 (24 pages) | Cited 1 time

Online Publication Date: April 03, 2009

Full Text: | Download PDF

Show Abstract
The optimality conditions of a nonlinear second-order cone program can be reformulated as a nonsmooth system of equations using a projection mapping. This allows the application of nonsmooth Newton methods for the solution of the nonlinear second-order cone program. Conditions for the local quadratic convergence of these nonsmooth Newton methods are investigated. Related conditions are also given for the special case of a linear second-order cone program. An interesting and important feature of these conditions is that they do not require strict complementarity of the solution. Some numerical results are included in order to illustrate the theoretical considerations.

Generating Set Search Methods for Piecewise Smooth Problems

C. Bogani, M. G. Gasparo, and A. Papini

SIAM J. Optim. 20, pp. 321-335 (15 pages)

Online Publication Date: April 24, 2009

Full Text: | Download PDF

Show Abstract
We consider a direct search approach for solving nonsmooth minimization problems where the objective function is locally Lipschitz continuous and piecewise continuously differentiable on a finite family of polyhedra. A generating set search method is proposed, which is named structured because the structure of the set of nondifferentiability near the current iterate is exploited to define the search directions at each iteration. Some numerical results are presented to validate the approach.

Regularization Methods for Semidefinite Programming

Jérôme Malick, Janez Povh, Franz Rendl, and Angelika Wiegele

SIAM J. Optim. 20, pp. 336-356 (21 pages) | Cited 4 times

Online Publication Date: April 24, 2009

Full Text: | Download PDF

Show Abstract
We introduce a new class of algorithms for solving linear semidefinite programming (SDP) problems. Our approach is based on classical tools from convex optimization such as quadratic regularization and augmented Lagrangian techniques. We study the theoretical properties and we show that practical implementations behave very well on some instances of SDP having a large number of constraints. We also show that the “boundary point method” from Povh, Rendl, and Wiegele [Computing, 78 (2006), pp. 277–286] is an instance of this class.

Convergence Analysis of Deflected Conditional Approximate Subgradient Methods

Giacomo d'Antonio and Antonio Frangioni

SIAM J. Optim. 20, pp. 357-386 (30 pages)

Online Publication Date: April 24, 2009

Full Text: | Download PDF

Show Abstract
Subgradient methods for nondifferentiable optimization benefit from deflection, i.e., defining the search direction as a combination of the previous direction and the current subgradient. In the constrained case they also benefit from projection of the search direction onto the feasible set prior to computing the steplength, that is, from the use of conditional subgradient techniques. However, combining the two techniques is not straightforward, especially if an inexact oracle is available which can only compute approximate function values and subgradients. We present a convergence analysis of several different variants, both conceptual and implementable, of approximate conditional deflected subgradient methods. Our analysis extends the available results in the literature by using the main stepsize rules presented so far, while allowing deflection in a more flexible way. Furthermore, to allow for (diminishing/square summable) rules where the stepsize is tightly controlled a priori, we propose a new class of deflection-restricted approaches where it is the deflection parameter, rather than the stepsize, which is dynamically adjusted using the “target value” of the optimization sequence. For both Polyak-type and diminishing/square summable stepsizes, we propose a “correction” of the standard formula which shows that, in the inexact case, knowledge about the error computed by the oracle (which is available in several practical applications) can be exploited in order to strengthen the convergence properties of the method. The analysis allows for several variants of the algorithm; at least one of them is likely to show numerical performances similar to these of “heavy ball” subgradient methods, popular within backpropagation approaches to train neural networks, while possessing stronger convergence properties.

Global Convergence of General Derivative-Free Trust-Region Algorithms to First- and Second-Order Critical Points

Andrew R. Conn, Katya Scheinberg, and Luís N. Vicente

SIAM J. Optim. 20, pp. 387-415 (29 pages) | Cited 3 times

Online Publication Date: April 29, 2009

Full Text: | Download PDF

Show Abstract
In this paper we prove global convergence for first- and second-order stationary points of a class of derivative-free trust-region methods for unconstrained optimization. These methods are based on the sequential minimization of quadratic (or linear) models built from evaluating the objective function at sample sets. The derivative-free models are required to satisfy Taylor-type bounds, but, apart from that, the analysis is independent of the sampling techniques. A number of new issues are addressed, including global convergence when acceptance of iterates is based on simple decrease of the objective function, trust-region radius maintenance at the criticality step, and global convergence for second-order critical points.

Shape Optimization in Three-Dimensional Contact Problems with Coulomb Friction

P. Beremlijski, J. Haslinger, M. Kočvara, R. Kučera, and J.V. Outrata

SIAM J. Optim. 20, pp. 416-444 (29 pages)

Online Publication Date: April 29, 2009

Full Text: | Download PDF

Show Abstract
We study the discretized problem of the shape optimization of three-dimensional (3D) elastic bodies in unilateral contact. The aim is to extend existing results to the case of contact problems obeying the Coulomb friction law. Mathematical modeling of the Coulomb friction problem leads to an implicit variational inequality. It is shown that for small coefficients of friction the discretized problem with Coulomb friction has a unique solution and that this solution is Lipschitzian as a function of a control variable describing the shape of the elastic body. The 2D case of this problem was studied by the authors in [P. Beremlijski, J. Haslinger, M. Kočvara, and J. V. Outrata, SIAM J. Optim., 13 (2002), pp. 561–587]; there we used the so-called implicit programming approach combined with the generalized differential calculus of Clarke. The extension of this technique to the 3D situation is by no means straightforward. The main source of difficulties is the nonpolyhedral character of the second-order (Lorentz) cone, arising in the 3D model. To facilitate the computation of the subgradient information, needed in the used numerical method, we exploit the substantially richer generalized differential calculus of Mordukhovich. Numerical examples illustrate the efficiency and reliability of the suggested approach.

A Progressive Barrier for Derivative-Free Nonlinear Programming

Charles Audet and J. E. Dennis, Jr.

SIAM J. Optim. 20, pp. 445-472 (28 pages) | Cited 3 times

Online Publication Date: April 29, 2009

Full Text: | Download PDF

Show Abstract
We propose a new constraint-handling approach for general constraints that is applicable to a widely used class of constrained derivative-free optimization methods. As in many methods that allow infeasible iterates, constraint violations are aggregated into a single constraint violation function. As in filter methods, a threshold, or barrier, is imposed on the constraint violation function, and any trial point whose constraint violation function value exceeds this threshold is discarded from consideration. In the new algorithm, unlike the filter method, the amount of constraint violation subject to the barrier is progressively decreased adaptively as the iteration evolves. We test this progressive barrier (PB) approach versus the extreme barrier (EB) with the generalized pattern search (Gps) and the lower triangular mesh adaptive direct search (LTMads) methods for nonlinear derivative-free optimization. Tests are also conducted using the Gps-filter, which uses a version of the Fletcher–Leyffer filter approach. We know that Gps cannot be shown to yield kkt points with this strategy or the filter, but we use the Clarke nonsmooth calculus to prove Clarke stationarity of the sequences of feasible and infeasible trial points for LTMads-PB. Numerical experiments are conducted on three academic test problems with up to 50 variables and on a chemical engineering problem. The new LTMads-PB method generally outperforms our LTMads-EB in the case where no feasible initial points are known, and it does as well when feasible points are known. which leads us to recommend LTMads-PB. Thus the LTMads-PB is a useful practical extension of our earlier LTMads-EB algorithm, particularly in the common case for real problems where no feasible point is known. The same conclusions hold for Gps-PB versus Gps-EB.

MPCC: Critical Point Theory

H. Th. Jongen, Jan.-J. Rückmann, and V. Shikhman

SIAM J. Optim. 20, pp. 473-484 (12 pages)

Online Publication Date: April 29, 2009

Full Text: | Download PDF

Show Abstract
We study mathematical programs with complementarity constraints (MPCC) from a topological point of view. Special focus will be on C-stationary points. Under the linear independence constraint qualification (LICQ) we derive an equivariant Morse lemma at nondegenerate C-stationary points. Then, two basic theorems from Morse theory (deformation theorem and cell-attachment theorem) are proved. Outside the C-stationary point set, continuous deformation of lower level sets can be performed. As a consequence, the topological data (such as the number of connected components) then remain invariant. However, when passing a C-stationary level, the topology of the lower level set changes via the attachment of a $q$-dimensional cell. The dimension $q$ equals the stationary C-index of the (nondegenerate) C-stationary point. The stationary C-index depends on both the restricted Hessian of the Lagrangian and the Lagrange multipliers related to biactive complementarity constraints. Finally, some relations with other stationarity concepts, such as W-, A-, M-, S-, and B-stationarity, are discussed.

Algebraic Degree of Polynomial Optimization

Jiawang Nie and Kristian Ranestad

SIAM J. Optim. 20, pp. 485-502 (18 pages)

Online Publication Date: April 29, 2009

Full Text: | Download PDF

Show Abstract
Consider the polynomial optimization problem whose objective and constraints are all described by multivariate polynomials. Under some genericity assumptions, we prove that the optimality conditions always hold on optimizers, and the coordinates of optimizers are algebraic functions of the coefficients of the input polynomials. We also give a general formula for the algebraic degree of the optimal coordinates. The derivation of the algebraic degree is equivalent to counting the number of all complex critical points. As special cases, we obtain the algebraic degrees of quadratically constrained quadratic programming (QCQP), second order cone programming (SOCP), and $p$th order cone programming (POCP), in analogy to the algebraic degree of semidefinite programming [J. Nie, K. Ranestad, and B. Sturmfels, The algebraic degree of semidefinite programming, Math. Programm., to appear].

A $T$-Algebraic Approach to Primal-Dual Interior-Point Algorithms

Chek Beng Chua

SIAM J. Optim. 20, pp. 503-523 (21 pages)

Online Publication Date: May 14, 2009

Full Text: | Download PDF

Show Abstract
Three primal-dual interior-point algorithms for homogeneous cone programming are presented. They are a short-step algorithm, a large-update algorithm, and a predictor-corrector algorithm. These algorithms are described and analyzed based on a characterization of homogeneous cones via $T$-algebras. The analysis shows that the algorithms have polynomial iteration complexity.

Free Material Optimization with Fundamental Eigenfrequency Constraints

M. Stingl, M. Kočvara, and G. Leugering

SIAM J. Optim. 20, pp. 524-547 (24 pages) | Cited 1 time

Online Publication Date: May 14, 2009

Full Text: | Download PDF

Show Abstract
The goal of this paper is to formulate and solve free material optimization problems with constraints on the smallest eigenfrequency of the optimal structure. A natural formulation of this problem as a linear semidefinite program turns out to be numerically intractable. As an alternative, we propose a new approach, which is based on a nonlinear semidefinite low-rank approximation of the semidefinite dual. We introduce an algorithm based on this approach and analyze its convergence properties. The article is concluded by numerical experiments proving the effectiveness of the new approach.

A Polynomial-Time Interior-Point Method for Conic Optimization, With Inexact Barrier Evaluations

Simon P. Schurr, Dianne P. O'Leary, and André L. Tits

SIAM J. Optim. 20, pp. 548-571 (24 pages)

Online Publication Date: May 14, 2009

Full Text: | Download PDF

Show Abstract
We consider a primal-dual short-step interior-point method for conic convex optimization problems for which exact evaluation of the gradient and Hessian of the primal and dual barrier functions is either impossible or prohibitively expensive. As our main contribution, we show that if approximate gradients and Hessians of the primal barrier function can be computed, and the relative errors in such quantities are not too large, then the method has polynomial worst-case iteration complexity. (In particular, polynomial iteration complexity ensues when the gradient and Hessian are evaluated exactly.) In addition, the algorithm requires no evaluation—or even approximate evaluation—of quantities related to the barrier function for the dual cone, even for problems in which the underlying cone is not self-dual.
Close

close