SIAM Digital Library
 
 
 

SIAM J. on Optimization

All Journal content published prior to 1997 is part of LOCUS.

Search Issue | RSS Feeds RSS
Previous Issue

2008

Volume 19, Issue 4, pp. 1511-2014


Multivariable Utility Functions

Maria B. Chiarolla and Ulrich G. Haussmann

SIAM J. Optim. 19, pp. 1511-1533 (23 pages)

Online Publication Date: December 31, 2008

Full Text: | Download PDF

Show Abstract
Utility functions of several variables are ubiquitous in economics. Their maximization requires inversion of the gradient map. Using convex analysis tools, we provide a representation of an extension of this inverse that accounts for possible constraints. To solve economic equilibrium problems, the utility functions of the agents are frequently aggregated into a representative (agent's) utility function. We establish regularity and inversion properties of such representative utility functions.

Sparse SOS Relaxations for Minimizing Functions that are Summations of Small Polynomials

Jiawang Nie and James Demmel

SIAM J. Optim. 19, pp. 1534-1558 (25 pages) | Cited 1 time

Online Publication Date: December 31, 2008

Full Text: | Download PDF

Show Abstract
This paper discusses how to find the global minimum of functions that are summations of small polynomials (“small” means involving a small number of variables). Some sparse sum of squares (SOS) techniques are proposed. We compare their computational complexity and lower bounds with prior SOS relaxations. Under certain conditions, we also discuss how to extract the global minimizers from these sparse relaxations. The proposed methods are especially useful in solving sparse polynomial system and nonlinear least squares problems. Numerical experiments are presented which show that the proposed methods significantly improve the computational performance of prior methods for solving these problems. Lastly, we present applications of this sparsity technique in solving polynomial systems derived from nonlinear differential equations and sensor network localization.

On Semidefinite Programming Relaxations of the Traveling Salesman Problem

Etienne de Klerk, Dmitrii V. Pasechnik, and Renata Sotirov

SIAM J. Optim. 19, pp. 1559-1573 (15 pages) | Cited 1 time

Online Publication Date: December 31, 2008

Full Text: | Download PDF

Show Abstract
We consider a new semidefinite programming (SDP) relaxation of the symmetric traveling salesman problem (TSP) that may be obtained via an SDP relaxation of the more general quadratic assignment problem (QAP). We show that the new relaxation dominates the one in [D. Cvetković, M. Cangalović, and V. Kovačević-Vujčić, Semidefinite programming methods for the symmetric traveling salesman problem, in Proceedings of the 7th International IPCO Conference on Integer Programming and Combinatorial Optimization, Springer-Verlag, London, UK, 1999, pp. 126–136]. Unlike the bound of Cvetković et al., the new SDP bound is not dominated by the Held–Karp linear programming bound, or vice versa. An erratum to this article has been appended at the end of the pdf file.

Robust Stochastic Approximation Approach to Stochastic Programming

A. Nemirovski, A. Juditsky, G. Lan, and A. Shapiro

SIAM J. Optim. 19, pp. 1574-1609 (36 pages) | Cited 5 times

Online Publication Date: January 21, 2009

Full Text: | Download PDF

Show Abstract
In this paper we consider optimization problems where the objective function is given in a form of the expectation. A basic difficulty of solving such stochastic optimization problems is that the involved multidimensional integrals (expectations) cannot be computed with high accuracy. The aim of this paper is to compare two computational approaches based on Monte Carlo sampling techniques, namely, the stochastic approximation (SA) and the sample average approximation (SAA) methods. Both approaches, the SA and SAA methods, have a long history. Current opinion is that the SAA method can efficiently use a specific (say, linear) structure of the considered problem, while the SA approach is a crude subgradient method, which often performs poorly in practice. We intend to demonstrate that a properly modified SA approach can be competitive and even significantly outperform the SAA method for a certain class of convex stochastic problems. We extend the analysis to the case of convex-concave stochastic saddle point problems and present (in our opinion highly encouraging) results of numerical experiments.

Shape Optimization Under Uncertainty—A Stochastic Programming Perspective

Sergio Conti, Harald Held, Martin Pach, Martin Rumpf, and Rüdiger Schultz

SIAM J. Optim. 19, pp. 1610-1632 (23 pages) | Cited 1 time

Online Publication Date: January 21, 2009

Full Text: | Download PDF

Show Abstract
We present an algorithm for shape optimization under stochastic loading and representative numerical results. Our strategy builds upon a combination of techniques from two-stage stochastic programming and level-set-based shape optimization. In particular, usage of linear elasticity and quadratic objective functions permits us to obtain a computational cost which scales linearly in the number of linearly independent applied forces, which often is much smaller than the number of different realizations of the stochastic forces. Numerical computations are performed using a level set method with composite finite elements both in two and in three spatial dimensions.

Error Bounds for Convex Polynomials

W. H. Yang

SIAM J. Optim. 19, pp. 1633-1647 (15 pages) | Cited 1 time

Online Publication Date: January 21, 2009

Full Text: | Download PDF

Show Abstract
The purpose of this paper is to investigate error bounds for convex polynomials. We prove that for a convex polynomial $f$ in $n$ variables which is not everywhere positive and which is not constant on any affine subspace, either $f$ is a sum of a convex polynomial in fewer variables and a linear form with negative coefficients or the negativity set of $f$ is compact. As an application, we deduce various types of error bounds for unconstrained and polyhedral-constrained convex polynomials.

Calmness for L-Subsmooth Multifunctions in Banach Spaces

Xi Yin Zheng and Kung Fu Ng

SIAM J. Optim. 19, pp. 1648-1673 (26 pages) | Cited 2 times

Online Publication Date: January 21, 2009

Full Text: | Download PDF

Show Abstract
Using variational analysis techniques, we study subsmooth multifunctions in Banach spaces. In terms of the normal cones and coderivatives, we provide some characterizations for such multifunctions to be calm. Sharper results are obtained for Asplund spaces. We also present some exact formulas of the modulus of the calmness. As applications, we provide some error bound results on nonconvex inequalities, which improve and generalize the existing error bound results.

Adaptive Barrier Update Strategies for Nonlinear Interior Methods

Jorge Nocedal, Andreas Wächter, and Richard A. Waltz

SIAM J. Optim. 19, pp. 1674-1693 (20 pages) | Cited 2 times

Online Publication Date: January 28, 2009

Full Text: | Download PDF

Show Abstract
This paper considers strategies for selecting the barrier parameter at every iteration of an interior-point method for nonlinear programming. Numerical experiments suggest that heuristic adaptive choices, such as Mehrotra's probing procedure, outperform monotone strategies that hold the barrier parameter fixed until a barrier optimality test is satisfied. A new adaptive strategy is proposed based on the minimization of a quality function. The paper also proposes a globalization framework that ensures the convergence of adaptive interior methods, and examines convergence failures of the Mehrotra predictor-corrector algorithm. The barrier update strategies proposed in this paper are applicable to a wide class of interior methods and are tested in the two distinct algorithmic frameworks provided by the ipopt and knitro software packages.

An Adaptive Scalarization Method in Multiobjective Optimization

Gabriele Eichfelder

SIAM J. Optim. 19, pp. 1694-1718 (25 pages)

Online Publication Date: January 28, 2009

Full Text: | Download PDF

Show Abstract
This paper presents a new method for the numerical solution of nonlinear multiobjective optimization problems with an arbitrary partial ordering in the objective space induced by a closed pointed convex cone. This algorithm is based on the well-known scalarization approach by Pascoletti and Serafini and adaptively controls the scalarization parameters using new sensitivity results. The computed image points give a nearly equidistant approximation of the whole Pareto surface. The effectiveness of this new method is demonstrated with various test problems and an applied problem from medicine.

Approximations of Stochastic Optimization Problems Subject to Measurability Constraints

Pierre Carpentier, Jean-Philippe Chancelier, and Michel De Lara

SIAM J. Optim. 19, pp. 1719-1734 (16 pages)

Online Publication Date: January 28, 2009

Full Text: | Download PDF

Show Abstract
Motivated by the numerical resolution of stochastic optimization problems subject to measurability constraints, we focus upon the issue of discretization. There exist indeed two components to be discretized for such problems, namely, the random variable modelling uncertainties (noise) and the $\sigma$-field modelling the knowledge (information) according to which decisions are taken. There is no reason to bind these two discretizations, which are a priori unrelated. In this setting, we present conditions under which the discretized problems converge to the original one. The focus is put on the convergence notions ensuring the quality of the approximation; we illustrate their importance by means of a counterexample based on the Monte Carlo approximation.

Strong Duality for the CDT Subproblem: A Necessary and Sufficient Condition

Wenbao Ai and Shuzhong Zhang

SIAM J. Optim. 19, pp. 1735-1756 (22 pages) | Cited 1 time

Online Publication Date: February 11, 2009

Full Text: | Download PDF

Show Abstract
In this paper, we consider the problem of minimizing a nonconvex quadratic function, subject to two quadratic inequality constraints. As an application, such a quadratic program plays an important role in the trust region method for nonlinear optimization; such a problem is known as the Celis, Dennis, and Tapia (CDT) subproblem in the literature. The Lagrangian dual of the CDT subproblem is a semidefinite program (SDP), hence convex and solvable. However, a positive duality gap may exist between the CDT subproblem and its Lagrangian dual because the CDT subproblem itself is nonconvex. In this paper, we present a necessary and sufficient condition to characterize when the CDT subproblem and its Lagrangian dual admits no duality gap (i.e., the strong duality holds). This necessary and sufficient condition is easy verifiable and involves only one (any) optimal solution of the SDP relaxation for the CDT subproblem. Moreover, the condition reveals that it is actually rare to render a positive duality gap for the CDT subproblems in general. Moreover, if the strong duality holds, then an optimal solution for the CDT problem can be retrieved from an optimal solution of the SDP relaxation, by means of a matrix rank-one decomposition procedure. The same analysis is extended to the framework where the necessary and sufficient condition is presented in terms of the Lagrangian multipliers at a KKT point. Furthermore, we show that the condition is numerically easy to work with approximatively.

Approximate Primal Solutions and Rate Analysis for Dual Subgradient Methods

Angelia Nedić and Asuman Ozdaglar

SIAM J. Optim. 19, pp. 1757-1780 (24 pages) | Cited 1 time

Online Publication Date: February 11, 2009

Full Text: | Download PDF

Show Abstract
In this paper, we study methods for generating approximate primal solutions as a byproduct of subgradient methods applied to the Lagrangian dual of a primal convex (possibly nondifferentiable) constrained optimization problem. Our work is motivated by constrained primal problems with a favorable dual problem structure that leads to efficient implementation of dual subgradient methods, such as the recent resource allocation problems in large-scale networks. For such problems, we propose and analyze dual subgradient methods that use averaging schemes to generate approximate primal optimal solutions. These algorithms use a constant stepsize in view of its simplicity and practical significance. We provide estimates on the primal infeasibility and primal suboptimality of the generated approximate primal solutions. These estimates are given per iteration, thus providing a basis for analyzing the trade-offs between the desired level of error and the selection of the stepsize value. Our analysis relies on the Slater condition and the inherited boundedness properties of the dual problem under this condition. It also relies on the boundedness of subgradients, which is ensured by assuming the compactness of the constraint set.

On Multivariate Discrete Moment Problems: Generalization of the Bivariate Min Algorithm for Higher Dimensions

Gergely Mádi-Nagy

SIAM J. Optim. 19, pp. 1781-1806 (26 pages)

Online Publication Date: February 11, 2009

Full Text: | Download PDF

Show Abstract
The objective of the multivariate discrete moment problem (MDMP) is to find the minimum and/or maximum of the expected value of a function of a random vector with a discrete finite support where the probability distribution is unknown, but some of the moments are given. The moments may be binomial, power, or of a more general type. The MDMP can be formulated as a linear programming problem with a very ill-conditioned coefficient matrix. Hence, the LP problem can be solved with difficulty or cannot be solved at all. The central results of the field of the MDMP concern the structure of the dual feasible bases. These bases, on one hand, provide us with bounds without any numerical difficulties. On the other hand, they can be used as an initial basis of the dual simplex method. That results in shorter running time and better numerical stability because the first phase can be skipped. This paper introduces a new type of MDMP, where the bivariate moments up to a certain order $m$ consisting of the first variable and further univariate moments up to the order $m_j$, $j=1,\ldots,s$, are given. Then we generalize the bivariate Min Algorithm of Mádi-Nagy and Prékopa [Math. Oper. Res., 29 (2004), pp. 229–258] for higher dimensions, which gives numerous dual feasible bases of the MDMP. By the aid of this, on one hand, we can give useful bounds for MDMPs with higher dimensional random vectors even if the usual solvers cannot give acceptable results. On the other hand, applying our algorithm for the binomial MDMP, we can give better bounds for probabilities of Boolean functions of event sequences than the recent bounds in the literature. These results are illustrated by numerical examples.

Smooth Optimization Approach for Sparse Covariance Selection

Zhaosong Lu

SIAM J. Optim. 19, pp. 1807-1827 (21 pages) | Cited 2 times

Online Publication Date: February 20, 2009

Full Text: | Download PDF

Show Abstract
In this paper we first study a smooth optimization approach for solving a class of nonsmooth strictly concave maximization problems whose objective functions admit smooth convex minimization reformulations. In particular, we apply Nesterov's smooth optimization technique [Y. E. Nesterov, Dokl. Akad. Nauk SSSR, 269 (1983), pp. 543–547; Y. E. Nesterov, Math. Programming, 103 (2005), pp. 127–152] to their dual counterparts that are smooth convex problems. It is shown that the resulting approach has ${\cal O}(1/{\sqrt{\epsilon}})$ iteration complexity for finding an $\epsilon$-optimal solution to both primal and dual problems. We then discuss the application of this approach to sparse covariance selection that is approximately solved as an $l_1$-norm penalized maximum likelihood estimation problem, and also propose a variant of this approach which has substantially outperformed the latter one in our computational experiments. We finally compare the performance of these approaches with other first-order methods, namely, Nesterov's ${\cal O}(1/\epsilon)$ smooth approximation scheme and block-coordinate descent method studied in [A. d'Aspremont, O. Banerjee, and L. El Ghaoui, SIAM J. Matrix Anal. Appl., 30 (2008), pp. 56–66; J. Friedman, T. Hastie, and R. Tibshirani, Biostatistics, 9 (2008), pp. 432–441] for sparse covariance selection on a set of randomly generated instances. It shows that our smooth optimization approach substantially outperforms the first method above, and moreover, its variant substantially outperforms both methods above.

Convergence Analysis of Generalized Iteratively Reweighted Least Squares Algorithms on Convex Function Spaces

Nicolai Bissantz, Lutz Dümbgen, Axel Munk, and Bernd Stratmann

SIAM J. Optim. 19, pp. 1828-1845 (18 pages)

Online Publication Date: February 20, 2009

Full Text: | Download PDF

Show Abstract
The computation of robust regression estimates often relies on minimization of a convex functional on a convex set. In this paper we discuss a general technique for a large class of convex functionals to compute the minimizers iteratively, which is closely related to majorization-minimization algorithms. Our approach is based on a quadratic approximation of the functional to be minimized and includes the iteratively reweighted least squares algorithm as a special case. We prove convergence on convex function spaces for general coercive and convex functionals $F$ and derive geometric convergence in certain unconstrained settings. The algorithm is applied to total variation (TV) penalized quantile regression and is compared with a step size corrected Newton–Raphson algorithm. It is found that typically in the first steps the iteratively reweighted least squares algorithm performs significantly better, whereas the Newton type method outpaces the former only after many iterations. Finally, in the setting of bivariate regression with unimodality constraints we illustrate how this algorithm allows one to utilize highly efficient algorithms for special quadratic programs in more complex settings.

On the Implementation of Interior Point Decomposition Algorithms for Two-Stage Stochastic Conic Programs

Sanjay Mehrotra and M. Gökhan Özevin

SIAM J. Optim. 19, pp. 1846-1880 (35 pages) | Cited 2 times

Online Publication Date: February 27, 2009

Full Text: | Download PDF

Show Abstract
In this paper we develop a practical primal interior decomposition algorithm for two-stage stochastic programming problems. The framework of this algorithm is similar to the framework in [S. Mehrotra and M. G. Özevin, “Decomposition based interior point methods for two-stage stochastic convex quadratic programs with recourse,” to appear in Oper. Res.; S. Mehrotra and M. G. Özevin, SIAM J. Optim., 18 (2007), pp. 206–222] and [G. Zhao, Math. Program., 90 (2001), pp. 507–536], however, their algorithm is altered in a simple yet fundamental way to achieve practical performance. In particular, this new algorithm weighs the log-barrier terms in the second stage problems differently from the theoretical algorithms analyzed in [S. Mehrotra and M. G. Özevin, Oper. Res., to appear], [S. Mehrotra and M. G. Özevin, SIAM J. Optim., 18 (2007), pp. 206–222], and [G. Zhao, Math. Program., 90 (2001), pp. 507–536]. We give a method for generating a suitable starting point; a method for selecting a good starting barrier parameter; a heuristic for first stage step-length calculation without performing line searches; and a method for adaptive addition of new scenarios over the course of the algorithm. The decomposition algorithm is implemented to solve two-stage stochastic conic programs with recourse whose underlying cones are Cartesian products of linear, second order, and semidefinite cones. The performance of primal decomposition method is studied on a set of randomly generated test problems as well as a two-stage stochastic programming extension of the Markowitz portfolio selection model. The computational results show that an efficient and stable implementation of the primal decomposition method is possible. These results also show that in problems with a large number of scenarios, the adaptive addition of scenarios can yield computational savings of up to 80%.

A Use of Conjugate Gradient Direction for the Convex Optimization Problem over the Fixed Point Set of a Nonexpansive Mapping

Hideaki Iiduka and Isao Yamada

SIAM J. Optim. 19, pp. 1881-1893 (13 pages)

Online Publication Date: February 27, 2009

Full Text: | Download PDF

Show Abstract
In this paper, we discuss the convex optimization problem over the fixed point set of a nonexpansive mapping. The main objective of the paper is to accelerate the hybrid steepest descent method for the problem. To this goal, we present a new iterative scheme that utilizes the conjugate gradient direction. Its convergence to the solution is guaranteed under certain assumptions. In order to demonstrate the effectiveness, performance, and convergence of our proposed algorithm, we present numerical comparisons of the algorithm with the existing algorithm.

An Invitation to Tame Optimization

A. D. Ioffe

SIAM J. Optim. 19, pp. 1894-1917 (24 pages) | Cited 1 time

Online Publication Date: February 27, 2009

Full Text: | Download PDF

Show Abstract
The word “tame” is used in the title in the same context as in expressions like “convex optimization,” “nonsmooth optimization,” etc.—as a reference to the class of objects involved in the formulation of optimization problems. Definable and tame functions and mappings associated with various o-minimal structures (e.g. semilinear, semialgebraic, globally subanalytic, and others) have a number of remarkable properties which make them an attractive domain for various applications. This relates both to the power of results that can be obtained and the power of available analytic techniques. The paper surveys certain ideas and recent results, some new, which have been or (hopefully) can be productively used in studies relating to variational analysis and nonsmooth optimization.

A Polynomial Predictor-Corrector Trust-Region Algorithm for Linear Programming

Guanghui Lan, Renato D. C. Monteiro, and Takashi Tsuchiya

SIAM J. Optim. 19, pp. 1918-1946 (29 pages)

Online Publication Date: February 27, 2009

Full Text: | Download PDF

Show Abstract
In this paper we present a scaling-invariant, interior-point, predictor-corrector type algorithm for linear programming (LP) whose iteration-complexity is polynomially bounded by the dimension and the logarithm of a certain condition number of the LP constraint matrix. At the predictor stage, the algorithm either takes the step along the standard affine scaling (AS) direction or a new trust-region type direction, whose construction depends on a scaling-invariant bipartition of the variables determined by the AS direction. This contrasts with the layered least squares direction introduced in S. Vavasis and Y. Ye [Math. Program., 74 (1996), pp. 79–120], whose construction depends on multiple-layered partitions of the variables that are not scaling-invariant. Moreover, it is shown that the overall arithmetic complexity of the algorithm (weakly) depends on the right-hand side and the cost of the LP in view of the work involved in the computation of the trust region steps.

A New Class of Minimum Norm Duality Theorems

Achiya Dax

SIAM J. Optim. 19, pp. 1947-1969 (23 pages)

Online Publication Date: February 27, 2009

Full Text: | Download PDF

Show Abstract
In this paper we derive new duality results on the width and the length of symmetrical convex bodies. Let $K$ be a symmetrical convex body in $R^m$ and let $\|\cdot\|$ be some (arbitrary) norm on $R^m$. The width of $K$ is obtained by searching for the smallest “sandwich” (or “slab”) that contains $K$. It is shown that the dual problem has the following form: Find the largest diameter of a norm ball that is contained in $K$. Indeed, the diameter of a maximal norm ball equals the width of the smallest sandwich. Moreover, the solutions of the two problems obey certain alignment relations. The length of $K$ is found by searching the largest “sandwich” that contains $K$. The last problem is closely related to the “maximal chord problem” whose optimal value is called the “diameter” of $K$. In this case the dual problem is to find the smallest norm ball that contains $K$. It is proved that the diameter of the smallest norm ball equals the diameter (the length) of $K$, and that primal and dual solutions satisfy certain alignment relations. Part of the results remain valid for more general convex sets.

Pareto Subdifferential Calculus for Convex Vector Mappings and Applications to Vector Optimization

Mounir El Maghri and Mohamed Laghdir

SIAM J. Optim. 19, pp. 1970-1994 (25 pages)

Online Publication Date: March 13, 2009

Full Text: | Download PDF

Show Abstract
This paper deals with the subdifferential of convex analysis defined in the Pareto sense, from a point of view of nonvacuity, characterizations, and calculus rules and their applications to the vector optimization, the convex maps being vector-valued in a finite- or infinite-dimensional ordered vector space. Subdifferentiability is characterized under conditions of Attouch–Brézis type. Formulations by derivatives, when they exist, are provided. Concerning the calculus rules, the first main result gives the gap between the Pareto subdifferential and the ordinary one, allowing thus the computation of the one from the other. Next, as central results, Pareto subdifferentials of the sum and/or composition of two convex vector mappings are developed. The formulas are obtained under Moreau–Rockafellar or Attouch–Brézis-type conditions, revealing, strangely, the presence of the ordinary subdifferential. These formulas actually allow the extension of the indicator function technique to the vector case, so that Pareto optimality (efficiency) conditions are easily derived and weakened with qualification conditions of the Attouch–Brézis kind. Finally, the gap between efficient and optimal sets is also deduced.

Convexity in SemiAlgebraic Geometry and Polynomial Optimization

Jean B. Lasserre

SIAM J. Optim. 19, pp. 1995-2014 (20 pages) | Cited 2 times

Online Publication Date: March 13, 2009

Full Text: | Download PDF

Show Abstract
We review several (and provide new) results on the theory of moments, sums of squares, and basic semialgebraic sets when convexity is present. In particular, we show that, under convexity, the hierarchy of semidefinite relaxations for polynomial optimization simplifies and has finite convergence, a highly desirable feature as convex problems are in principle easier to solve. In addition, if a basic semialgebraic set $\mathbf{K}$ is convex but its defining polynomials are not, we provide two algebraic certificates of convexity which can be checked numerically. The second is simpler and holds if a sufficient (and almost necessary) condition is satisfied; it also provides a new condition for $\mathbf{K}$ to have semidefinite representation. For this we use (and extend) some of the recent results from the author and Helton and Nie [Math. Program., to appear]. Finally, we show that, when restricting to a certain class of convex polynomials, the celebrated Jensen's inequality in convex analysis can be extended to linear functionals that are not necessarily probability measures.
Close

Your Recent History Recent History Help

Recently Viewed

  1. On the Solution of the Tikhonov Regularization of the Total Least Squares Problem
  2. On the Stability of Convex-valued Mappings and Their Relative Boundary and Extreme Points Set Mappin...
  3. Constraint Reduction for Linear Programs with Many Inequality Constraints
  4. A Note on Trust-Region Radius Update
  5. Convergence of the Iterates of Descent Methods for Analytic Cost Functions
© SIAM By using SIAM Publications Online you agree to abide by the Terms and Conditions of Use. Banner art adapted from a figure by Hinke M. Osinga and Bernd Krauskopf (University of Bristol, UK).
close