Jump to Content
Your Recent History
View CartTop 20 Most Read Articles
April 2012
The 20 articles with the most full-text downloads during the month, in descending order.
 |
Smoothing Functions for Second-Order-Cone Complementarity Problems SIAM J. Optim. 12, pp. 436-460 (25 pages) Online Publication Date: July 28, 2006
Full Text:
|
Download PDF
|
||
|
Show Abstract
Smoothing functions have been much studied in the solution of optimization and complementarity problems with nonnegativity constraints. In this paper, we extend smoothing functions to problems in which the nonnegative orthant is replaced by the direct product of second-order cones. These smoothing functions include the Chen--Mangasarian class and the smoothed Fischer--Burmeister function. We study the Lipschitzian and differential properties of these functions and, in particular, we derive computable formulas for these functions and their Jacobians. These properties and formulas can then be used to develop and analyze noninterior continuation methods for solving the corresponding optimization and complementarity problems. In particular, we establish the existence and uniqueness of the Newton direction when the underlying mapping is monotone. |
|||
|
|
Some Regularity Results for the Pseudospectral Abscissa and Pseudospectral Radius of a Matrix SIAM J. Optim. 22, pp. 281-285 (5 pages) Online Publication Date: April 04, 2012
Full Text:
|
Download PDF
|
||
|
Show Abstract
The $\varepsilon$-pseudospectral abscissa $\alpha_\varepsilon$ and radius $\rho_\varepsilon$ of an $n\times n$ matrix are, respectively, the maximal real part and the maximal modulus of points in its $\varepsilon$-pseudospectrum, defined using the spectral norm. It was proved in [A.S. Lewis and C.H.J. Pang, SIAM J. Optim., 19 (2008), pp. 1048–1072] that for fixed $\varepsilon>0$, $\alpha_\varepsilon$ and $\rho_\varepsilon$ are Lipschitz continuous at a matrix $A$ except when $\alpha_\varepsilon$ and $\rho_\varepsilon$ are attained at a critical point of the norm of the resolvent (in the nonsmooth sense), and it was conjectured that the points where $\alpha_\varepsilon$ and $\rho_\varepsilon$ are attained are not resolvent-critical. We prove this conjecture, which leads to the new result that $\alpha_\varepsilon$ and $\rho_\varepsilon$ are Lipschitz continuous, and also establishes the Aubin property with respect to both $\varepsilon$ and $A$ of the $\varepsilon$-pseudospectrum for the points $z \in {\mathbb C}$ where $\alpha_\varepsilon$ and $\rho_\varepsilon$ are attained. Finally, we give a proof showing that the pseudospectrum can never be “pointed outwards.” |
|||
|
|
Efficiency of Coordinate Descent Methods on Huge-Scale Optimization Problems SIAM J. Optim. 22, pp. 341-362 (22 pages) Online Publication Date: April 24, 2012
Full Text:
|
Download PDF
|
||
|
Show Abstract
In this paper we propose new methods for solving huge-scale optimization problems. For problems of this size, even the simplest full-dimensional vector operations are very expensive. Hence, we propose to apply an optimization technique based on random partial update of decision variables. For these methods, we prove the global estimates for the rate of convergence. Surprisingly, for certain classes of objective functions, our results are better than the standard worst-case bounds for deterministic algorithms. We present constrained and unconstrained versions of the method and its accelerated variant. Our numerical test confirms a high efficiency of this technique on problems of very big size. |
|||
|
|
SIAM J. Optim. 22, pp. 286-312 (27 pages) Online Publication Date: April 04, 2012
Full Text:
|
Download PDF
|
||
|
Show Abstract
We define a risk-averse nonanticipative feasible policy for multistage stochastic programs and propose a methodology to implement it. The approach is based on dynamic programming equations written for a risk-averse formulation of the problem. This formulation relies on a new class of multiperiod risk functionals called extended polyhedral risk measures. Dual representations of such risk functionals are given and used to derive conditions of coherence. In the one-period case, conditions for convexity and consistency with second order stochastic dominance are also provided. The risk-averse dynamic programming equations are specialized considering convex combinations of one-period extended polyhedral risk measures such as spectral risk measures. To implement the proposed policy, the approximation of the risk-averse recourse functions for stochastic linear programs is discussed. In this context, we detail a stochastic dual dynamic programming algorithm which converges to the optimal value of the risk-averse problem. (A corrected PDF is attached to the original pdf.) |
|||
|
|
Alternating Direction Method with Gaussian Back Substitution for Separable Convex Programming SIAM J. Optim. 22, pp. 313-340 (28 pages) Online Publication Date: April 12, 2012
Full Text:
|
Download PDF
|
||
|
Show Abstract
We consider the linearly constrained separable convex minimization problem whose objective function is separable into $m$ individual convex functions with nonoverlapping variables. A Douglas–Rachford alternating direction method of multipliers (ADM) has been well studied in the literature for the special case of $m=2$. But the convergence of extending ADM to the general case of $m\ge 3$ is still open. In this paper, we show that the straightforward extension of ADM is valid for the general case of $m\ge 3$ if it is combined with a Gaussian back substitution procedure. The resulting ADM with Gaussian back substitution is a novel approach towards the extension of ADM from $m=2$ to $m\ge 3$, and its algorithmic framework is new in the literature. For the ADM with Gaussian back substitution, we prove its convergence via the analytic framework of contractive-type methods, and we show its numerical efficiency by some application problems. |
|||
|
|
Convergence Properties of the Nelder--Mead Simplex Method in Low Dimensions SIAM J. Optim. 9, pp. 112-147 (36 pages) Online Publication Date: July 31, 2006
Full Text:
|
Download PDF
|
||
|
Show Abstract
The Nelder--Mead simplex algorithm, first published in 1965, is an enormously popular direct search method for multidimensional unconstrained minimization. Despite its widespread use, essentially no theoretical results have been proved explicitly for the Nelder--Mead algorithm. This paper presents convergence properties of the Nelder--Mead algorithm applied to strictly convex functions in dimensions 1 and 2. We prove convergence to a minimizer for dimension 1, and various limited convergence results for dimension 2. A counterexample of McKinnon gives a family of strictly convex functions in two dimensions and a set of initial conditions for which the Nelder--Mead algorithm converges to a nonminimizer. It is not yet known whether the Nelder--Mead method can be proved to converge to a minimizer for a more specialized class of convex functions in two dimensions. |
|||
|
|
Sparse Optimization with Least-Squares Constraints SIAM J. Optim. 21, pp. 1201-1229 (29 pages) Online Publication Date: October 04, 2011
Full Text:
|
Download PDF
|
||
|
Show Abstract
The use of convex optimization for the recovery of sparse signals from incomplete or compressed data is now common practice. Motivated by the success of basis pursuit in recovering sparse vectors, new formulations have been proposed that take advantage of different types of sparsity. In this paper we propose an efficient algorithm for solving a general class of sparsifying formulations. For several common types of sparsity we provide applications, along with details on how to apply the algorithm, and experimental results. |
|||
|
|
A New Relaxation Scheme for Mathematical Programs with Equilibrium Constraints SIAM J. Optim. 20, pp. 2504-2539 (36 pages) Online Publication Date: July 06, 2010
Full Text:
|
Download PDF
|
||
|
Show Abstract
We present a new relaxation scheme for mathematical programs with equilibrium constraints (MPEC), where the complementarity constraints are replaced by a reformulation that is exact for the complementarity conditions corresponding to sufficiently nondegenerate complementarity components and relaxes only the remaining complementarity conditions. A positive parameter determines to what extent the complementarity conditions are relaxed. The relaxation scheme is such that a strongly stationary solution of the MPEC is also a solution of the relaxed problem if the relaxation parameter is chosen sufficiently small. We discuss the properties of the resulting parameterized nonlinear programs and compare stationary points and solutions. We further prove that a limit point of a sequence of stationary points of a sequence of relaxed problems is Clarke-stationary if it satisfies a so-called MPEC-constant rank constraint qualification, and it is Mordukhovich-stationary if it satisfies the MPEC-linear independence constraint qualification and the stationary points satisfy a second order sufficient condition. From this relaxation scheme, a numerical approach is derived that is applied to a comprehensive test set. The numerical results show that the approach combines good efficiency with high robustness. |
|||
|
|
A Sequential Ascending Parameter Method for Solving Constrained Minimization Problems SIAM J. Optim. 22, pp. 244-260 (17 pages) Online Publication Date: March 22, 2012
Full Text:
|
Download PDF
|
||
|
Show Abstract
In this paper, a method for solving constrained convex optimization problems is introduced. The problem is cast equivalently as a parametric unconstrained one, the (single) parameter being the optimal value of the original problem. At each stage of the algorithm the parameter is updated, and the resulting subproblem is only approximately solved. A linear rate of convergence of the parameter sequence is established. Using an optimal gradient method due to Nesterov [Dokl. Akad. Nauk SSSR, 269 (1983), pp. 543–547] to solve the arising subproblems, it is proved that the resulting gradient-based algorithm requires an overall of $O({\log(1/\varepsilon)}/ {\sqrt{\varepsilon}})$ inner iterations to obtain an $\varepsilon$-optimal and feasible solution. An image deblurring problem is solved, demonstrating the capability of the algorithm to solve large-scale problems within reasonable accuracy. |
|||
|
|
Accelerated Block-coordinate Relaxation for Regularized Optimization SIAM J. Optim. 22, pp. 159-186 (28 pages) Online Publication Date: February 28, 2012
Full Text:
|
Download PDF
|
||
|
Show Abstract
We discuss minimization of a smooth function to which is added a separable regularization function that induces structure in the solution. A block-coordinate relaxation approach with proximal linearized subproblems yields convergence to critical points, while identification of the optimal manifold (under a nondegeneracy condition) allows acceleration techniques to be applied on a reduced space. The work is motivated by experience with an algorithm for regularized logistic regression, and computational results for the algorithm on problems of this type are presented. |
|||
|
|
The Primal-Dual Active Set Strategy as a Semismooth Newton Method SIAM J. Optim. 13, pp. 865-888 (24 pages) Online Publication Date: July 28, 2006
Full Text:
|
Download PDF
|
||
|
Show Abstract
This paper addresses complementarity problems motivated by constrained optimal control problems. It is shown that the primal-dual active set strategy, which is known to be extremely efficient for this class of problems, and a specific semismooth Newton method lead to identical algorithms. The notion of slant differentiability is recalled and it is argued that the $\max$-function is slantly differentiable in Lp-spaces when appropriately combined with a two-norm concept. This leads to new local convergence results of the primal-dual active set strategy. Global unconditional convergence results are obtained by means of appropriate merit functions. |
|||
|
|
A Distance For Multistage Stochastic Optimization Models SIAM J. Optim. 22, pp. 1-23 (23 pages) Online Publication Date: January 05, 2012
Full Text:
|
Download PDF
|
||
|
Show Abstract
We describe multistage stochastic programs in a purely in-distribution setting, i.e., without any reference to a concrete probability space. The concept is based on the notion of nested distributions, which encompass in one mathematical object the scenario values as well as the information structure under which decisions have to be made. The nested distance between these distributions is introduced and turns out to be a generalization of the Wasserstein distance for stochastic two-stage problems. We give characterizations of this distance and show its usefulness in examples. The main result states that the difference of the optimal values of two multistage stochastic programs, which are Lipschitz and differ only in the nested distribution of the stochastic parameters, can be bounded by the nested distance of these distributions. This theorem generalizes the well-known Kantorovich–Rubinstein theorem, which is applicable only in two-stage situations, to multistage. Moreover, a dual characterization for the nested distance is established. The setup is applicable both for general stochastic processes and for finite scenario trees. In particular, the nested distance between general processes and scenario trees is well defined and becomes the important tool for judging the quality of the scenario tree generation. Minimizing—at least heuristically—this distance is what good scenario tree generation is all about. |
|||
|
|
Second Order Analysis for Optimal Control Problems: Improving Results Expected From Abstract Theory SIAM J. Optim. 22, pp. 261-279 (19 pages) Online Publication Date: March 22, 2012
Full Text:
|
Download PDF
|
||
|
Show Abstract
An abstract optimization problem of minimizing a functional on a convex subset of a Banach space is considered. We discuss natural assumptions on the functional that permit establishing sufficient second-order optimality conditions with minimal gap with respect to the associated necessary ones. Though the two-norm discrepancy is taken into account, the obtained results exhibit the same formulation as the classical ones known from finite-dimensional optimization. We demonstrate that these assumptions are fulfilled, in particular, by important optimal control problems for partial differential equations. We prove that, in contrast to a widespread common belief, the standard second-order conditions formulated for these control problems imply strict local optimality of the controls not only in the sense of $L^\infty$, but also of $L^2$. |
|||
|
|
Projection-like Retractions on Matrix Manifolds SIAM J. Optim. 22, pp. 135-158 (24 pages) Online Publication Date: January 26, 2012
Full Text:
|
Download PDF
|
||
|
Show Abstract
This paper deals with constructing retractions, a key step when applying optimization algorithms on matrix manifolds. For submanifolds of Euclidean spaces, we show that the operation consisting of taking a tangent step in the embedding Euclidean space followed by a projection onto the submanifold is a retraction. We also show that the operation remains a retraction if the projection is generalized to a projection-like procedure that consists of coming back to the submanifold along “admissible” directions, and we give a sufficient condition on the admissible directions for the generated retraction to be second order. This theory offers a framework in which previously proposed retractions can be analyzed, as well as a toolbox for constructing new ones. Illustrations are given for projection-like procedures on some specific manifolds for which we have an explicit, easy-to-compute expression. |
|||
|
|
Direct Multisearch for Multiobjective Optimization SIAM J. Optim. 21, pp. 1109-1140 (32 pages) Online Publication Date: September 27, 2011
Full Text:
|
Download PDF
|
||
|
Show Abstract
In practical applications of optimization it is common to have several conflicting objective functions to optimize. Frequently, these functions are subject to noise or can be of black-box type, preventing the use of derivative-based techniques. We propose a novel multiobjective derivative-free methodology, calling it direct multisearch (DMS), which does not aggregate any of the objective functions. Our framework is inspired by the search/poll paradigm of direct-search methods of directional type and uses the concept of Pareto dominance to maintain a list of nondominated points (from which the new iterates or poll centers are chosen). The aim of our method is to generate as many points in the Pareto front as possible from the polling procedure itself, while keeping the whole framework general enough to accommodate other disseminating strategies, in particular, when using the (here also) optional search step. DMS generalizes to multiobjective optimization (MOO) all direct-search methods of directional type. We prove under the common assumptions used in direct search for single objective optimization that at least one limit point of the sequence of iterates generated by DMS lies in (a stationary form of) the Pareto front. However, extensive computational experience has shown that our methodology has an impressive capability of generating the whole Pareto front, even without using a search step. Two by-products of this paper are (i) the development of a collection of test problems for MOO and (ii) the extension of performance and data profiles to MOO, allowing a comparison of several solvers on a large set of test problems, in terms of their efficiency and robustness to determine Pareto fronts. |
|||
|
|
SIAM J. Optim. 7, pp. 86-125 (40 pages) Online Publication Date: July 31, 2006
Full Text:
|
Download PDF
|
||
|
Show Abstract
The SDLCP (semidefinite linear complementarity problem) in symmetric matrices introduced in this paper provides a unified mathematical model for various problems arising from systems and control theory and combinatorial optimization. It is defined as the problem of finding a pair $(\X,\Y)$ of $n \times n$ symmetric positive semidefinite matrices which lies in a given $n(n+1)/2$ dimensional affine subspace $\FC$ of $\SC^2$ and satisfies the complementarity condition $\X \bullet \Y = 0$, where $\SC$ denotes the $n(n+1)/2$-dimensional linear space of symmetric matrices and $\X \bullet \Y$ the inner product of $\X$ and $\Y$. The problem enjoys a close analogy with the LCP in the Euclidean space. In particular, the central trajectory leading to a solution of the problem exists under the nonemptiness of the interior of the feasible region and a monotonicity assumption on the affine subspace $\FC$. The aim of this paper is to establish a theoretical basis of interior-point methods with the use of Newton directions toward the central trajectory for the monotone SDLCP. |
|||
|
|
Fixed Points of Averages of Resolvents: Geometry and Algorithms SIAM J. Optim. 22, pp. 24-40 (17 pages) Online Publication Date: January 13, 2012
Full Text:
|
Download PDF
|
||
|
Show Abstract
To provide generalized solutions if a given problem admits no actual solution is an important task in mathematics and the natural sciences. It has a rich history dating back to the early 19th century, when Carl Friedrich Gauss developed the method of least squares of a system of linear equations—its solutions can be viewed as fixed points of averaged projections onto hyperplanes. A powerful generalization of this problem is to find fixed points of averaged resolvents (i.e., firmly nonexpansive mappings). This paper concerns the relationship between the set of fixed points of averaged resolvents and certain fixed point sets of compositions of resolvents. It partially extends recent work for two mappings on a question of C. Byrne. The analysis suggests a reformulation in a product space. Algorithmic consequences are also presented. |
|||
|
|
A New Sequential Optimality Condition for Constrained Optimization and Algorithmic Consequences SIAM J. Optim. 20, pp. 3533-3554 (22 pages) Online Publication Date: December 16, 2010
Full Text:
|
Download PDF
|
||
|
Show Abstract
Necessary first-order sequential optimality conditions provide adequate theoretical tools to justify stopping criteria for nonlinear programming solvers. Sequential optimality conditions are satisfied by local minimizers of optimization problems independently of the fulfillment of constraint qualifications. A new condition of this type is introduced in the present paper. It is proved that a well-established augmented Lagrangian algorithm produces sequences whose limits satisfy the new condition. Practical consequences are discussed. |
|||
|
|
Constrained Polynomial Optimization Problems with Noncommuting Variables SIAM J. Optim. 22, pp. 363-383 (21 pages) Online Publication Date: April 24, 2012
Full Text:
|
Download PDF
|
||
|
Show Abstract
In this paper we study constrained eigenvalue optimization of noncommutative (nc) polynomials, focusing on the polydisc and the ball. Our three main results are as follows: (1) an nc polynomial is nonnegative if and only if it admits a weighted sum of hermitian squares decomposition; (2) (eigenvalue) optima for nc polynomials can be computed using a single semidefinite program (SDP)—this sharply contrasts with the commutative case where sequences of SDPs are needed; (3) the dual solution to this “single” SDP can be exploited to extract eigenvalue optimizers with an algorithm based on two ingredients: solution to a truncated nc moment problem via flat extensions, and Gelfand–Naimark–Segal construction. The implementation of these procedures in our computer algebra system NCSOStools is presented, and several examples pertaining to matrix inequalities are given to illustrate our results. |
|||
|
|
SIAM J. Optim. 22, pp. 66-86 (21 pages) Online Publication Date: January 17, 2012
Full Text:
|
Download PDF
|
||
|
Show Abstract
The (optimal) function/gradient evaluations worst-case complexity analysis available for the adaptive regularization algorithms with cubics (ARC) for nonconvex smooth unconstrained optimization is extended to finite-difference versions of this algorithm, yielding complexity bounds for first-order and derivative-free methods applied on the same problem class. A comparison with the results obtained for derivative-free methods by Vicente [Worst Case Complexity of Direct Search, Technical report, Preprint 10-17, Department of Mathematics, University of Coimbra, Coimbra, Portugal, 2010] is also discussed, giving some theoretical insight into the relative merits of various methods in this popular class of algorithms. |
|||
ALL SIAM Content
Scitation
Google Scholar