SIAM Digital Library
 
 
 

SIAM J. on Optimization

All Journal content published prior to 1997 is part of LOCUS.

Search Issue | RSS Feeds RSS
Previous Issue

1992

Volume 2, Issue 4, pp. 533-672


On the Behavior of Broyden’s Class of Quasi-Newton Methods

Richard H. Byrd, Dong C. Liu, and Jorge Nocedal

SIAM J. Optim. 2, pp. 533-557 (25 pages) | Cited 19 times

Online Publication Date: July 13, 2006

Full Text: | Download PDF

Show Abstract
This paper analyzes algorithms from the Broyden class of quasi-Newton methods for nonlinear unconstrained optimization. This class depends on a parameter $\phi_k $, for which the choices $\phi_k = 0$ and $\phi_k = 1$ give the well-known BFGS and DFP methods. This paper examines algorithms that allow for negative values of the parameter $\phi_k $. It shows that severe restrictions have to be imposed on the selection of $\phi_k $ to guarantee q-superlinear convergence. It is argued that negative values of $\phi_k $ are desirable, and conditions on $\phi_k $ that guarantee superlinear convergence are given. However, practical algorithms that preserve the excellent properties of the BFGS method are not easy to design.

New Results on a Continuously Differentiable Exact Penalty Function

Stefano Lucidi

SIAM J. Optim. 2, pp. 558-574 (17 pages) | Cited 15 times

Online Publication Date: July 13, 2006

Full Text: | Download PDF

Show Abstract
The main motivation of this paper is to weaken the conditions that imply the correspondence between the solution of a constrained problem and the unconstrained minimization of a continuously differentiable function.
In particular, a new continuously differentiable exact penalty function is proposed for the solution of nonlinear programming problems. Under mild assumptions, a complete equivalence can be established between the solution of the original constrained problem and the unconstrained minimization of this penalty function on a perturbation of the feasible set.
This new penalty function and its exactness properties allow us to define globally and superlinearly convergent algorithms to solve nonlinear programming problems. As an example, a Newton-type algorithm is described which converges locally in one iteration in case of quadratic programming problems.

On the Implementation of a Primal-Dual Interior Point Method

Sanjay Mehrotra

SIAM J. Optim. 2, pp. 575-601 (27 pages) | Cited 221 times

Online Publication Date: July 13, 2006

Full Text: | Download PDF

Show Abstract
This paper gives an approach to implementing a second-order primal-dual interior point method. It uses a Taylor polynomial of second order to approximate a primal-dual trajectory. The computations for the second derivative are combined with the computations for the centering direction. Computations in this approach do not require that primal and dual solutions be feasible. Expressions are given to compute all the higher-order derivatives of the trajectory of interest. The implementation ensures that a suitable potential function is reduced by a constant amount at each iteration.
There are several salient features of this approach. An adaptive heuristic for estimating the centering parameter is given. The approach used to compute the step length is also adaptive. A new practical approach to compute the starting point is given. This approach treats primal and dual problems symmetrically.
Computational results on a subset of problems available from netlib are given. On mutually tested problems the results show that the proposed method requires approximately 40 percent fewer iterations than the implementation proposed in Lustig, Marsten, and Shanno [Tech. Rep. TR J-89-11, Georgia Inst. of Technology, Atlanta, 1989]. It requires approximately 50 percent fewer iterations than the dual affine scaling method in Adler, Karmarkar, Resende, and Veiga [Math. Programming, 44 (1989), pp. 297–336], and 35 percent fewer iterations than the second-order dual affine scaling method in the same paper. The new approach for estimating the centering parameter and finding the step length and the starting point have contributed to the reduction in the number of iterations. However, the contribution due to the use of second derivative is most significant.
On the tested problems, on the average the implementation shown was found to be approximately two times faster than OBl (version 02/90) described in Lustig, Marsten, and Shanno and 2.5 times faster than MINOS 5.3 described in Murtagh and Saunders [Tech. Rep. SOL 83-20, Dept. of Operations Research, Stanford Univ., Stanford, CA, 1983].

On Regularized Least Norm Problems

Achiya Dax

SIAM J. Optim. 2, pp. 602-618 (17 pages) | Cited 10 times

Online Publication Date: July 13, 2006

Full Text: | Download PDF

Show Abstract
This paper investigates the regularized least norm problem \[ \text{minimize }F(\mathbf{x}) = (\varepsilon /s) \| \mathbf{x} \|_s^s + \| A\mathbf{x} - \mathbf{b} \|_p , \] where $\varepsilon $ is a positive constant, $1 < s < \infty $, and $1 < p < \infty $. Let $\mathbf{x}_\varepsilon $ denote the solution that corresponds to a given value of $\varepsilon $, and let $\mathbf{x}^* $ denote the minimum $l_s $ norm solution of the unregularized least $l_p $ norm problem. It is shown that $\mathbf{x}_\varepsilon $ is a continuous function of $\varepsilon $, $\|\mathbf{x}_\varepsilon \|_s \leqq \| \mathbf{x}^*\|_s$, $\lim_{\varepsilon \to \infty } \mathbf{x}_\varepsilon = \mathbf{0}$, and $\lim_{\varepsilon \to 0} \mathbf{x}_\varepsilon = \mathbf{x}^* $. Furthermore, if the system $A\mathbf{x} = \mathbf{b}$ is solvable then there exists a positive constant $\delta $ such that $\mathbf{x}_\varepsilon = \mathbf{x}^* $ for all $\varepsilon \in ( 0,\delta ]$. The question of whether $\mathbf{x}_\varepsilon = \mathbf{x}^* $ is related to a new theorem of the alternative. The main result is the observation that the dual of the regularized least norm problem has the form \[ \text{maximize}\quad D(\mathbf{y}) = \mathbf{b}^T \mathbf{y} - (\varepsilon /t) \| {A^T \mathbf{y}/\varepsilon } \|_t^t \]\[ \text{subject to}\quad \| \mathbf{y} \|_q \leqq 1 ,\] where $t = s/(s - 1)$ and $q = p/(p - 1)$. Moreover, the primal solution $\hat{\mathbf{x}}$ is easily recovered from a dual solution $\hat{\mathbf{y}}$, and vice versa. This pair of points satisfies $D(\hat{\mathbf{y}}) = F(\hat{\mathbf{x}})$ and the classical primal-dual inequality $D(\mathbf{y})\leqq F(\mathbf{x})$ holds for all $\mathbf{x }\in \mathbb{R}^n $ and $\mathbf{y} \in \mathbb{R}^m $ such that $\| \mathbf{y} \|_q \leqq 1$. The paper presents an iterative improvement process which, under certain conditions, converges toward a solution of the unregularized least norm problem. The inequality $\| \mathbf{y} \|_q \leqq 1$ introduces an obstacle into the solution of the dual problem, but this obstacle may be removed by applying penalty function methods.

On the Continuity of the Solution Map in Linear Complementarity Problems

M. Seetharama Gowda

SIAM J. Optim. 2, pp. 619-634 (16 pages) | Cited 13 times

Online Publication Date: July 13, 2006

Full Text: | Download PDF

Show Abstract
The continuity properties of the solution map $\mathcal{S}(M,q) \mapsto \mathcal{S}(M,q)$ are investigated, where $\mathcal{S}(M,q)$ denotes the solution set corresponding to the linear complementarily problem LCP $(M,q)$. A Robinson-type upper semicontinuity result is established for $\mathcal{S}$, and a generalization of the Mangasarian-Shiau result concerning the Lipschitzian property of $\mathcal{S}$ in the $q$-variable is proved. It is also shown that when the matrix is positive semidefinite (or more generally a $\mathbf{G}$-matrix), the solution map is Lipschitz continuous with respect to the $q$-vector if and only if the matrix is a $\mathbf{P}$-matrix.

Linear Inequality Scaling Problems

Uriel G. Rothblum

SIAM J. Optim. 2, pp. 635-648 (14 pages)

Online Publication Date: July 13, 2006

Full Text: | Download PDF

Show Abstract
Let $a \in R^m $, $b \in R^m $, and $C \in R^{m \times n} $ be given, where $a$ is strictly positive. A $C$-scaling of the vector $a$ is defined to be a vector $a' \in R^n $ with $a'_j = a_j [ \Pi_{k = 1}^m (u_k )^{C_{kj} } ]$ for some strictly positive vector $u \in R^m $. The problem of finding a $C$-scaling of the vector $a$ that satisfies the linear system $Cx = b$ is called the linear equality scaling problem (LESP). The current paper considers the linear inequality scaling problem (LISP), which concerns the identification of a $C$-scaling of $a$ which satisfies the linear inequality system $Cx\leqq b$, where it is required that $u\leqq 1$ and that $u_i = 1$ for each $i$ with $(Cx)_i < b_i $. It is shown that LISP generalizes LESP and that it unifies a number of matrix-scaling problems that have been studied recently. Further, it is shown that LISP can be reduced to one of two convex optimization problems and these reductions are used to characterize solutions to LISP and to derive necessary and sufficient conditions for their existence. In addition, uniqueness of solutions is established and perturbed relaxations of LISP are considered.

New Proximal Point Algorithms for Convex Minimization

Osman Güler

SIAM J. Optim. 2, pp. 649-664 (16 pages) | Cited 9 times

Online Publication Date: July 13, 2006

Full Text: | Download PDF

Show Abstract
This paper introduces two new proximal point algorithms for minimizing a proper, lower-semicontinuous convex function $f: \mathbf{R}^n \to R \cup \{ \infty \}$. Under this minimal assumption on $f$, the first algorithm possesses the global convergence rate estimate $f(x_k ) - \min_{x \in \mathbf{R}^n } f(x) = O(1/(\sum_{j = 0}^{k - 1} {\sqrt \lambda_j } )^2 )$, where $\{ \lambda_k \}_{k = 0}^\infty $ are the proximal parameters. It is shown that this algorithm converges, and global convergence rate estimates for it are provided, even if minimizations are performed inexactly at each iteration. Both algorithms converge even if $f$ has no minimizers or is unbounded from below. These algorithms and results are valid in infinite-dimensional Hilbert spaces.

A Necessary and Sufficient Condition for a Constrained Minimum

J. Warga

SIAM J. Optim. 2, pp. 665-667 (3 pages) | Cited 4 times

Online Publication Date: July 13, 2006

Full Text: | Download PDF

Show Abstract
Let $U$ be an open subset of $\mathbb{R}^n $, $X$ a compact semi-analytic subset of $U$, $(f_0 ,f):U \to \mathbb{R} \times \mathbb{R}^n $ analytic, and $0 \in f(X)$. It is proven that a point $x_0 \in X$ minimizes $f_0 (x)$ subject to $f(x) = 0$ if and only if $x_0 \in X$ minimizes $f_0 (x) + c | {f(x)} |^{1/N} $ for all sufficiently large $c$ and $N$. This reduces the constrained minimization problem to a finite number of unconstrained problems.

Diagonal Matrix Scaling and Linear Programming

Leonid Khachiyan and Bahman Kalantari

SIAM J. Optim. 2, pp. 668-672 (5 pages) | Cited 2 times

Online Publication Date: July 13, 2006

Full Text: | Download PDF

Show Abstract
A positive semidefinite symmetric matrix either has a nontrivial nonnegative zero or can be scaled by a positive diagonal matrix into a doubly quasi-stochastic matrix. This paper describes a simple path-following Newton algorithm of the complexity $O(\sqrt{n} L)$ iterations to either scale an $n \times n$ matrix or give a nontrivial nonnegative zero. The latter problem is well known to be equivalent to linear programming.
Close

Your Recent History Recent History Help

Recently Viewed

  1. On the Solution of the Tikhonov Regularization of the Total Least Squares Problem
  2. On the Stability of Convex-valued Mappings and Their Relative Boundary and Extreme Points Set Mappin...
  3. Constraint Reduction for Linear Programs with Many Inequality Constraints
  4. A Note on Trust-Region Radius Update
  5. Convergence of the Iterates of Descent Methods for Analytic Cost Functions
© SIAM By using SIAM Publications Online you agree to abide by the Terms and Conditions of Use. Banner art adapted from a figure by Hinke M. Osinga and Bernd Krauskopf (University of Bristol, UK).
close