SIAM Digital Library
 
 
 

SIAM J. on Optimization

All Journal content published prior to 1997 is part of LOCUS.

Search Issue | RSS Feeds RSS
Previous Issue

1991

Volume 1, Issue 4, pp. 425-669


An Auction Algorithm for Shortest Paths

Dimitri P. Bertsekas

SIAM J. Optim. 1, pp. 425-447 (23 pages) | Cited 15 times

Online Publication Date: July 31, 2006

Full Text: | Download PDF

Show Abstract
A new and simple algorithm for finding shortest paths in a directed graph is proposed. In the single origin-single destination case, the algorithm maintains a single path starting at the origin, which is extended or contracted by a single node at each iteration. Simultaneously, at most one dual variable is adjusted at each iteration so as to either improve or maintain the value of a dual function. For the case of multiple origins, the algorithm is well suited for parallel computation. It maintains multiple paths that can be extended or contracted in parallel by several processors that share the results of their computations. Based on experiments with randomly generated problems on a serial machine, the algorithm substantially outperforms its closest competitors for problems with few origins and a single destination. It also seems better suited for parallel computation than other shortest path algorithms.

Direct Search Methods on Parallel Machines

J. E. Dennis, Jr. and Virginia Torczon

SIAM J. Optim. 1, pp. 448-474 (27 pages) | Cited 44 times

Online Publication Date: July 31, 2006

Full Text: | Download PDF

Show Abstract
This paper describes an approach to constructing derivative-free algorithms for unconstrained optimization that are easy to implement on parallel machines. A special feature of this approach is the ease with which algorithms can be generated to take advantage of any number of processors and to adapt to any cost ratio of communication to function evaluation.
Numerical tests show speed-ups on two fronts. The cost of synchronization being minimal, the speed-up is almost linear with the addition of more processors, i.e., given a problem and a search strategy, the decrease in execution time is proportional to the number of processors added. Even more encouraging, however, is that different search strategies, devised to take advantage of additional (or more powerful) processors, may actually lead to dramatic improvements in the performance of the basic algorithm. Thus search strategies intended for many processors actually may generate algorithms that are better even when implemented sequentially. The key difference is that the additional processors are not used simply to enhance the performance of an inherently sequential algorithm; they are used to spur the design of ever more ambitious—and effective—search strategies.
The algorithms given here are supported by a strong convergence theorem, promising computational results on a variety of problems, and an intuitively appealing interpretation as multidirectional line search methods.

On the Impact of Automatic Differentiation on the Relative Performance of Parallel Truncated Newton and Variable Metric Algorithms

L. C. W. Dixon

SIAM J. Optim. 1, pp. 475-486 (12 pages) | Cited 2 times

Online Publication Date: July 31, 2006

Full Text: | Download PDF

Show Abstract
The sparse doublet method for obtaining the gradient of a function or the Jacobian of a vector will be described and contrasted with reverse automatic differentiation. Its extension, the sparse triplet method for finding the Hessian of a function, will also be described and the effect of using these within classic optimisation algorithms discussed.
Results obtained using a parallel implementation of sparse triplet automatic differentiation of a partially separable function on the Sequent Balance will be presented.
In this paper it is shown that:• automatic differentiation can no longer be neglected as a method for calculating derivatives;• sparse triplets provide an effective method that can be implemented in parallel for calculating the Hessian matrix;• this approach can be combined effectively with the truncated Newton method when solving large unconstrained optimisation problems on parallel processors.

Parallel Constraint Distribution

M. C. Ferris and O. L. Mangasarian

SIAM J. Optim. 1, pp. 487-500 (14 pages) | Cited 9 times

Online Publication Date: July 31, 2006

Full Text: | Download PDF

Show Abstract
Constraints of a mathematical program are distributed among parallel processors together with an appropriately constructed augmented Lagrangian for each processor, which contains Lagrangian information on the constraints handled by the other processors. Lagrange multiplier information is then exchanged between processors. Convergence is established under suitable conditions for strongly convex quadratic programs and for general convex programs.

Parallel Solution of Large-Scale, Block-Diagonal Concave Maximization Problems

J. H. Glick, R. S. Maier, and J. B. Rosen

SIAM J. Optim. 1, pp. 501-514 (14 pages) | Cited 1 time

Online Publication Date: July 31, 2006

Full Text: | Download PDF

Show Abstract
A feasible-point algorithm for structured, large-scale, constrained optimization problems which may have many nonlinear constraints is described. The constraint structure is characterized by a block-diagonal coefficient matrix corresponding to linear variables, coupled by nonlinear variables. Problems of this structure arise in many applications, including structural design optimization and certain multiperiod or multiplant applications. Maximization problems with a concave objective and concave inequality constraints which define a convex region are considered. For such problems a KKT point is a global maximum. A basic version of the algorithm is presented and justified by showing that it will find an optimal solution in a finite number of iterations. The algorithm has been implemented on a CRAY-2 and a 64-processor NCUBE hypercube. It has been tested on a series of randomly generated test problems, and its performance has been compared with that of MINOS 5.3.

Parallel Genetic Algorithms Applied to the Traveling Salesman Problem

Prasanna Jog, Jung Y. Suh, and Dirk Van Gucht

SIAM J. Optim. 1, pp. 515-529 (15 pages) | Cited 9 times

Online Publication Date: July 31, 2006

Full Text: | Download PDF

Show Abstract
Genetic algorithms are adaptive search algorithms that have been shown to be robust optimization algorithms for multimodal real-valued functions and a variety of combinatorial optimization problems. In contrast to more standard search algorithms, genetic algorithms base their progress on the performance of a population of candidate solutions, rather than on a single candidate solution.
The authors will concentrate on the application of genetic algorithms to the traveling salesman problem. For this problem, there exist several such algorithms, ranging from pure genetic algorithms to genetic algorithms that incorporate heuristic information. These algorithms will be reviewed and their performance contrasted.
A serious drawback of genetic algorithms is their inefficiency when implemented on a sequential machine. However, due to their inherent parallel properties, they can be successfully implemented on parallel machines, resulting in considerable speedup. Parallel genetic algorithms will be reviewed and their uses in the traveling salesman problem will be indicated.

A General-Purpose Parallel Algorithm for Unconstrained Optimization

Stephen G. Nash and Ariela Sofer

SIAM J. Optim. 1, pp. 530-547 (18 pages)

Online Publication Date: July 31, 2006

Full Text: | Download PDF

Show Abstract
This paper describes a general-purpose algorithm for unconstrained optimization that is suitable for a parallel computer. It is designed to be as easy to use as traditional algorithms for this problem, requiring only that a (scalar) subroutine be provided to evaluate the objective function and its gradient vector of first derivatives. The algorithm used is a block truncated-Newton method. Truncated-Newton methods are a class of methods that compromise on Newton’s method so that large problems can be solved. Enhancements to the basic method suitable for a parallel computer are described. These include a revised data storage scheme, new preconditioning and initialization strategies to accelerate the method, a parallel line search, revised stopping rules for the inner algorithm, and a new “nonlinearity” test to determine the adequacy of the quadratic model. Numerical results are presented to illustrate the performance of the method, and comparisons are made with other scalar and parallel algorithms.

Acceleration and Parallelization of the Path-Following Interior Point Method for a Linearly Constrained Convex Quadratic Problem

Y. Nesterov and A. Nemirovsky

SIAM J. Optim. 1, pp. 548-564 (17 pages) | Cited 1 time

Online Publication Date: July 31, 2006

Full Text: | Download PDF

Show Abstract
In this paper, the strategies for acceleration of the path-following polynomial time interior point method for linear and linearly constrained quadratic programming problems are studied. These strategies are based on (i) exploiting the results of computations done at the previous iterations (Karmarkar’s acceleration scheme and a scheme based on the preconditioned conjugate gradient method); (ii) implementation of “fast” linear algebra routines; (iii) parallel computations.

Orderings for Conjugate Gradient Preconditionings

James M. Ortega

SIAM J. Optim. 1, pp. 565-582 (18 pages) | Cited 7 times

Online Publication Date: July 31, 2006

Full Text: | Download PDF

Show Abstract
Many preconditioners (e.g., SSOR, ILU) for the conjugate gradient method require the solution of sparse triangular systems of equations. For elliptic boundary value problems, one approach to obtaining additional parallelism in the solution of these systems is the use of red/black or multicolor orderings. There has been increasing evidence, however, that these orderings degrade the rate of convergence compared with the natural ordering. An alternative is the diagonal ordering, which maintains the rate of convergence of the natural ordering but has less parallelism than multicolor orderings. This paper reviews these as well as other orderings and then gives some results that help to explain why the red/black ordering gives an inferior rate of convergence.

An Interior Point Method for Block Angular Optimization

Gary L. Schultz and Robert R. Meyer

SIAM J. Optim. 1, pp. 583-602 (20 pages) | Cited 9 times

Online Publication Date: July 31, 2006

Full Text: | Download PDF

Show Abstract
An interior point method for block angular optimization is developed and the convergence properties of the method are described. A major motivation for such a method is that most of the computation is easily parallelized. Computational results are presented for a class of large-scale linear programming models. These models are multicommodity flow problems that arise from an Air Force (Military Airlift Command) application and generate problems as large as 100,000 rows and 300,000 columns.

On the Rate of Convergence of a Partially Asynchronous Gradient Projection Algorithm

Paul Tseng

SIAM J. Optim. 1, pp. 603-619 (17 pages) | Cited 6 times

Online Publication Date: July 31, 2006

Full Text: | Download PDF

Show Abstract
Recently, Bertsekas and Tsitsiklis proposed a partially asynchronous implementation of the gradient projection algorithm of Goldstein and Levitin and Polyak for the problem of minimizing a differentiable function over a closed convex set. In this paper, the rate of convergence of this algorithm is analyzed. It is shown that if the standard assumptions hold (that is, the solution set is nonempty and the gradient of the function is Lipschitz continuous) and (i) the isocost surfaces of the objective function, restricted to the solution set, are properly separated and (ii) a certain multifunction associated with the problem is locally upper Lipschitzian, then this algorithm attains a linear rate of convergence.

Partitioned Dynamic Programming for Optimal Control

Stephen J. Wright

SIAM J. Optim. 1, pp. 620-642 (23 pages) | Cited 4 times

Online Publication Date: July 31, 2006

Full Text: | Download PDF

Show Abstract
Parallel algorithms for the solution of linear-quadratic optimal control problems are described. The algorithms are based on a straightforward decomposition of the domain of the problem, and are related to multiple shooting methods for two-point boundary value problems. Their arithmetic cost is approximately twice that of the serial dynamic programming approach; however, they have the advantage that they can be efficiently implemented on a wide variety of parallel architectures. Extension to the case in which there are box constraints on the controls is simple. The algorithms can be used to solve linear-quadratic subproblems arising from the application of Newton’s method or two-metric gradient projection methods to nonlinear problems.

On the Fine-Grain Decomposition of Multicommodity Transportation Problems

Stavros A. Zenios

SIAM J. Optim. 1, pp. 643-669 (27 pages) | Cited 7 times

Online Publication Date: July 31, 2006

Full Text: | Download PDF

Show Abstract
A simple algorithm for nonlinear optimization problems with multicommodity transportation constraints is developed. The algorithm is of the row-action type and, when properly applied, decomposes the underlying graph alternatingly by nodes and arcs. Hence, a fine-grain decomposition scheme is developed that is suitable for massively parallel computer architectures of the SIMD (i.e., single instruction stream, multiple data stream) class.
Data structures are developed for the implementation of both dense and sparse problems, and details of an implementation on a Connection Machine CM-2 with 32K processing elements are given. The dense implementation achieves computing rate of up to three GFLOPS. Several aspects of the algorithm are investigated empirically. Computational results are reported for the solution of quadratic programs with approximately 10 million columns and 100 thousand rows.
Close

Your Recent History Recent History Help

Recently Viewed

  1. Optimal Error Estimates of the Semidiscrete Local Discontinuous Galerkin Methods for High Order Wave...
  2. On Rapid Computation of Expansions in Ultraspherical Polynomials
  3. A Computational Measure Theoretic Approach to Inverse Sensitivity Problems II: A Posteriori Error An...
  4. Strong and Weak Error Estimates for Elliptic Partial Differential Equations with Random Coefficients
  5. Convergence of a Particle Method and Global Weak Solutions of a Family of Evolutionary PDEs
© SIAM By using SIAM Publications Online you agree to abide by the Terms and Conditions of Use. Banner art adapted from a figure by Hinke M. Osinga and Bernd Krauskopf (University of Bristol, UK).
close