Abstract

The Nelder--Mead simplex algorithm, first published in 1965, is an enormously popular direct search method for multidimensional unconstrained minimization. Despite its widespread use, essentially no theoretical results have been proved explicitly for the Nelder--Mead algorithm. This paper presents convergence properties of the Nelder--Mead algorithm applied to strictly convex functions in dimensions 1 and 2. We prove convergence to a minimizer for dimension 1, and various limited convergence results for dimension 2. A counterexample of McKinnon gives a family of strictly convex functions in two dimensions and a set of initial conditions for which the Nelder--Mead algorithm converges to a nonminimizer. It is not yet known whether the Nelder--Mead method can be proved to converge to a minimizer for a more specialized class of convex functions in two dimensions.

MSC codes

  1. 49D30
  2. 65K05

Keywords

  1. direct search methods
  2. Nelder--Mead simplex methods
  3. nonderivative optimization

Get full access to this article

View all available purchase options and get full access to this article.

References

1.
J. Dennis, Jr., Virginia Torczon, Direct search methods on parallel machines, SIAM J. Optim., 1 (1991), 448–474
2.
C. Kelley, Detection and remediation of stagnation in the Nelder‐Mead algorithm using a sufficient decrease condition, SIAM J. Optim., 10 (1999), 43–55
3.
J. C. Lagarias, B. Poonen, and M. H. Wright, Convergence of the restricted Nelder‐Mead algorithm in two dimensions, in preparation, 1998.
4.
Math Works, Matlab, The Math Works, Natick, MA, 1994.
5.
K. McKinnon, Convergence of the Nelder‐Mead simplex method to a nonstationary point, SIAM J. Optim., 9 (1999), 148–158
6.
J. A. Nelder and R. Mead, A simplex method for function minimization, Computer Journal 7 (1965), 308–313.
7.
W. H. Press, B. P. Flannery, S. A. Teukolsky, and W. T. Vettering, Numerical Recipes in C, Cambridge University Press, Cambridge, UK, 1988.
8.
A. Rykov, Simplex direct search algorithms, 0–0
9.
A. Rykov, Simplex methods of direct search, 0–0
10.
Meng Lin, Secant‐simplex algorithm for unconstrained optimization, J. Numer. Methods Comput. Appl., 7 (1986), 153–157
11.
A. Stuart, A. Humphries, Dynamical systems and numerical analysis, Cambridge Monographs on Applied and Computational Mathematics, Vol. 2, Cambridge University Press, 1996xxii+685
12.
V. Torczon, Multi‐directional Search: A Direct Search Algorithm for Parallel Machines, Ph.D. thesis, Rice University, Houston, TX, 1989.
13.
Virginia Torczon, On the convergence of pattern search algorithms, SIAM J. Optim., 7 (1997), 1–25
14.
V. Torczon, Private communication, 1997.
15.
Paul Tseng, Fortified‐descent simplicial search method: a general approach, SIAM J. Optim., 10 (1999), 269–288
16.
F. H. Walters, L. R. Parker, S. L. Morgan, and S. N. Deming, Sequential Simplex Optimization, CRC Press, Boca Raton, FL, 1991.
17.
D. J. Woods, An Interactive Approach for Solving Multi‐objective Optimization Problems, Ph.D. thesis, Rice University, Houston, TX, 1985.
18.
M. Wright, Direct search methods: once scorned, now respectable, Pitman Res. Notes Math. Ser., Vol. 344, Longman, Harlow, 1996, 191–208

Information & Authors

Information

Published In

cover image SIAM Journal on Optimization
SIAM Journal on Optimization
Pages: 112 - 147
ISSN (online): 1095-7189

History

Published online: 31 July 2006

MSC codes

  1. 49D30
  2. 65K05

Keywords

  1. direct search methods
  2. Nelder--Mead simplex methods
  3. nonderivative optimization

Authors

Affiliations

Metrics & Citations

Metrics

Citations

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

Cited By