An algorithm for solving large nonlinear optimization problems with simple bounds is described. It is based on the gradient projection method and uses a limited memory BFGS matrix to approximate the Hessian of the objective function. It is shown how to take advantage of the form of the limited memory approximation to implement the algorithm efficiently. The results of numerical tests on a set of large problems are reported.

MSC codes

  1. 65
  2. 49


  1. bound constrained optimization
  2. limited memory method
  3. nonlinear optimization
  4. quasi-Newton method
  5. large-scale optimization

Get full access to this article

View all available purchase options and get full access to this article.


A. V. Aho, J. E. Hopcroff, J. D. Ullman, The Design and Analysis of Computer Algorithms, Addison-Wesley, Reading, MA, 1974
B. M. Averick, J. J. Moré, User Guide for the MINPMCK-2 Test Problem Collection, Report, ANLIMCS-TM-157, Argonne National Laboratory, Mathematics and Computer Science Division, Argonne, IL, 1991
Dimitri P. Bertsekas, Projected Newton methods for optimization problems with simple constraints, SIAM J. Control Optim., 20 (1982), 221–246
I. Bongartz, A. R. Conn, N. I. M. Gould, PH. L. Toint, CUTE: Constrained and Unconstrained Testing Environment, Research Report, IBM T.J. Watson Research Center, Yorktown, NY, 1993
James V. Burke, Jorge J. Moré, On the identification of active constraints, SIAM J. Numer. Anal., 25 (1988), 1197–1211
Richard H. Byrd, Jorge Nocedal, Robert B. Schnabel, Representations of quasi-Newton matrices and their use in limited memory methods, Math. Programming, 63 (1994), 129–156
Paul H. Calamai, Jorge J. Moré, Projected gradient methods for linearly constrained problems, Math. Programming, 39 (1987), 93–116
Andrew R. Conn, Nicholas I. M. Gould, Ph. L. Toint, Testing a class of methods for solving minimization problems with simple bounds on the variables, Math. Comp., 50 (1988), 399–430
A. R. Conn, N. I. M. Gould, Ph. L. Toint, Global convergence of a class of trust region algorithms for optimization with simple bounds, SIAM J. Numer. Anal., 25 (1988), 433–460
A. R. Conn, N. I. M. Gould, Ph. L. Toint, LANCELOT: A FORTRAN package for Large-Scale Nonlinear Optimization, Release A, Springer Series in Computational Mathematics, Vol. 17, Springer-Verlag, Berlin, 1992xx+330
A. R. Conn, J. Moré, 1993, Private communication
John E. Dennis, Jr., Robert B. Schnabel, Numerical methods for unconstrained optimization and nonlinear equations, Prentice Hall Series in Computational Mathematics, Prentice Hall Inc., Englewood Cliffs, NJ, 1983xiii+378
Harwell Subroutine Library, Release 10, Oxfordshire, UK, 1990
Jean Charles Gilbert, Claude Lemaréchal, Some numerical experiments with variable-storage quasi-Newton algorithms, Math. Programming, 45 (1989), 407–435
Philip E. Gill, Walter Murray, Margaret H. Wright, Practical optimization, Academic Press Inc. [Harcourt Brace Jovanovich Publishers], London, 1981xvi+401
A. A. Goldstein, Convex programming in Hilbert space, Bull. Amer. Math. Soc., 70 (1964), 709–710
E. S. Levitin, B. T. Polyak, Constrained minimization problems, USSR Comput. Math. Math. Phys., 6 (1966), 1–50
Dong C. Liu, Jorge Nocedal, On the limited memory BFGS method for large scale optimization, Math. Programming, 45 (1989), 503–528
J. J. Moré, D. J. Thuente, On Line Search Algorithms with Guaranteed Sufficient Decrease, 1990, Mathematics and Computer Science Division Preprint MCS-P153-0590, Argonne National Laboratory, Argonne, IL
Jorge J. Moré, Gerardo Toraldo, Algorithms for bound constrained quadratic programming problems, Numer. Math., 55 (1989), 377–400
Jorge Nocedal, Updating quasi-Newton matrices with limited storage, Math. Comp., 35 (1980), 773–782
J. M. Ortega, W. C. Rheinboldt, Iterative solution of nonlinear equations in several variables, Academic Press, New York, 1970xx+572
C. Zhu, R. H. Byrd, P. Lu, J. Nocedal, LBFGS-B : Fortran Subroutines for Large-Scale Bound Constrained Optimization, Report, NAM-11, EELS Department, Northwestern University, 1994

Information & Authors


Published In

cover image SIAM Journal on Scientific Computing
SIAM Journal on Scientific Computing
Pages: 1190 - 1208
ISSN (online): 1095-7197


Submitted: 29 November 1993
Accepted: 3 August 1994
Published online: 13 July 2006

MSC codes

  1. 65
  2. 49


  1. bound constrained optimization
  2. limited memory method
  3. nonlinear optimization
  4. quasi-Newton method
  5. large-scale optimization



Metrics & Citations



If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

Cited By