Precision Control for Generalized Pattern Search Algorithms with Adaptive Precision Function Evaluations

In the literature on generalized pattern search algorithms, convergence to a stationary point of a once continuously differentiable cost function is established under the assumption that the cost function can be evaluated exactly. However, there is a large class of engineering problems where the numerical evaluation of the cost function involves the solution of systems of differential algebraic equations. Since the termination criteria of the numerical solvers often depend on the design parameters, computer code for solving these systems usually defines a numerical approximation to the cost function that is discontinuous with respect to the design parameters. Standard generalized pattern search algorithms have been applied heuristically to such problems, but no convergence properties have been stated.

In this paper we extend a class of generalized pattern search algorithms to include a subprocedure that adaptively controls the precision of the approximating cost functions. The numerical approximations to the cost function need not define a continuous function. Our algorithms can be used for solving linearly constrained problems with cost functions that are at least locally Lipschitz continuous.

Assuming that the cost function is smooth, we prove that our algorithms converge to a stationary point. Under the weaker assumption that the cost function is only locally Lipschitz continuous, we show that our algorithms converge to points at which the Clarke generalized directional derivatives are nonnegative in predefined directions.

An important feature of our adaptive precision scheme is the use of coarse approximations in the early iterations, with the approximation precision controlled by a test. We show by numerical experiments that such an approach leads to substantial time savings in minimizing computationally expensive functions.

  • [1]  M. S. Al‐Homoud, Optimum thermal design of office buildings, Intern. J. Energy Res., 21 (1997), pp. 941–957. 9ka IJERDN 0363-907X Int. J. Energy Res. CrossrefISIGoogle Scholar

  • [2]  Google Scholar

  • [3]  Charles Audet and , J. Dennis, Analysis of generalized pattern searches, SIAM J. Optim., 13 (2002), 0–0889–903 (electronic) (2003) 10.1137/S1052623400378742 2004e:90122 LinkISIGoogle Scholar

  • [4]  A. J. Booker, J. E. Dennis, P. D. Frank, D. B. Serafini, V. Torczon and , and M. W. Trosset, A rigorous framework for optimization of expensive functions by surrogates, Struct. Optim., 17 (1999), pp. 1–13. 9gy SOPTEQ 0934-4373 Struct. Optim. CrossrefGoogle Scholar

  • [5]  K. Brenan, S. Campbell and , L. Petzold, Numerical solution of initial‐value problems in differential‐algebraic equations, Classics in Applied Mathematics, Vol. 14, Society for Industrial and Applied Mathematics (SIAM), 1996x+256, Revised and corrected reprint of the 1989 original 96h:65083 Google Scholar

  • [6]  Peter Brown, Alan Hindmarsh and , Linda Petzold, Using Krylov methods in the solution of large‐scale differential‐algebraic systems, SIAM J. Sci. Comput., 15 (1994), 1467–1488 10.1137/0915088 95g:65092 LinkISIGoogle Scholar

  • [7]  Peter Brown, Alan Hindmarsh and , Linda Petzold, Consistent initial condition calculation for differential‐algebraic systems, SIAM J. Sci. Comput., 19 (1998), 1495–1512 10.1137/S1064827595289996 99d:65202 LinkISIGoogle Scholar

  • [8]  Google Scholar

  • [9]  J. Dennis and , Virginia Torczon, Direct search methods on parallel machines, SIAM J. Optim., 1 (1991), 448–474 10.1137/0801027 92g:65069 LinkISIGoogle Scholar

  • [10]  J. Dennis and , Virginia Torczon, Managing approximation models in optimization, SIAM, Philadelphia, PA, 1997, 330–347 MR1438322 Google Scholar

  • [11]  Lawrence Evans, Partial differential equations, Graduate Studies in Mathematics, Vol. 19, American Mathematical Society, 1998xviii+662 99e:35001 Google Scholar

  • [12]  Google Scholar

  • [13]  Google Scholar

  • [14]  R. Hooke and  and T. A. Jeeves, “Direct search” solution of numerical and statistical problems, J. Assoc. Comput. Machinery, 8 (1961), pp. 212–229. jcm JACOAH 0004-5411 J. Assoc. Comput. Mach. CrossrefGoogle Scholar

  • [15]  Google Scholar

  • [16]  C. Kelley, Iterative methods for optimization, Frontiers in Applied Mathematics, Vol. 18, Society for Industrial and Applied Mathematics (SIAM), 1999xvi+180 2000h:90003 LinkGoogle Scholar

  • [17]  Tamara Kolda, Robert Lewis and , Virginia Torczon, Optimization by direct search: new perspectives on some classical and modern methods, SIAM Rev., 45 (2003), 385–482 10.1137/S003614450242889 2004m:90229 LinkISIGoogle Scholar

  • [18]  Google Scholar

  • [19]  Google Scholar

  • [20]  Robert Lewis and , Virginia Torczon, Pattern search algorithms for bound constrained minimization, SIAM J. Optim., 9 (1999), 1082–1099, Dedicated to John E. Dennis, Jr., on his 60th birthday 10.1137/S1052623496300507 2000m:90038 LinkISIGoogle Scholar

  • [21]  Robert Lewis and , Virginia Torczon, Pattern search methods for linearly constrained minimization, SIAM J. Optim., 10 (2000), 917–941 10.1137/S1052623497331373 2001h:90081 LinkISIGoogle Scholar

  • [22]  Google Scholar

  • [23]  Google Scholar

  • [24]  Google Scholar

  • [25]  Google Scholar

  • [26]  Virginia Torczon, On the convergence of pattern search algorithms, SIAM J. Optim., 7 (1997), 1–25 10.1137/S1052623493250780 97m:93047 LinkISIGoogle Scholar

  • [27]  Google Scholar

  • [28]  Google Scholar

  • [29]  Google Scholar

  • [30]  M. Wetter, BuildOpt – A new building energy simulation program that is built on smooth models, Building and Environment, 40 (2005), pp. 1085–1092. a55 BUSCBC 0360-1323 Build. Environ. CrossrefISIGoogle Scholar

  • [31]  Google Scholar

  • [32]  Google Scholar

  • [33]  M. Wetter and  and J. Wright, A comparison of deterministic and probabilistic optimization algorithms for nonsmooth simulation‐based optimization, Building and Environment, 39 (2004), pp. 989–999. a55 BUSCBC 0360-1323 Build. Environ. CrossrefISIGoogle Scholar

  • [34]  Google Scholar

  • [35]  Google Scholar