The Correlated Knowledge Gradient for Simulation Optimization of Continuous Parameters using Gaussian Process Regression

Abstract

We extend the concept of the correlated knowledge-gradient policy for the ranking and selection of a finite set of alternatives to the case of continuous decision variables. We propose an approximate knowledge gradient for problems with continuous decision variables in the context of a Gaussian process regression model in a Bayesian setting, along with an algorithm to maximize the approximate knowledge gradient. In the problem class considered, we use the knowledge gradient for continuous parameters to sequentially choose where to sample an expensive noisy function in order to find the maximum quickly. We show that the knowledge gradient for continuous decisions is a generalization of the efficient global optimization algorithm proposed in [D. R. Jones, M. Schonlau and W. J. Welch, J. Global Optim., 13 (1998), pp. 455–492].

MSC codes

  1. 62L05
  2. 62L10
  3. 62L20

Keywords

  1. model calibration
  2. Bayesian global optimization
  3. Gaussian process regression
  4. knowledge gradient
  5. expected improvement

Get full access to this article

View all available purchase options and get full access to this article.

References

1.
A. J. Booker, J. E. Dennis, P. D. Frank, D. B. Serafini, V. Torczon and M. W. Trosset, A rigorous framework for optimization of expensive functions by surrogates, Struct. Multidiscip. Optim., 17 (1999), pp. 1–13.
2.
G. E. Box and N. R. Draper, A basis for the selection of a response surface design, J. Amer. Statist. Assoc., 54 (1959), pp. 622–654.
3.
X. Chen, B. Ankenman and B. L. Nelson, The Effects of Common Random Numbers on Stochastic Kriging Metamodels, Technical report, Department of Industrial Engineering and Management Sciences, Northwestern University, Evanston, IL, 2010.
4.
A. R. Conn and K. Scheinberg, Geometry of sample sets in derivative-free optimization: Polynomial regression and underdetermined interpolation, IMA J. Numer. Anal., 28 (2008), p. 721–748.
5.
A. R. Conn, K. Scheinberg and P. L. Toint, Recent progress in unconstrained nonlinear optimization without derivatives, Math. Program., 79 (1997), pp. 397–414.
6.
N. Cressie, The origins of kriging, Math. Geol., 22 (1990), pp. 239–252.
7.
G. Deng and M. C. Ferris, Adaptation of the UOBYQA algorithm for noisy functions, in Proceedings of the 2006 Winter Simulation Conference, 2006, pp. 312–319.
8.
A. I. J. Forrester, A. Sóbester and A. J. Keane, Multi-fidelity optimization via surrogate modelling, Proc. R. Soc. London Ser. A, 463 (2007), pp. 3251–3269.
9.
A. I. J. Forrester, A. Sobester and A. J. Keane, Engineering Design via Surrogate Modelling: A Practical Guide, John Wiley & Sons, Chichester, UK, 2008.
10.
P. I. Frazier, W. B. Powell and S. Dayanik, A knowledge-gradient policy for sequential information collection, SIAM J. Control Optim., 47 (2008), pp. 2410–2439.
11.
P. Frazier, W. B. Powell and S. Dayanik, The knowledge gradient policy for correlated normal beliefs, INFORMS J. Comput., 21 (2009), pp. 599–613.
12.
M. C. Fu, F. W. Glover and J. April, Simulation optimization: A review, new developments, and applications, in Proceedings of the 2005 Winter Simulation Conference, 2005, pp. 83–95.
13.
P. Glasserman, Gradient Estimation via Perturbation Analysis, Kluwer Academic, Norwell, MA, 1991.
14.
H. M. Gutmann, A radial basis function method for global optimization, J. Global Optim., 19 (2001), pp. 201–227.
15.
W. J. Hill and W. G. Hunter, A review of response surface methodology: A literature survey, Technometrics, 8 (1966), pp. 571–590.
16.
D. Huang, T. T. Allen, W. I. Notz and N. Zeng, Global optimization of stochastic black-box systems via sequential kriging meta-models, J. Global Optim., 34 (2006), pp. 441–446.
17.
D. R. Jones, M. Schonlau and W. J. Welch, Efficient global optimization of expensive black-box functions, J. Global Optim., 13 (1998), pp. 455–492.
18.
J. Kiefer and J. Wolfowitz, Stochastic estimation of the maximum of a regression function, Ann. Math. Statist., 23 (1952), pp. 462–466.
19.
J. P. C. Kleijnen, Kriging metamodeling in simulation: A review, Eur. J. Oper. Res., 192 (2009), pp. 707–716.
20.
J. P. C. Kleijnen, W. Beers and I. Nieuwenhuyse, Constrained optimization in expensive simulation; novel approach, Eur. J. Oper. Res., 202 (2010), pp. 164–174.
21.
H. J. Kushner, A new method of locating the maximum point of an arbitrary multipeak curve in the presence of noise, J. Basic Eng., 86 (1964), pp. 97–106.
22.
M. Locatelli, Bayesian algorithms for one-dimensional global optimization, J. Global Optim., 10 (1997), pp. 57–76.
23.
J. L. Loeppky, J. Sacks and W. J. Welch, Choosing the sample size of a computer experiment: A practical guide, Technometrics, 51 (2009), pp. 366–376.
24.
G. Matheron, Principles of geostatistics, Economic Geology, 58 (1963), pp. 1246–1266.
25.
R. J. Meinhold and N. D. Singpurwalla, Understanding the Kalman filter, Amer. Statist., 37 (1983), pp. 123–127.
26.
J. Mockus, Application of Bayesian approach to numerical methods of global and stochastic optimization, J. Global Optim., 4 (1993), pp. 347–365.
27.
R. H. Myers, A. I. Khuri and W. H. Carter Jr., Response surface methodology: 1966–1988, Technometrics, 31 (1989), pp. 137–157.
28.
R. H. Myers, A. I. Khuri and G. Vining, Response surface alternatives to the Taguchi robust parameter design approach, Amer. Statist., 46 (1992), pp. 131–139.
29.
R. H. Myers, D. C. Montgomery and C. M. Anderson-Cook, Response Surface Methodology, John Wiley & Sons, New York, 2009.
30.
M. J. D. Powell, UOBYQA: Unconstrained optimization by quadratic approximation, Math. Program., 92 (2002), pp. 555–582.
31.
C. E. Rasmussen and C. K. I. Williams, Gaussian Processes for Machine Learning, MIT Press, Cambridge, MA, 2006.
32.
R. G. Regis and C. A. Shoemaker, Constrained global optimization of expensive black box functions using radial basis functions, J. Global Optim., 31 (2005), pp. 153–171.
33.
H. Robbins and S. Monro, A stochastic approximation method, Ann. Math. Statist., 22 (1951), pp. 400–407.
34.
R. Y. Rubinstein and A. Shapiro, Optimization of static simulation models by the score function method, Math. Comput. Simul., 32 (1990), pp. 373–392.
35.
J. Sacks, W. J. Welch, T. J. Mitchell and H. P. Wynn, Design and analysis of computer experiments, Statist. Sci., 4 (1989), pp. 409–423.
36.
M. J. Sasena, Flexibility and Efficency Enhancements for Constrained Global Design Optimization with Kriging Approximations, Ph.D. thesis, University of Michigan, 2002.
37.
M. Schonlau and W. J. Welch, Global optimization with nonparametric function fitting, in Proceedings of the Section on Physical and Engineering Sciences, American Statistical Association, 1996, pp. 183–186.
38.
M. Schonlau, W. J. Welch and D. R. Jones, Global versus local search in constrained optimization of computer models, in New Developments and Applications in Experimental Design, IMS Lecture Notes Monogr. Ser. 34, Institute of Mathematical Statistics, Hayward, CA, 1998, pp. 11–25.
39.
D. W. Scott, Parametric statistical model by minimum integrated square error, Technometrics, 43 (2001), pp. 274–285.
40.
J. C. Spall, Introduction to Stochastic Search and Optimization, Wiley-Interscience, New York, 2003.
41.
A. Torn and A. Zilinskas, Global Optimization, Lecture Notes in Comput. Sci. 1, Springer-Verlag, Berlin, 1989.
42.
E. Vazquez and J. Bect, Convergence properties of the expected improvement algorithm with fixed mean and covariance functions, J. Statist. Plann. Inference, 140 (2010), pp. 3088–3095.
43.
E. Vazquez, J. Villemonteix, M. Sidorkiewicz and E. Walter, Global optimization based on noisy evaluations: An empirical study of two statistical approaches, J. Phys. Conf. Ser., 135 (2008),.
44.
J. Villemonteix, E. Vazquez and E. Walter, An informational approach to the global optimization of expensive-to-evaluate functions, J. Global Optim., 44 (2009), pp. 509–534.
45.
B. L. Welch, The generalization of “Student’s” problem when several different population variances are involved, Biometrika, 34 (1947), pp. 28–35.
46.
A. G. Zhilinskas, Single-step Bayesian search method for an extremum of functions of a single variable, Cybernet. Systems Anal., 11 (1975), pp. 160–166.

Information & Authors

Information

Published In

cover image SIAM Journal on Optimization
SIAM Journal on Optimization
Pages: 996 - 1026
ISSN (online): 1095-7189

History

Submitted: 7 July 2010
Accepted: 20 May 2011
Published online: 22 September 2011

MSC codes

  1. 62L05
  2. 62L10
  3. 62L20

Keywords

  1. model calibration
  2. Bayesian global optimization
  3. Gaussian process regression
  4. knowledge gradient
  5. expected improvement

Authors

Affiliations

Metrics & Citations

Metrics

Citations

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

Cited By

View Options

View options

PDF

View PDF

Figures

Tables

Media

Share

Share

Copy the content Link

Share with email

Email a colleague

Share on social media