Global Optimization Of Expensive Functions Research Articles

Variable-fidelity hypervolume-based expected improvement criteria for multi-objective efficient global optimization of expensive functions

Variable-fidelity surrogate-based efficient global optimization (EGO) method with the ability to adaptively select low-fidelity (LF) and high-fidelity (HF) infill point has emerged as an alternative to further save the computational cost of the single-fidelity EGO method. However, in terms of the variable-fidelity surrogate-assisted multi-objective optimization methods, existing methods rely on empirical parameters or are unable to adaptively select LF/HF sample in the optimal search process. In this paper, two variable-fidelity hypervolume-based expected improvement criteria with analytic expressions for variable-fidelity multi-objective EGO method are proposed. The first criterion relies on the concept of variable-fidelity expected improvement matrix (VFEIM) and is obtained by aggregating the VFEIM using a simplified hypervolume-based aggregation scheme. The second criterion termed as VFEMHVI is derived analytically based on a modified hypervolume definition. Both criteria can adaptively select new LF/HF samples in the iterative optimal search process to update the variable-fidelity models towards the HF Pareto front, distinguishing the proposed methods to the rests in the open literature. The constrained versions of the two criteria are also derived for problems with constraints. The effectiveness and efficiency of the proposed methods are verified and validated over analytic problems and demonstrated by two engineering problems including aerodynamic shape optimizations of the NACA0012 and RAE2822 airfoils. The results show that the VFEMHVI combined with the normalization-based strategy to define the reference point is the most efficient one over the compared methods.

Parallel Bayesian Global Optimization of Expensive Functions

Large-Scale Parallel Bayesian Optimization

Finding optimal points for expensive functions using adaptive RBF-based surrogate model via uncertainty quantification

Global optimization of expensive functions has important applications in physical and computer experiments. It is a challenging problem to develop efficient optimization scheme, because each function evaluation can be costly and the derivative information of the function is often not available. We propose a novel global optimization framework using adaptive radial basis functions (RBF) based surrogate model via uncertainty quantification. The framework consists of two iteration steps. It first employs an RBF-based Bayesian surrogate model to approximate the true function, where the parameters of the RBFs can be adaptively estimated and updated each time a new point is explored. Then it utilizes a model-guided selection criterion to identify a new point from a candidate set for function evaluation. The selection criterion adopted here is a sample version of the expected improvement criterion. We conduct simulation studies with standard test functions, which show that the proposed method has some advantages, especially when the true function has many local optima. In addition, we also propose modified approaches to improve the search performance for identifying optimal points.

Variable-fidelity expected improvement method for efficient global optimization of expensive functions

The efficient global optimization method (EGO) based on kriging surrogate model and expected improvement (EI) has received much attention for optimization of high-fidelity, expensive functions. However, when the standard EI method is directly applied to a variable-fidelity optimization (VFO) introducing assistance from cheap, low-fidelity functions via hierarchical kriging (HK) or cokriging, only high-fidelity samples can be chosen to update the variable-fidelity surrogate model. The theory of infilling low-fidelity samples towards the improvement of high-fidelity function is still a blank area. This article proposes a variable-fidelity EI (VF-EI) method that can adaptively select new samples of both low and high fidelity. Based on the theory of HK model, the EI of the high-fidelity function associated with adding low- and high-fidelity sample points are analytically derived, and the resulting VF-EI is a function of both the design variables x and the fidelity level l. Through maximizing the VF-EI, both the sample location and fidelity level of next numerical evaluation are determined, which in turn drives the optimization converging to the global optimum of high-fidelity function. The proposed VF-EI is verified by six analytical test cases and demonstrated by two engineering problems, including aerodynamic shape optimizations of RAE 2822 airfoil and ONERA M6 wing. The results show that it can remarkably improve the optimization efficiency and compares favorably to the existing methods.

A kind of balance between exploitation and exploration on kriging for global optimization of expensive functions

In this paper, a novel kriging-based algorithm for global optimization of computationally expensive black-box functions is presented. This algorithm utilizes a multi-start approach to find all of the local optimal values of the surrogate model and performs searches within the neighboring area around these local optimal positions. Compared with traditional surrogate-based global optimization method, this algorithm provides another kind of balance between exploitation and exploration on kriging-based model. In addition, a new search strategy is proposed and coupled into this optimization process. The local search strategy employs a kind of improved “Minimizing the predictor” method, which dynamically adjusts search direction and radius until finds the optimal value. Furthermore, the global search strategy utilizes the advantage of kriging-based model in predicting unexplored regions to guarantee the reliability of the algorithm. Finally, experiments on 13 test functions with six algorithms are set up and the results show that the proposed algorithm is very promising.

A quasi-multistart framework for global optimization of expensive functions using response surface models

We present the AQUARS (A QUAsi-multistart Response Surface) framework for finding the global minimum of a computationally expensive black-box function subject to bound constraints. In a traditional multistart approach, the local search method is blind to the trajectories of the previous local searches. Hence, the algorithm might find the same local minima even if the searches are initiated from points that are far apart. In contrast, AQUARS is a novel approach that locates the promising local minima of the objective function by performing local searches near the local minima of a response surface (RS) model of the objective function. It ignores neighborhoods of fully explored local minima of the RS model and it bounces between the best partially explored local minimum and the least explored local minimum of the RS model. We implement two AQUARS algorithms that use a radial basis function model and compare them with alternative global optimization methods on an 8-dimensional watershed model calibration problem and on 18 test problems. The alternatives include EGO, GLOBALm, MLMSRBF (Regis and Shoemaker in INFORMS J Comput 19(4):497---509, 2007), CGRBF-Restart (Regis and Shoemaker in J Global Optim 37(1):113---135 2007), and multi level single linkage (MLSL) coupled with two types of local solvers: SQP and Mesh Adaptive Direct Search (MADS) combined with kriging. The results show that the AQUARS methods generally use fewer function evaluations to identify the global minimum or to reach a target value compared to the alternatives. In particular, they are much better than EGO and MLSL coupled to MADS with kriging on the watershed calibration problem and on 15 of the test problems.

Parallel Stochastic Global Optimization Using Radial Basis Functions

We develop a parallel implementation of a stochastic radial basis function (RBF) algorithm for global optimization by Regis and Shoemaker [Regis, R. G., C. A. Shoemaker. 2007a. A stochastic radial basis function method for the global optimization of expensive functions. INFORMS J. Comput. 19(4) 497–509]. The proposed parallel algorithm is suitable for the global optimization of computationally expensive objective functions and does not require derivatives. Each iteration of the algorithm consists of building an RBF model to approximate the expensive function and using this model to select multiple points for simultaneous function evaluation on multiple processors. The function evaluation points are selected from a set of random candidate points according to two criteria: estimated function value based on the RBF model, and minimum distance from previously evaluated points and previously selected points within each iteration. We compare the performance of our parallel stochastic RBF algorithm against alternative parallel global optimization methods, including two multistart parallel finite-difference quasi-Newton methods, a multistart implementation of Asynchronous Parallel Pattern Search [Hough, P., T. G. Kolda, V. J. Torczon. 2001. Asynchronous parallel pattern search for nonlinear optimization. SIAM J. Sci. Comput. 23(1) 134–156], a parallel implementation of Probabilistic Global Search Lausanne [Raphael, B., I. F. C. Smith. 2003. A direct stochastic algorithm for global search. Appl. Math. Comput. 146 729–758], a parallel evolutionary algorithm, and a deterministic parallel RBF algorithm by Regis and Shoemaker [Regis, R. G., C. A. Shoemaker. 2007c. Parallel radial basis function methods for the global optimization of expensive functions. Eur. J. Oper. Res. 182(2) 514–535]. We report good results for our parallel stochastic RBF method when using one, four, or eight processors in comparison with the alternatives on 20 test problems and on 3 optimization problems involving groundwater bioremediation.

A Stochastic Radial Basis Function Method for the Global Optimization of Expensive Functions

We introduce a new framework for the global optimization of computationally expensive multimodal functions when derivatives are unavailable. The proposed Stochastic Response Surface (SRS) Method iteratively utilizes a response surface model to approximate the expensive function and identifies a promising point for function evaluation from a set of randomly generated points, called candidate points. Assuming some mild technical conditions, SRS converges to the global minimum in a probabilistic sense. We also propose Metric SRS (MSRS), which is a special case of SRS where the function evaluation point in each iteration is chosen to be the best candidate point according to two criteria: the estimated function value obtained from the response surface model, and the minimum distance from previously evaluated points. We develop a global optimization version and a multistart local optimization version of MSRS. In the numerical experiments, we used a radial basis function (RBF) model for MSRS and the resulting algorithms, Global MSRBF and Multistart Local MSRBF, were compared to 6 alternative global optimization methods, including a multistart derivative-based local optimization method. Multiple trials of all algorithms were compared on 17 multimodal test problems and on a 12-dimensional groundwater bioremediation application involving partial differential equations. The results indicate that Multistart Local MSRBF is the best on most of the higher dimensional problems, including the groundwater problem. It is also at least as good as the other algorithms on most of the lower dimensional problems. Global MSRBF is competitive with the other alternatives on most of the lower dimensional test problems and also on the groundwater problem. These results suggest that MSRBF is a promising approach for the global optimization of expensive functions.

Parallel radial basis function methods for the global optimization of expensive functions

We introduce a master–worker framework for parallel global optimization of computationally expensive functions using response surface models. In particular, we parallelize two radial basis function (RBF) methods for global optimization, namely, the RBF method by Gutmann [Gutmann, H.M., 2001a. A radial basis function method for global optimization. Journal of Global Optimization 19(3), 201–227] (Gutmann-RBF) and the RBF method by Regis and Shoemaker [Regis, R.G., Shoemaker, C.A., 2005. Constrained global optimization of expensive black box functions using radial basis functions, Journal of Global Optimization 31, 153–171] (CORS-RBF). We modify these algorithms so that they can generate multiple points for simultaneous evaluation in parallel. We compare the performance of the two parallel RBF methods with a parallel multistart derivative-based algorithm, a parallel multistart derivative-free trust-region algorithm, and a parallel evolutionary algorithm on eleven test problems and on a 6-dimensional groundwater bioremediation application. The results indicate that the two parallel RBF algorithms are generally better than the other three alternatives on most of the test problems. Moreover, the two parallel RBF algorithms have comparable performances on the test problems considered. Finally, we report good speedups for both parallel RBF algorithms when using a small number of processors.

A constraint mapping approach to the structural optimization of an expensive model using surrogates (in special issue on surrogate modelling and space mapping for engineering optimization)

The use of response surface methods are well established in the global optimization of expensive functions, the response surface acting as a surrogate to the expensive function objective.In structural design however, the change in objective may vary little between the two models: it is more often the constraints that change with models of varying fidelity. Here approaches are described whereby the coarse model constraints are mapped so that the mapped constraints more faithfully approximate the fine model constraints. The shape optimization of a simple structure demonstrates the approach.