Number Of Expensive Function Evaluations Research Articles

Surrogate-assisted evolutionary framework with an ensemble of teaching-learning and differential evolution for expensive optimization

Surrogate-Assisted Evolutionary Algorithms (SAEAs) integrate Evolutionary Algorithms (EAs) with surrogate models to reduce the actual number of expensive function evaluations and have been widely used in solving Expensive Optimization Problems (EOPs). However, because of the insufficient diversity of new solutions, SAEAs often fall into local optima and result in evolutionary stagnation when dealing with complex problems. To improve the diversity of new solutions, this paper introduces a Surrogate-Assisted evolutionary Framework with an ensemble of Teaching-learning and Differential evolution (SAF-TD), which successfully integrates the strengths of both EAs to produce more diverse and effective solutions. The main contributions are summarized as follows: First, a novel framework that effectively integrates two distinct EAs using a radial basis function surrogate model and strategic sampling is introduced. Second, a mutation strategy with a supervisory mechanism is proposed to enhance mutation effectiveness and avoid stagnation. Third, a volatility index derived from dimensional improvements is incorporated into parameter control to address fitness-dependent weaknesses. Experiments on five expensive optimization functions and the CEC2017 test suite across multiple dimensions were conducted to evaluate the performance of SAF-TD. The results demonstrate that SAF-TD is highly competitive compared with state-of-the-art SAEAs and exhibits excellent performance in solving EOPs.

A performance indicator-based evolutionary algorithm for expensive high-dimensional multi-/many-objective optimization

Surrogate-assisted multi-objective evolutionary algorithms have shown considerable potential for solving optimization problems in which only a small number of expensive function evaluations are available. However, most existing research remains restricted to low-/medium-dimensional problems, with very little attention paid to addressing problems involving decision variables with more than 100 dimensions. In this study, a performance indicator-based evolutionary algorithm (PIEA) is proposed for expensive high-dimensional multi-/many-objective optimization. A surrogate model is employed to approximate the performance indicator rather than directly predicting the objective function, thus simplifying the optimization complexity and mitigating the impact of cumulative errors. An efficient indicator-based optimization strategy emphasising the balance between exploration and exploitation is designed for surrogate-assisted evolution and infill sampling. A history-based selection strategy is implemented to select a suitable indicator from the preset pool for each optimization cycle. An empirical study was conducted on two well-known benchmark suites, and the results demonstrate the superiority of the proposed algorithm over several state-of-the-art algorithms. Moreover, we integrate this concept into a classification-based framework, which further verifies its effectiveness.

Data-driven Harris Hawks constrained optimization for computationally expensive constrained problems

Aiming at the constrained optimization problem where function evaluation is time-consuming, this paper proposed a novel algorithm called data-driven Harris Hawks constrained optimization (DHHCO). In DHHCO, Kriging models are utilized to prospect potentially optimal areas by leveraging computationally expensive historical data during optimization. Three powerful strategies are, respectively, embedded into different phases of conventional Harris Hawks optimization (HHO) to generate diverse candidate sample data for exploiting around the existing sample data and exploring uncharted region. Moreover, a Kriging-based data-driven strategy composed of data-driven population construction and individual selection strategy is presented, which fully mines and utilizes the potential available information in the existing sample data. DHHCO inherits and develops HHO's offspring updating mechanism, and meanwhile exerts the prediction ability of Kriging, reduces the number of expensive function evaluations, and provides new ideas for data-driven constraint optimization. Comprehensive experiments have been conducted on 13 benchmark functions and a real-world expensive optimization problem. The experimental results suggest that the proposed DHHCO can achieve quite competitive performance compared with six representative algorithms and can find the near global optimum with 200 function evaluations for most examples. Moreover, DHHCO is applied to the structural optimization of the internal components of the real underwater vehicle, and the final satisfactory weight reduction effect is more than 18%.

A Survey on Sustainable Surrogate-Based Optimisation

Surrogate-based optimisation (SBO) algorithms are a powerful technique that combine machine learning and optimisation to solve expensive optimisation problems. This type of problem appears when dealing with computationally expensive simulators or algorithms. By approximating the expensive part of the optimisation problem with a surrogate, the number of expensive function evaluations can be reduced. This paper defines sustainable SBO, which consists of three aspects: applying SBO to a sustainable application, reducing the number of expensive function evaluations, and considering the computational effort of the machine learning and optimisation parts of SBO. The paper reviews sustainable applications that have successfully applied SBO over the past years, and analyses the used framework, type of surrogate used, sustainable SBO aspects, and open questions. This leads to recommendations for researchers working on sustainability-related applications who want to apply SBO, as well as recommendations for SBO researchers. It is argued that transparency of the computation resources used in the SBO framework, as well as developing SBO techniques that can deal with a large number of variables and objectives, can lead to more sustainable SBO.

A Kriging-Assisted Two-Archive Evolutionary Algorithm for Expensive Many-Objective Optimization

Only a small number of function evaluations can be afforded in many real-world multiobjective optimization problems (MOPs) where the function evaluations are economically/computationally expensive. Such problems pose great challenges to most existing multiobjective evolutionary algorithms (EAs), which require a large number of function evaluations for optimization. Surrogate-assisted EAs (SAEAs) have been employed to solve expensive MOPs. Specifically, a certain number of expensive function evaluations are used to build computationally cheap surrogate models for assisting the optimization process without conducting expensive function evaluations. The infill sampling criteria in most existing SAEAs take all requirements on convergence, diversity, and model uncertainty into account, which is, however, not the most efficient in exploiting the limited computational budget. Thus, this article proposes a Kriging-assisted two-archive EA for expensive many-objective optimization. The proposed algorithm uses one influential point-insensitive model to approximate each objective function. Moreover, an adaptive infill criterion that identifies the most important requirement on convergence, diversity, or uncertainty is proposed to determine an appropriate sampling strategy for reevaluations using the expensive objective functions. The experimental results on a set of expensive multi/many-objective test problems have demonstrated its superiority over five state-of-the-art SAEAs.

A surrogate-assisted particle swarm optimization using ensemble learning for expensive problems with small sample datasets

Solving the real-world optimization problems often needs a large number of expensive function evaluations (FEs) by using evolutionary algorithms (EAs). To alleviate this difficulty, surrogate-assisted EAs (SAEAs) have attracted more and more attention from academia and industry. However, the existing SAEAs need a large amount of sample points to construct surrogate model within the expected times of FEs, otherwise it cannot achieve satisfactory prediction accuracy. Few SAEAs can reduce the times of expensive FEs while a high-quality surrogate model is constructed using a small number of sample points. In this paper, a novel SAEAs inspired from ensemble learning is proposed. In the proposed algorithm, the small sample date set is divided into multiple subsets, and the surrogate model is trained on each subset. Two new model management strategies based on ensemble learning are applied to global search and local search respectively. Two search methods are cleverly combined to form a high precision surrogate ensemble. In order to verify the performance of the proposed method, we performed comprehensive tests on eight benchmark functions from 10 to 50 dimensions, and compared their result with the five state-of-the-art SAEAs. Experimental results demonstrate that the proposed method shows superior performance in a majority of benchmarks when only a limited computational budget is available. In addition, we apply the proposed algorithm to three real-time optimization problems. The results of each problem are compared with the solutions to verify the effectiveness of the algorithm in solving engineering application problems.

Occupancy map building through Bayesian exploration

We propose a novel holistic approach to safe autonomous exploration and map building based on constrained Bayesian optimization. This method finds optimal continuous paths instead of discrete sensing locations that inherently satisfy motion and safety constraints. Evaluating both the objective and constraints functions requires forward simulation of expected observations. As such, evaluations are costly, and therefore the Bayesian optimizer proposes only paths that are likely to yield optimal results and satisfy the constraints with high confidence. By balancing the reward and risk associated with each path, the optimizer minimizes the number of expensive function evaluations. We demonstrate the effectiveness of our approach in a series of experiments both in simulation and with a real ground robot and provide comparisons with other exploration techniques. The experimental results show that our method provides robust and consistent performance in all tests and performs better than or as good as the state of the art.

Model inversion via multi-fidelity Bayesian optimization: a new paradigm for parameter estimation in haemodynamics, and beyond.

We present a computational framework for model inversion based on multi-fidelity information fusion and Bayesian optimization. The proposed methodology targets the accurate construction of response surfaces in parameter space, and the efficient pursuit to identify global optima while keeping the number of expensive function evaluations at a minimum. We train families of correlated surrogates on available data using Gaussian processes and auto-regressive stochastic schemes, and exploit the resulting predictive posterior distributions within a Bayesian optimization setting. This enables a smart adaptive sampling procedure that uses the predictive posterior variance to balance the exploration versus exploitation trade-off, and is a key enabler for practical computations under limited budgets. The effectiveness of the proposed framework is tested on three parameter estimation problems. The first two involve the calibration of outflow boundary conditions of blood flow simulations in arterial bifurcations using multi-fidelity realizations of one- and three-dimensional models, whereas the last one aims to identify the forcing term that generated a particular solution to an elliptic partial differential equation.

Efficient infill sampling for unconstrained robust optimization problems

A novel infill sampling criterion is proposed for efficient estimation of the global robust optimum of expensive computer simulation based problems. The algorithm is especially geared towards addressing problems that are affected by uncertainties in design variablesand problem parameters. The method is based on constructing metamodels using Kriging and adaptively sampling the response surface via a principle of expected improvement adapted for robust optimization. Several numerical examples and an engineering case study are used to demonstrate the ability of the algorithm to estimate the global robust optimum using a limited number of expensive function evaluations.

Comparative study of surrogate approaches while optimizing computationally expensive reaction networks

Process modeling and optimization of polymerization processes with long chain branching is currently an area of extensive research owing to the advantages and growing popularity of branched polymers. The highly complex nature of these reaction networks results in a large set of stiff ordinary differential equations to model them mathematically with adequate precision and accuracy. In such a scenario, where execution time of the model is expensive, the idea of going for online optimization and control of such processes seems to be a near impossible task. Catering these problems in the ongoing research, optimization using surrogate model obtained from a novel algorithm is proposed in this work as a solution. A Sobol set assisted artificial neural network replaces the computationally expensive kinetic model of long chain branched poly vinyl acetate as the fast and efficient surrogate model. The proposed multi-objective methodology allows the computationally expensive first principle model to determine the configuration of the neural network, which can emulate it with maximum accuracy along with sample size required. The algorithm introduces a logical way of designing ANN architectures where the outperformance of multiple layer networks justifies the elimination of heuristics approach to consider only single layer. The results of the proposed algorithm are compared with the results obtained using Kriging interpolator based another surrogate approach, for testing, validation and scope of improvement. The use of fast and efficient Sobol assisted ANN surrogate model makes the optimization process ~10 times more efficient as compared to the case of optimization with computationally expensive kinetic model. The proposed ANN based surrogate is nearly 1.5 times as efficient as Kriging model in terms of number of expensive function evaluations.

Efficient global robust optimization of unconstrained problems affected by parametric uncertainties

A novel technique for efficient global robust optimization of problems affected by parametric uncertainties is proposed. The method is especially relevant to problems that are based on expensive computer simulations. The globally robust optimal design is obtained by searching for the best worst-case cost, which involves a nested min-max optimization problem. In order to reduce the number of expensive function evaluations, we fit response surfaces using Kriging and use adapted versions of expected improvement to direct the search for the robust optimum. The numerical performance of the algorithm is compared against other techniques for min-max optimization on established test problems. The proposed approach exhibits reliable convergence, is more efficient than previous methods and shows strong scalability.

On Estimating the Reliability of Multiple Failure Region Problems Using Approximate Metamodels

Abstract In a complex system it is desirable to reduce the number of expensive function evaluations required for an accurate estimation of the probability of failure. An efficient reliability estimation method is presented for engineering systems with multiple failure regions and potentially multiple most probable points. The method can handle implicit nonlinear limit state functions with correlated or noncorrelated random variables, which can be described by any probabilistic distribution. It uses a combination of approximate or “accurate-on-demand,” global and local metamodels, which serve as indicators to determine the failure and safe regions. Sample points close to limit states define transition regions between safe and failure domains. A clustering technique identifies all transition regions, which can be, in general, disjoint, and local metamodels of the actual limit states are generated for each transition region. Importance sampling generates sample points only in the identified transition and failure regions, thus, allowing the method to focus on the areas near the failure region and not expend computational effort on the sample points in the safe domain. A robust maximin “space-filling” sampling technique is used to construct the metamodels. Two numerical examples highlight the accuracy and efficiency of the method.

ORBIT: Optimization by Radial Basis Function Interpolation in Trust-Regions

We present a new derivative-free algorithm, ORBIT, for unconstrained local optimization of computationally expensive functions. A trust-region framework using interpolating Radial Basis Function (RBF) models is employed. The RBF models considered often allow ORBIT to interpolate nonlinear functions using fewer function evaluations than the polynomial models considered by present techniques. Approximation guarantees are obtained by ensuring that a subset of the interpolation points is sufficiently poised for linear interpolation. The RBF property of conditional positive definiteness yields a natural method for adding additional points. We present numerical results on test problems to motivate the use of ORBIT when only a relatively small number of expensive function evaluations are available. Results on two very different application problems, calibration of a watershed model and optimization of a PDE-based bioremediation plan, are also encouraging and support ORBIT's effectiveness on blackbox functions for which no special mathematical structure is known or available.