Surrogate-assisted Evolutionary Algorithms Research Articles

Surrogate-assisted evolutionary algorithms for expensive combinatorial optimization: a survey

AbstractAs potent approaches for addressing computationally expensive optimization problems, surrogate-assisted evolutionary algorithms (SAEAs) have garnered increasing attention. Prevailing endeavors in evolutionary computation predominantly concentrate on expensive continuous optimization problems, with a notable scarcity of investigations directed toward expensive combinatorial optimization problems (ECOPs). Nevertheless, numerous ECOPs persist in practical applications. The widespread prevalence of such problems starkly contrasts the limited development of relevant research. Motivated by this disparity, this paper conducts a comprehensive survey on SAEAs tailored to address ECOPs. This survey comprises two primary segments. The first segment synthesizes prevalent global, local, hybrid, and learning search strategies, elucidating their respective strengths and weaknesses. Subsequently, the second segment furnishes an overview of surrogate-based evaluation technologies, delving into three pivotal facets: model selection, construction, and management. The paper also discusses several potential future directions for SAEAs with a focus towards expensive combinatorial optimization.

An XGBoost-assisted evolutionary algorithm for expensive multiobjective optimization problems

Many expensive optimization problems exist in various real-world applications. However traditional evolutionary algorithms are inadequate for solving these problems directly. Surrogate-assisted evolutionary algorithm (SAEA) can effectively solve expensive optimization problems using computationally inexpensive surrogate models. However, both the Kriging and ensemble models most SAEAs adopted have limited uncertainty of prediction, especially for expensive multiobjective optimization problems (EMOPs). To enhance the optimization performance of SAEA for EMOPs, this paper proposes a new XGBoost-assisted evolutionary algorithm, calling XGBEA. Specifically, XGBoost is used as the surrogate model, and a neighborhood density selection strategy based on a mixed population and archive space (NDS-MPA) is proposed to measure the uncertainties of individuals. XGBoost helps to best fit objective functions with different fitness landscapes. NDS-MPA selects non-dominated individuals with minimal density for re-evaluation, incorporating considerations of convergence, diversity and uncertainty. Experimental results on two well-studied benchmarks demonstrated the superiority of XGBEA over seven state-of-the-art SAEAs.

Competitive Knowledge Transfer–Enhanced Surrogate-Assisted Search for Production Optimization

Summary Production optimization is a crucial component of closed-loop reservoir management, which typically aims to search for the best development scheme for maximum economic benefit. Over the decades, a large body of algorithms have been proposed to address production optimization problems, among which the surrogate-assisted evolutionary algorithm (SAEA) gained much research popularity due to its problem information-agnostic implementation and strong global search capability. However, existing production optimization methods often optimize individual tasks from scratch in an isolated manner, ignoring the available optimization experience hidden in previously optimized tasks. The incapability of transferring knowledge from possibly related tasks makes these algorithms always require a considerable number of simulation runs to obtain high-quality development schemes, which could be computationally prohibitive. To address this issue, this paper proposes a novel competitive knowledge transfer (CKT) method to leverage the knowledge from previously solved tasks toward enhanced production optimization performance. The proposed method consists of two parts: (1) similarity measurement that uses both reservoir features and optimization data for identifying the most promising previously solved task and (2) CKT that launches a competition between the development schemes of different tasks to decide whether to trigger the knowledge transfer. The efficacy of the proposed method is validated on a number of synthetic benchmark functions as well as two production optimization tasks. The experimental results demonstrate that the proposed method can significantly improve production optimization performance and achieve better optimization results when certain helpful previously optimized tasks are available.

Evolutionary multiobjective optimization assisted by scalarization function approximation for high-dimensional expensive problems

Surrogate-assisted evolutionary algorithms (SAEAs) are a promising approach for solving expensive multiobjective optimization problems, but they often cannot address high-dimensional problems. Although one common approach to overcoming this challenge is to construct reliable surrogates, their accuracy inevitably deteriorates in a high-dimensional search space. Thus, this paper presents a novel SAEA based on scalarization function approximation, which is designed to strengthen its robustness against this deterioration. The proposed algorithm constructs an approximation model for each scalarization function defined in a decomposition-based framework. Each decomposed problem is then solved using multiple independent models trained for its neighbor problems. The intent is to decrease the risk of search performance degradations caused by unreliable approximations and retain the redundancy of the surrogate-assisted search to hedge the risk of over-fitting. Furthermore, each approximation model is adapted to a promising region of its corresponding decomposed problem to reduce the complexity of model fitting given a limited number of training samples. Experimental results show that the proposed algorithm is competitive with state-of-the-art SAEAs adapted for high-dimensional problems.

Surrogate-Assisted Evolutionary Algorithm for the Calibration of Distributed Hydrological Models Based on Two-Dimensional Shallow Water Equations

Distributed hydrological models based on shallow water equations have gained popularity in recent years for the simulation of storm events, due to their robust and physically based routing of surface runoff through the whole catchment, including hill slopes and water streams. However, significant challenges arise in their calibration due to their relatively high computational cost and the extensive parameter space. This study presents a surrogate-assisted evolutionary algorithm (SA-EA) for the calibration of a distributed hydrological model based on 2D shallow water equations. A surrogate model is used to reduce the computational cost of the calibration process by creating a simulation of the solution space, while an evolutionary algorithm guides the search for suitable parameter sets within the simulated space. The proposed methodology is evaluated in four rainfall events located in the northwest of Spain: one synthetic storm and three real storms in the Mandeo River basin. The results show that the SA-EA accelerates convergence and obtains superior fit values when compared to a conventional global calibration technique, reducing the execution time by up to six times and achieving between 98% and 100% accuracy in identifying behavioral parameter sets after four generations of the SA-EA. The proposed methodology offers an efficient solution for the calibration of complex hydrological models, delivering improved computational efficiency and robust performance.

Regularity model based offspring generation in surrogate-assisted evolutionary algorithms for expensive multi-objective optimization

Evolutionary algorithms face significant challenges when it comes to solving expensive multi-objective optimization problems, which require costly evaluations. One of the most popular approaches to addressing this issue is to use surrogate models, which can replace the expensive real function evaluations with cheaper approximations. However, in many surrogate-assisted evolutionary algorithms (SAEAs), the process of offspring generation has not received sufficient attention. In this paper, we propose a novel framework for expensive multi-objective optimization called RM-SAEA, which utilizes a regularity model (RM) operator to generate offspring more effectively. The regularity model operator is combined with a general genetic algorithm operator to create a heterogeneous offspring generation module that can better approximate the Pareto front. Moreover, to overcome the data deficiency issue in expensive optimization scenarios, we employ a data augmentation strategy while training the regularity model. Finally, we embed three representative SAEAs into the proposed RM-SAEA to demonstrate its efficacy. Experimental results on several benchmark test suites with up to 10 objectives and real-world applications show that RM-SAEA achieves superior overall performance compared to eight state-of-the-art algorithms. By focusing on more effective offspring generation and addressing data deficiencies, our proposed framework is able to generate better approximations of the Pareto front and improve the optimization process in expensive multi-objective optimization scenarios.

Emulation-based adaptive differential evolution: fast and auto-tunable approach for moderately expensive optimization problems

In the field of expensive optimization, numerous papers have proposed surrogate-assisted evolutionary algorithms (SAEAs) for a few thousand or even hundreds of function evaluations. However, in reality, low-cost simulations suffice for a lot of real-world problems, in which the number of function evaluations is moderately restricted, e.g., to several thousands. In such moderately restricted scenario, SAEAs become unnecessarily time-consuming and tend to struggle with premature convergence. In addition, tuning the SAEA parameters becomes impractical under the restricted budgets of function evaluations—in some cases, inadequate configuration may degrade performance instead. In this context, this paper presents a fast and auto-tunable evolutionary algorithm for solving moderately restricted expensive optimization problems. The presented algorithm is a variant of adaptive differential evolution (DE) algorithms, and is called emulation-based adaptive DE or EBADE. The primary aim of EBADE is to emulate the principle of sample-efficient optimization, such as that in SAEAs, by adaptively tuning the DE parameter configurations. Specifically, similar to Expected Improvement-based sampling, EBADE identifies parameter configurations that may produce expected-to-improve solutions, without using function evaluations. Further, EBADE incepts a multi-population mechanism and assigns a parameter configuration to each subpopulation to estimate the effectiveness of parameter configurations with multiple samples carefully. This subpopulation-based adaptation can help improve the selection accuracy of promising parameter configurations, even when using an expected-to-improve indicator with high uncertainty, by validating with respect to multiple samples. The experimental results demonstrate that EBADE outperforms modern adaptive DEs and is highly competitive compared to SAEAs with a much shorter runtime.

A Surrogate-Assisted Evolutionary Algorithm for Seeking Multiple Solutions of Expensive Multimodal Optimization Problems

A Surrogate-Assisted Evolutionary Algorithm for Seeking Multiple Solutions of Expensive Multimodal Optimization Problems

Evolutionary Optimization with Simplified Helper Task for High-dimensional Expensive Multiobjective Problems

In recent years, surrogate-assisted evolutionary algorithms (SAEAs) have been sufficiently studied for tackling computationally expensive multiobjective optimization problems (EMOPs), as they can quickly estimate the qualities of solutions by using surrogate models to substitute for expensive evaluations. However, most existing SAEAs only show promising performance for solving EMOPs with no more than 10 dimensions, and become less efficient for tackling EMOPs with higher dimensionality. Thus, this article proposes a new SAEA with a simplified helper task for tackling high-dimensional EMOPs. In each generation, one simplified task will be generated artificially by using random dimension reduction on the target task (i.e., the target EMOPs). Then, two surrogate models are trained for the helper task and the target task, respectively. Based on the trained surrogate models, evolutionary multitasking optimization is run to solve these two tasks, so that the experiences of solving the helper task can be transferred to speed up the convergence of tackling the target task. Moreover, an effective model management strategy is designed to select new promising samples for training the surrogate models. When compared to five competitive SAEAs on four well-known benchmark suites, the experiments validate the advantages of the proposed algorithm on most test cases.

A surrogate-assisted evolutionary algorithm with knowledge transfer for expensive multimodal optimization problems

A surrogate-assisted evolutionary algorithm with knowledge transfer for expensive multimodal optimization problems

Surrogate-Assisted Evolutionary Algorithm with Model and Infill Criterion Auto-Configuration

Surrogate-Assisted Evolutionary Algorithm with Model and Infill Criterion Auto-Configuration

Surrogate and Autoencoder-Assisted Multitask Particle Swarm Optimization for High-Dimensional Expensive Multimodal Problems

In practice, some optimization problems require expensive calculation and exhibit multimodal characteristics simultaneously. These problems are called high-dimensional expensive multimodal optimization problems. When addressing such problems, existing surrogate-assisted evolutionary algorithms (SAEAs) encounter the “curse of dimensionality," which severely affects their capability to search optimal solutions. Therefore, this study proposed a surrogate and autoencoder-assisted multitask particle swarm optimization algorithm. First, an autoencoder-embedded multitask evolutionary framework was established to transform a high-dimensional multimodal optimization problem into multiple low-dimensional subproblems or subtasks. Further, a multi-level surrogate model management mechanism combining mirror learning was proposed. An appropriate local surrogate model can be rapidly generated for each modality of the problem. Moreover, a dual-mode local exploitation strategy was developed to improve the capability of swarm to exploit each subtask. The proposed algorithm was compared with seven existing SAEAs on 33 benchmark functions and the aeroengine aerodynamic design optimization problem. Experimental results revealed that the proposed algorithm can obtain multiple highly competitive optimal solutions, including global optimal solutions.

DP-FSAEA: Differential Privacy for Federated Surrogate-Assisted Evolutionary Algorithms

DP-FSAEA: Differential Privacy for Federated Surrogate-Assisted Evolutionary Algorithms

Engine Calibration With Surrogate-Assisted Bilevel Evolutionary Algorithm.

Engine calibration problems are black-box optimization problems which are evaluation costly and most of them are constrained in the objective space. In these problems, decision variables may have different impacts on objectives and constraints, which could be detected by sensitivity analysis. Most existing surrogate-assisted evolutionary algorithms do not analyze variable sensitivity, thus, useless effort may be made on some less sensitive variables. This article proposes a surrogate-assisted bilevel evolutionary algorithm to solve a real-world engine calibration problem. Principal component analysis is performed to investigate the impact of variables on constraints and to divide decision variables into lower-level and upper-level variables. The lower-level aims at optimizing lower-level variables to make candidate solutions feasible, and the upper-level focuses on adjusting upper-level variables to optimize the objective. In addition, an ordinal-regression-based surrogate is adapted to estimate the ordinal landscape of solution feasibility. Computational studies on a gasoline engine model demonstrate that our algorithm is efficient in constraint handling and also achieves a smaller fuel consumption value than other state-of-the-art calibration methods.

A Classifier-Ensemble-Based Surrogate-Assisted Evolutionary Algorithm for Distributed Data-Driven Optimization

A Classifier-Ensemble-Based Surrogate-Assisted Evolutionary Algorithm for Distributed Data-Driven Optimization

A hierarchical surrogate assisted optimization algorithm using teaching-learning-based optimization and differential evolution for high-dimensional expensive problems

Surrogate-assisted evolutionary algorithms (SAEAs) are increasingly used in solving computationally expensive optimization problems. However, when tackling high-dimensional expensive problems, a large number of exact function evaluations (FEs) need to be consumed for existing SAEAs to achieve an acceptable solution. In this paper, a hierarchical surrogate assisted optimization algorithm (HSAOA) using teaching-learning-based optimization and differential evolution is proposed for solving high-dimensional expensive problems with a relatively small number of exact FEs. To keep a balance between global exploration and local exploitation, a hierarchical surrogate framework with hybrid evolutionary algorithms is devised. In the global search phase, a radial basis function surrogate is utilized to assist the teaching-learning-based optimization in locating the promising sub-regions. In the local search phase, a novel dynamic ensemble of surrogates is proposed to assist the differential evolution in speeding up the convergence process. Eight test functions with 10 to 100 dimensions and a spatial truss design problem are employed to compare the proposed method with several state-of-the-art SAEAs. The results show that the proposed HSAOA is superior to the comparison algorithms for solving expensive optimization problems, and needs a much smaller number of exact FEs than other competing SAEAs to produce competitive or even better results for high-dimensional expensive problems.

Inverse distance weighting and radial basis function based surrogate model for high-dimensional expensive multi-objective optimization

Radial basis function (RBF) models have attracted a lot of attention in assisting evolutionary algorithms for solving computationally expensive optimization problems. However, most RBFs cannot directly provide the uncertainty information of their predictions, making it difficult to adopt principled infill sampling criteria for model management. To overcome this limitation, an inverse distance weighting (IDW) and RBF based surrogate assisted evolutionary algorithm, named IR-SAEA, is proposed to address high-dimensional expensive multi-objective optimization problems. First, an RBF-IDW model is developed, which can provide both the predicted objective values and the uncertainty of the predictions. Moreover, a modified lower confidence bound infill criterion is proposed based on the RBF-IDW for the balance of exploration and exploitation. Extensive experiments have been conducted on widely used benchmark problems with up to 100 dimensions. The empirical results have validated that the proposed algorithm is able to achieve a competitive performance compared with state-of-the-art SAEAs.

A computationally efficient procedure for calibrating model parameters of multiple specimens

Model calibration can be a very intensive and time-consuming task, especially when dealing with non-linear and large finite element models. The computational effort further increases when multiple specimens have to be calibrated. This is typical of laboratory experiments where several specimens made with the same and/or different constituent materials are tested. This paper proposes a calibration procedure aimed at reducing the computational effort of multiple specimen model calibration. The calibration procedure combines the robustness of a surrogate-assisted evolutionary algorithm with the exploitation of a database collecting the results of the previously calibrated specimens. In this research, the proposed procedure is applied to the calibration of the parameters of a cohesive crack model for fiber-reinforced concrete specimens. The benefit of the proposed procedure is shown by comparing the results with those obtained from the same calibration method but without accounting for the previous results in the calibration of a new specimen.

Radial projection-based adaptive sampling strategies for surrogate-assisted many-objective optimization

Intelligent manufacturing and industrial control systems frequently encounter expensive many-objective optimization problems (EMaOPs). Surrogate-assisted evolutionary algorithms (SAEAs) build predictive models to substitute the expensive fitness evaluation, enabling them to solve optimization more efficiently. In SAEAs, to enhance the exploration of optimization and the generalization of the surrogate model, the diversity of infill offspring and training database should be well-maintained, which is challenging in high-dimensional spaces or problems with disconnected Pareto front. This paper suggests a radial projection-based surrogate-assisted framework for solving EMaOPs. The radial projection can map the high-dimensional objective space into a 2-dimensional radial space. Based on this, a dynamic quadratic division method is proposed to enhance the diversity of solutions. Furthermore, an adaptive infill sampling criterion is introduced based on the distribution of selected convergent solutions, and a training database updating strategy is designed under the premise of maintaining its diversity and the model training efficiency. The presented framework exhibits a notable level of flexibility and adaptability as it can be effortlessly combined with other multi-objective optimization algorithms. Several experimental results on a set of expensive multi/many-objective test problems have demonstrated the superiority of the framework.

A surrogate-assisted evolutionary algorithm with clustering-based sampling for high-dimensional expensive blackbox optimization

A surrogate-assisted evolutionary algorithm with clustering-based sampling for high-dimensional expensive blackbox optimization