Surrogate-assisted Multi-objective Evolutionary Algorithm Research Articles

Surrogate-assisted multi-objective evolutionary feature selection of generation-based fixed evolution control for time series forecasting with LSTM networks

Surrogate-assisted multi-objective evolutionary algorithms are powerful techniques to solve computationally expensive multi-objective optimization problems. In this paper, we propose a direct fitness replacement method with generation-based fixed evolution control to implement a multi-objective evolutionary algorithm that uses a surrogate model for wrapper-type feature selection, where long short-term memory is established as the learning algorithm. The importance of the work and its benefits lie in the need to reduce the excessive computational time required by conventional wrapper-type feature selection methods based on multi-objective evolutionary algorithms and LSTM networks, maintaining or improving the predictive capacity of the models. We analyze the use of incremental learning to update the surrogate model, in comparison with the conventional non-incremental learning approach. We applied these methods in real-life time series forecasting of air quality, indoor temperature in a smart building and oil temperature in electricity transformers. Multi-step ahead predictions of the forecast models obtained with different meta-learners of the surrogate model were compared by using the Diebold–Mariano statistical test on a multi-criteria performance metric. The proposed method outperformed other approaches for feature selection including, among others, methods based on surrogate-assisted multi-objective evolutionary algorithms developed by the authors in previous research, other surrogate-assisted deterministic methods for feature selection and the conventional wrapper-type feature selection method based on LSTM, improving the prediction on test dataset by 23.98%, 34.61% and 13.77%, respectively.

A noise-resistant infill sampling criterion in surrogate-assisted multi-objective evolutionary algorithms

Most existing surrogate-assisted multi-objective evolutionary algorithms are susceptible to the noise, since the noise affects both the approximation performance of objective surrogate models and the selection accuracy. Therefore, the keys to these algorithms are how to reduce the noise impact without additional function evaluations burden and how to maximize the noise immunity of the algorithms. In this work, a noise-resistant surrogate-assisted multi-objective evolutionary algorithm is proposed to solve the noisy expensive optimization problems. In the proposed algorithm, a novel classification-based noise handling method is used to reduce the noise impact without additional function evaluations burden before optimization. In the optimization process, the denoised data and model selection strategy are used to construct appropriate objective surrogate models to assist in generating promising candidates. Then, a noise-resistant infill sampling criterion considers convergence, diversity, and model uncertainty to select the most potential individual from candidates for re-evaluation. The experimental results on a series of expensive test problems with additive noise have demonstrated the competitiveness of the proposed algorithm against the other comparative algorithms.

Surrogate-Assisted Multi-Objective Evolutionary Optimization With Pareto Front Model-Based Local Search Method.

Some local search methods have been incorporated into surrogate-assisted multi-objective evolutionary algorithms to accelerate the search toward the real Pareto front (PF). In this article, a PF model-based local search method is proposed to accelerate the exploration and exploitation of the PF. It first builds a predicted PF model with current nondominated solutions. Then, some sparse points in the predicted PF are selected to guide the search directions of the local search in order to promote the search of promising sparse areas. The approximation degree of the predicted and real PFs will influence the speed of the local search, while extreme points can significantly influence the shape of the PF. To accelerate the search progress, the optima of surrogate models are utilized to promote the progress of finding extreme points. The proposed local search method is incorporated into a surrogate-assisted multi-objective evolutionary algorithm. The proposed surrogate-assisted multi-objective evolutionary algorithm with the proposed local search method is tested with Zitzler-Deb-Thiele (ZDT), Deb-Thiele-Laummans-Zitzler (DTLZ), and MAF instances. The experimental results demonstrated the efficiency of the proposed local search method and the superiority of the proposed algorithm.

Multi-objective constrained black-box optimization algorithm based on feasible region localization and performance-improvement exploration

Over the past decade, surrogate-assisted evolutionary algorithms have demonstrated their effectiveness across various computationally expensive real-world domains. Nevertheless, the focus of surrogate-assisted multi-objective evolutionary algorithms has primarily centered on non-constrained optimization problems. There has been relatively limited exploration into addressing expensive constrained multi-objective optimization problems, which inherently require a delicate equilibrium between convergence, diversity, and feasibility. To bridge this gap, this paper concentrates on constrained multi-objective optimization problems where both objectives and constraints involve substantial computational costs. In response, a novel data-driven constrained multi-objective evolutionary algorithm is introduced, leveraging feasible region localization and performance-improvement exploration. For feasible region localization, a constraints-domain-search strategy is presented to locate the feasible region quickly. To enhance performance-improvement exploration, a progressive enhancement of convergence and diversity is achieved through the incorporation of constraint penalties. With the help of exploration and exploitation, the proposed algorithm can balance convergence, diversity, and feasibility while working within a limited number of function evaluations. By comparing the proposed algorithm with state-of-the-art algorithms on 66 mathematical problems and a resource-intensive black-box problem, its outstanding performance for solving multi-objective constrained black-box problems is validated.

Surrogate-assisted operator-repeated evolutionary algorithm for computationally expensive multi-objective problems

Surrogate-assisted strategies work differently in surrogate-assisted multi-objective evolutionary algorithms. How to select suitable surrogate-assisted strategies to balance global and local search and diversity of Pareto optimal solutions is usually difficult for efficient optimization of computationally expensive multi-objective problems, which only allow limited time costs in the optimization process. Therefore, to further improve the optimization efficiency for expensive multi-objective problems, a surrogate-assisted operator-repeated multi-objective evolutionary algorithm is proposed by reasonably integrating several surrogate-assisted strategies in this study. Specifically, the proposed algorithm is based on a operator-repeated offspring creation strategy, which can produce many diverse candidate offspring individuals and thus make the proposed algorithm search globally and efficiently. In addition, a novel infill criterion termed as reference-vector-guided surrogate-assisted penalty-based boundary intersection is proposed and combined with a common Kriging-based expected improvement matrix infill criterion for complementary prescreening over the candidate offspring individuals. These two infill criteria can make a good balance between global search and diversity of Pareto optimal solutions. In addition, to fasten convergence speed, an improved surrogate-based multi-objective local search method with minimum distance-angle sampling is also proposed and embedded in the proposed algorithm. Several benchmark problems with dimensions varying from 8 to 50 and a practical two-objective airfoil design optimization problem with 14 design variables are tested to validate efficiency of the proposed algorithm. The experimental results demonstrate that the proposed algorithm significantly outperforms some state-of-the-art surrogate-assisted multi-objective optimization algorithms on most of problems.

Complementary surrogate-assisted differential evolution algorithm for expensive multi-objective problems under a limited computational budget

Different surrogate-assisted strategies can greatly influence the optimization efficiency of surrogate-assisted multi-objective evolutionary algorithms. By hybridizing two complementary surrogate-assisted strategies, this study proposed an efficient surrogate-assisted differential evolution algorithm to optimize expensive multi-objective problems under a limited computational budget. The two proposed surrogate-assisted strategies balance global and local search for multi-objective optimization. Specifically, one strategy is an improved surrogate-based multi-objective local search method that is based on maximin angle-distance sequential sampling. Compared with the previous local search method that is based on Euclidian distance-based sampling, the improved local search method is more efficient because it can mitigate the scale difference of different objectives. The other surrogate-assisted strategy is prescreening based on a diversity-enhanced expected improvement matrix infill criterion. The proposed infill criterion aims to improve the diversity of approximate Pareto optimal solutions by considering distribution of candidate individuals in the objective space in the infill function. Within a limited computational burden, the performance of the proposed algorithm is demonstrated on a large set of multi-objective benchmark problems, as well as a real-world airfoil design problem. Experimental results show that the proposed algorithm performs significantly better than some existing algorithms on most problems investigated in this study.

Bi-indicator driven surrogate-assisted multi-objective evolutionary algorithms for computationally expensive problems

This paper presents a bi-indicator-based surrogate-assisted evolutionary algorithm (BISAEA) for multi-objective optimization problems (MOPs) with computationally expensive objectives. In BISAEA, a Pareto-based bi-indictor strategy is proposed based on convergence and diversity indicators, where a nondominated sorting approach is adopted to carry out two-objective optimization (convergence and diversity indicators) problems. The radius-based function (RBF) models are used to approximate the objective values. In addition, the proposed algorithm adopts a one-by-one selection strategy to obtain promising samples from new samples for evaluating the true objectives by their angles and Pareto dominance relationship with real non-dominated solutions to improve the diversity. After the comparison with four state-of-the-art surrogate-assisted evolutionary algorithms and three evolutionary algorithms on 76 widely used benchmark problems, BISAEA shows high efficiency and a good balance between convergence and diversity. Finally, BISAEA is applied to the multidisciplinary optimization of blend-wing-body underwater gliders with 30 decision variables and three objectives, and the results demonstrate that BISAEA has superior performance on computationally expensive engineering problems.

A performance approximation assisted expensive many-objective evolutionary algorithm

Surrogate-assisted multi-objective evolutionary algorithms have been paid much attention to solve expensive multi-objective problems in recent years. However, with the number of objectives increasing, an improper solution may be picked for expensive objective evaluation due to the accumulation error of approximated values on objective functions. Furthermore, the time to construct surrogate models for all objectives will significantly increase. Thus, in this paper, Gaussian process (GP) models are proposed for performance indicators instead of for objective functions. Furthermore, solutions are selected from either of two ways to be evaluated using the expensive objective function. When there are non-dominated solutions found so far that are approximated, they will be exactly evaluated using the objective function. Otherwise, the solution with the maximum approximation uncertainty among the current population will be evaluated using the real objective functions. The efficiency of the presented approach is validated on the DTLZ test suite with 3, 6, 10, 15, and 20 objectives, MaF benchmark problems with 3, 6, 10, 15, and 20 objectives, and a real-world optimization problem called filter design. The experimental results show that the method proposed in this paper is competitive compared to recently proposed peer algorithms for expensive many-objective problems.

Adaptive surrogate-assisted multi-objective evolutionary algorithm using an efficient infill technique

Surrogate-assisted multi-objective evolutionary algorithms have become increasingly popular for solving computationally expensive problems, profiting from surrogate modeling and infill approaches to reduce the time cost of optimization. Most existing algorithms have specified the type of surrogate model before a run and keep the type static during the optimization process. However, a sole surrogate model may not consistently perform well for all problems without any prior knowledge. In this context, this paper proposes an adaptive technique for surrogate models with multiple radial basis functions (RBFs), as the technique can dynamically establish the most promising RBF for each objective, thereby enhancing the reliability of surrogate prediction. Moreover, multi-objective evolutionary algorithms (MOEAs) that are employed as optimizers for infilling criteria can highly affect the search behavior of a surrogate-assisted evolutionary algorithm. The proposed infill technique develops a crowding distance-based prescreening operator to embed various MOEAs. Two techniques collaboratively promote the convergence, coverage, and diversity of the predicted Pareto front. Representative benchmark problems and a structural optimization problem are given to show the effectiveness of the algorithm that employs these techniques. Empirical experiments demonstrate that the proposed algorithm significantly outperforms other state-of-the-art algorithms in most cases.

Multiple Classifiers-Assisted Evolutionary Algorithm Based on Decomposition for High-Dimensional Multiobjective Problems

Surrogate-assisted multiobjective evolutionary algorithms (MOEAs) have advanced the field of computationally expensive optimization, but their progress is often restricted to low-dimensional problems. This manuscript presents a multiple classifiers-assisted evolutionary algorithm based on decomposition, which is adapted for high-dimensional expensive problems in terms of the following two insights. Compared to approximation-based surrogates, the accuracy of classification-based surrogates is robust for few high-dimensional training samples. Furthermore, multiple local classifiers can hedge the risk of overfitting issues. Accordingly, the proposed algorithm builds multiple classifiers with support vector machines (SVMs) on a decomposition-based multiobjective algorithm, wherein each local classifier is trained for a corresponding scalarization function. Experimental results confirm that the proposed algorithm is competitive to the state-of-the-art algorithms and computationally efficient as well.

A classification surrogate-assisted multi-objective evolutionary algorithm for expensive optimization

Surrogate-assisted multi-objective evolutionary algorithms (SAMOEAs) have been developed for solving expensive optimization problems. According to the roles that the surrogate models play in SAMOEAs, they can be divided into two categories: prediction-based and classification-based algorithms. Though prediction-based SAMOEAs are the mainstream methods, classification-based ones are gaining their fast developments. In this article, a classification surrogate-assisted multi-objective evolutionary algorithm (CSA-MOEA) is proposed for expensive optimization. The algorithm adopts a classification tree as the surrogate model to predict promising offsprings, which may be non-dominated solutions with good convergence. Then based on two effective infilling strategies, some of these promising individuals are added to the sample archive. By repeating the above steps iteratively, valuable solutions can be obtained. To evaluate the performance of CSA-MOEA, it is compared with several state-of-the-art surrogate-assisted evolutionary algorithms on three sets of multi-objective optimization test problems and an engineering shape optimization problem. The experimental results demonstrate the competitiveness of CSA-MOEA.

A novel self-adaptive multi-fidelity surrogate-assisted multi-objective evolutionary algorithm for simulation-based production optimization

Multi-objective production optimization (MOPO) which aims at finding an optimal well control scheme is regarded as a win-win strategy for improving economic and production gains. Generally, the MOPO processes involve computationally expensive numerical simulations and high-dimensional decision variables which limit their application to actual reservoir optimization problems. Surrogate-assisted multi-objective optimization method which uses a simple yet efficient approximation model shows huge potential in solving production optimization problems since it lessens the computational burden by decreasing the use of computationally expensive numerical simulations. In this paper, a novel self-adaptive multi-fidelity surrogate-assisted (SAMFS) multi-objective production optimization algorithm (SAMFS-MOPO) is proposed to reduce the computational burden and enhance the accuracy of the surrogate model. A similar method has been applied to well spacing optimization but the uniqueness of this method is that two fidelity samples are used to establish a multi-fidelity (MF) surrogate model, while i-updating and g-updating strategies are used to renew the MF surrogate model during the optimization process so as to improve its accuracy and reduce the computational burden. To the best of our knowledge, this is the first time a self-adaptive multi-fidelity surrogate model is used for production optimization. Furthermore, three classic multi-objective benchmark problems and two reservoirs with different complexities were applied to illustrate the effectiveness and accuracy of the proposed SAMFS-MOPO method. It was found that the SAMFS-MOPO method had superior performance in convergence, diversity, and efficiency than other conventional methods.

A Bi- Indicator -Based Surrogate-Assisted Multi-Objective Evolutionary Algorithm For Computationally Expensive Problems

A Bi- Indicator -Based Surrogate-Assisted Multi-Objective Evolutionary Algorithm For Computationally Expensive Problems

A federated data-driven evolutionary algorithm for expensive multi-/many-objective optimization

Data-driven optimization has found many successful applications in the real world and received increased attention in the field of evolutionary optimization. Most existing algorithms assume that the data used for optimization are always available on a central server for construction of surrogates. This assumption, however, may fail to hold when the data must be collected in a distributed way and are subject to privacy restrictions. This paper aims to propose a federated data-driven evolutionary multi-/many-objective optimization algorithm. To this end, we leverage federated learning for surrogate construction so that multiple clients collaboratively train a radial-basis-function-network as the global surrogate. Then a new federated acquisition function is proposed for the central server to approximate the objective values using the global surrogate and estimate the uncertainty level of the approximated objective values based on the local models. The performance of the proposed algorithm is verified on a series of multi-/many-objective benchmark problems by comparing it with two state-of-the-art surrogate-assisted multi-objective evolutionary algorithms.

A Guidance method for robustness surrogate assisted multi-objective evolutionary algorithms

In the real world, multi-objective problems(MOPs) are relatively common in optimization in the areasof design, planning, decision support... In fact, problemsinclude two or many objectives, there is a class of problemscalled expensive problems that are problems with complexmathematical models, large computational costs,... Theycan not be solved by normal techniques, they are usually tobe solved with techniques such as simulation, decomposing,problem transformation. In particular, using a surrogatemodel with Kriging, neuron networks techniques in combination with an evolutionary algorithm is a subtle choice,with many positive results, being studied and applied inpractice. However, the use of a surrogate model withKriging, neuron networks combining selection strategy,sampling... can reduce the robustness of the algorithmsduring the search. This paper analyzes the issues affectingthe robustness of the multi-objective evolutionary algorithms (MOEAs) using surrogate models and suggests theuse of a guidance technique to increase the robustness ofthe algorithm, through analysis, experiment and results arecompetitive and effective to improve the quality of MOEAsusing a surrogate model to solve expensive problems.

Neighborhood samples and surrogate assisted multi-objective evolutionary algorithm for expensive many-objective optimization problems

Many surrogate-assisted meta-heuristic algorithms have been proposed for single-objective expensive optimization problems, however, not so much attention has been paid to multi-objective expensive problems, especially for those with more than four objectives. In this paper, we use reference vector guided evolutionary algorithm (RVEA) to select suitable individuals, and radial basis function (RBF) networks are used to estimate the fitness of the original objective function to reduce the computational cost. These suitable individuals are optimized by the RBF network for several iterations. Then in surrogate model management, an infill strategy is proposed to select promising individuals for exact evaluations. Euclidean distance to origin or uncertainty is adaptively considered, according to the convergence degree of the current population in the infill strategy. The approximation uncertainty of each solution is calculated according to its distance to the modeling samples in the decision space. The experimental results on a number of many-objective optimization problems showed that the proposed method is competitive to three state-of-the-art algorithms for solving computationally expensive many-objective optimization problems.

Comparison of synchronous and asynchronous parallelization of extreme surrogate-assisted multi-objective evolutionary algorithm

This paper investigates the integration of a surrogate-assisted multi-objective evolutionary algorithm (MOEA) and a parallel computation scheme to reduce the computing time until obtaining the optimal solutions in evolutionary algorithms (EAs). A surrogate-assisted MOEA solves multi-objective optimization problems while estimating the evaluation of solutions with a surrogate function. A surrogate function is produced by a machine learning model. This paper uses an extreme learning surrogate-assisted MOEA/D (ELMOEA/D), which utilizes one of the well-known MOEA algorithms, MOEA/D, and a machine learning technique, extreme learning machine (ELM). A parallelization of MOEA, on the other hand, evaluates solutions in parallel on multiple computing nodes to accelerate the optimization process. We consider a synchronous and an asynchronous parallel MOEA as a master-slave parallelization scheme for ELMOEA/D. We carry out an experiment with multi-objective optimization problems to compare the synchronous parallel ELMOEA/D with the asynchronous parallel ELMOEA/D. In the experiment, we simulate two settings of the evaluation time of solutions. One determines the evaluation time of solutions by the normal distribution with different variances. On the other hand, another evaluation time correlates to the objective function value. We compare the quality of solutions obtained by the parallel ELMOEA/D variants within a particular computing time. The experimental results show that the parallelization of ELMOEA/D significantly reduces the computational time. In addition, the integration of ELMOEA/D with the asynchronous parallelization scheme obtains higher quality of solutions quicker than the synchronous parallel ELMOEA/D.

Two infill criteria driven surrogate-assisted multi-objective evolutionary algorithms for computationally expensive problems with medium dimensions

Abstract This paper proposed a surrogate-assisted dominance-based multi-objective evolutionary algorithm to solve multi-objective computationally expensive problems with medium dimensions. Two infill criteria are collaboratively used to select promising individuals for exact evaluations. The convergence-based criterion is used to promote the exploitation of current promising areas. This criterion also considers the dispersion of selected solutions to exploit current non-dominant front. The diversity-based criterion is used to enhance the exploration of the population and enhance the accuracy of surrogate models. The feedback information from the convergence-based criterion is used to adjust the frequency of using the diversity-based criterion in order to reduce the consumed function evaluations. Benchmark functions with dimensions varying from 8 to 30 and a reactive power optimization problem are used to test the proposed algorithm. The experimental results demonstrate that the proposed algorithm significantly outperforms some state-of-the-art evolutionary algorithms on most problems.

A Classification-Based Surrogate-Assisted Multiobjective Evolutionary Algorithm for Production Optimization under Geological Uncertainty

Summary Multiobjective optimization (MOO) is a popular procedure for waterflooding optimization under geological uncertainty that maximizes the expectation of net present value (NPV) over all possible uncertainty models and minimizes the variance simultaneously. However, the optimization process involves a large number of decision variables, and the problem is computationally expensive. Surrogate-assisted evolutionary algorithms (SAEAs), which have proved to be an effective way to solve expensive problems, design computationally inexpensive functions to approximate each objective function. On the basis of characterization, we have designed an efficient multiobjective evolutionary algorithm (MOEA) to effectively deal with computationally expensive simulation-based optimization problems. The uniqueness of this algorithm is that it incorporates a Pareto-rank-learning scheme with surrogate-assisted infill criterion. The first is to introduce a multiclass error-correcting output codes (ECOC) model that directly predicts the dominance relationship between candidate solutions and prescreens, and the second is to train a radial-basis function (RBF) network that predicts the objective functions of prescreened solutions to calculate the hypervolume (HV) improvement that maintains convergence and diversity. Compared with typical surrogate-based methods, the developed method provides a classifier first that can enhance the accuracy in high dimensions and reduce computational complexity. To the best of our knowledge, the proposed method compares with state-of-the-art surrogate frameworks for multiobjective production-optimization problems. In this paper, the proposed approach is applied to two 200D benchmark problems and two synthetic reservoir models. The results show that the proposed method can achieve more comprehensive and efficient reservoir management (RM) with a higher convergence speed compared with traditional MOEAs and surrogate-assisted optimization methods.

A surrogate-assisted multi-objective evolutionary algorithm with dimension-reduction for production optimization

Multi-objective optimization (MOO), which involves more than one conflicting objective to be optimized simultaneously, is expected to provide efficient and comprehensive reservoir management (RM) solutions. The multi-objective production optimization problems are considered to be expensive due to the difficulties and cost of operations. Surrogate-assisted evolutionary algorithms (SAEAs), which has proved to be an effective way to solve expensive problems, design computationally cheap function to approximate each objective function. Meanwhile, the optimization process involves a large number of decision variables. However, building a high-quality surrogate model has become difficult due to the “curse of dimensionality”. Base on characterization, an efficient multi-objective optimization framework called SA-RVEA-PCA is proposed to effectively deal with large-scale and computationally expensive simulation-based optimization problems, including three parts:1) Given a set of simulation results, a Gaussian process (GP) model based on Principal Component Analysis (PCA) for each objective function is trained so that the surrogate models can guide the optimization more accurately. 2) A reference vector guided evolutionary algorithm (RVEA) recently developed is employed as a multi-objective optimizer. 3) The information of uncertainty given by GP and angle-penalized penalized (APD) proposed in RVEA are used to update the surrogate models. To the best of our knowledge, the proposed algorithm is applied to a benchmark function, and two typical applications of MOO with synthetic reservoir models. Results show that the proposed method can provide more comprehensive and efficient RM with a higher convergence speed.