Surrogate-assisted Evolutionary Optimization Research Articles

Enhancing surrogate-assisted evolutionary optimization for medium-scale expensive problems: a two-stage approach with unsupervised feature learning and Q-learning

Enhancing surrogate-assisted evolutionary optimization for medium-scale expensive problems: a two-stage approach with unsupervised feature learning and Q-learning

Surrogate-Assisted Evolutionary Multi-Objective Optimization of Medium-Scale Problems by Random Grouping and Sparse Gaussian Modeling

Surrogate-Assisted Evolutionary Multi-Objective Optimization of Medium-Scale Problems by Random Grouping and Sparse Gaussian Modeling

An integrated surrogate model-driven and improved termite life cycle optimizer for damage identification in dams

Structural Damage Identification (SDI) is a crucial branch in the field of structural health monitoring, providing an essential support for the safe operation of structures. In this paper, a novel structural damage identification method based on a surrogate-assisted evolutionary optimization algorithm is proposed. This method incorporates cost-effective surrogate modelling techniques and swarm intelligence optimization algorithms with powerful global optimization capabilities. Three popular surrogate models are fused based on a weighted average strategy to construct an integrated surrogate model with stronger generalization ability and higher accuracy. In addition, the self-designed movement strategy is more suitable for the characteristics of Termite Life Cycle Optimizer (TLCO), allowing the improved TLCO to have stronger robustness and the capability to escape from local optimum. The effectiveness of the proposed method for solving the SDI problem is confirmed in a damaged dam model with different complexities. Some important findings are as follows: (i) Compared with the conventional SDI methods, which directly combines optimization algorithms and finite element models, the proposed method maintains reliable accuracy, while improving computational efficiency by a factor of more than 100. (ii) There is no direct relationship between the accuracy of the surrogate models and the effectiveness of the SDI method based on their combination with ITLCO. The proposed method can serve as a promising tool for damage identification in large-scale structures.

Surrogate-assisted evolutionary optimization for perishable inventory management in multi-echelon distribution systems

Simulation is widely used for analyzing supply chains with complex structures and stochastic nature. However, optimizing supply chain simulation models is usually computationally expensive. This study proposes a surrogate-assisted evolutionary optimization approach to optimize the inventory policies in multi-echelon distribution systems for perishable items under a limited number of evaluations. The random forest algorithm is used to build the surrogate model for a faster estimation of the performance of the inventory policies. A co-evolutionary differential evolution algorithm is proposed to simultaneously evolve the population through multiple searching strategies. The generated solutions are estimated by the low-cost surrogate model to select the promising solutions, which will be evaluated by the inventory model. Moreover, this study also integrates the surrogate model into two classic metaheuristic algorithms, particle swarm optimization and differential evolution. Also, a new performance indicator is proposed to examine the efficiency of the surrogate model in evolutionary computation. Two case studies are used to investigate the performance of the proposed algorithms. The experimental results show that both particle swarm optimization and differential evolution exhibit performance improvements exceeding 55% by using surrogate models under the limited number of function evaluations. Furthermore, the surrogate model reduces computational time for both algorithms by over 34% to achieve equivalent objective values. Finally, the proposed co-evolutionary differential evolution algorithm is compared with 12 algorithms, and the results show that the proposed algorithm consistently outperforms them. These findings confirm the usefulness of the surrogate model in evolutionary algorithms and the effectiveness of the proposed co-evolutionary strategy for solving perishable inventory problems.

Surrogate-Assisted Evolutionary Optimization of CO2-ESGR and Storage

Surrogate-Assisted Evolutionary Optimization of CO₂-ESGR and Storage

Surrogate-assisted evolutionary sampling particle swarm optimization for high-dimensional expensive optimization

Surrogate-assisted evolutionary sampling particle swarm optimization for high-dimensional expensive optimization

An Uncertainty Measure for Prediction of Non-Gaussian Process Surrogates.

Model management is an essential component in data-driven surrogate-assisted evolutionary optimization. In model management, the solutions with a large degree of uncertainty in approximation play an important role. They can strengthen the exploration ability of algorithms and improve the accuracy of surrogates. However, there is no a theoretical method to measure the uncertainty of prediction of Non-Gaussian process surrogates. To address this issue, this paper proposes a method to measure the uncertainty. In this method, a stationary random field with a known zero mean is used to measure the uncertainty of prediction of Non-Gaussian process surrogates. Based on experimental analyses, this method is able to measure the uncertainty of prediction of Non-Gaussian process surrogates. The method's effectiveness is demonstrated on a set of benchmark problems in single surrogate and ensemble surrogates cases.

An enhanced surrogate-assisted differential evolution for constrained optimization problems

The application of evolutionary algorithms (EAs) to complex engineering optimization problems may present difficulties as they require many evaluations of the objective functions by computationally expensive simulation procedures. To deal with this issue, surrogate models have been employed to replace those expensive simulations. In this work, a surrogate-assisted evolutionary optimization procedure is proposed. The procedure combines the differential evolution method with a $$k$$ -nearest neighbors (k-NN) similarity-based surrogate model. In this approach, the database that stores the solutions evaluated by the exact model, which are used to approximate new solutions, is managed according to a merit scheme. Constraints are handled by a rank-based technique that builds multiple separate queues based on the values of the objective function and the violation of each constraint. Also, to avoid premature convergence of the method, a strategy that triggers a random reinitialization of the population is considered. The performance of the proposed method is assessed by numerical experiments using 24 constrained benchmark functions and 5 mechanical engineering problems. The results show that the method achieves optimal solutions with a remarkably reduction in the number of function evaluations compared to the literature.

An Analysis of the RBF Hyperparameter Impact on Surrogate-Assisted Evolutionary Optimization

Computationally expensive optimization problems are often solved using surrogates and a common variant is the radial basis functions (RBF) model. It aggregates several basis functions which all depend on a hyperparameter affecting their individual outputs and consequentially the overall surrogate prediction. However, the optimal value of the hyperparameter is typically unknown and should therefore be calibrated. This raises the question how does the hyperparameter affect the overall optimization search effectiveness (and not just the stand-alone surrogate accuracy) and to what extent is such a calibration beneficial, which is an important consideration both for end-users and algorithm researchers alike. To rigorously address this issue this paper presents an analysis based on an extensive set of numerical experiments with an RBF surrogate-assisted evolutionary algorithm. It follows that the hyperparameter strongly affected performance and that the extent of its impact varied depending on the basis function, objective function modality, and problem dimension. Overall, calibration of the hyperparameter was typically highly beneficial to the search performance while dynamically optimizing the hyperparameter during the search yielded additional performance gains.

Evolutionary Optimization of High-Dimensional Multiobjective and Many-Objective Expensive Problems Assisted by a Dropout Neural Network

Gaussian processes (GPs) are widely used in surrogate-assisted evolutionary optimization of expensive problems mainly due to the ability to provide a confidence level of their outputs, making it possible to adopt principled surrogate management methods, such as the acquisition function used in the Bayesian optimization. Unfortunately, GPs become less practical for high-dimensional multiobjective and many-objective optimization as their computational complexity is cubic in the number of training samples. In this article, we propose a computationally efficient dropout neural network (EDN) to replace the Gaussian process and a new model management strategy to achieve a good balance between convergence and diversity for assisting evolutionary algorithms to solve high-dimensional multiobjective and many-objective expensive optimization problems. While the conventional dropout neural network needs to save a large number of network models during the training for calculating the confidence level, only one single network model is needed in the EDN to estimate the fitness and its confidence level by randomly ignoring neurons in both training and testing the neural network. Extensive experimental studies on benchmark problems with up to 100 decision variables and 20 objectives demonstrate that, compared to state of the art, the proposed algorithm is not only highly competitive in performance but also computationally more scalable to high-dimensional many-objective optimization problems. Finally, the proposed algorithm is validated on an operational optimization problem of crude oil distillation units, further confirming its capability of handling expensive problems given a limited computational budget.

Surrogate-assisted evolutionary optimization of expensive many-objective irregular problems

Surrogate-assisted evolutionary algorithms are one effective approach to handling expensive problems and have attracted increasing attention over the past decades. However, existing surrogate-assisted evolutionary algorithms pay little attention to expensive many-objective problems with irregular Pareto fronts, also called irregular problems. In this study, we propose a surrogate-assisted evolutionary algorithm for dealing with expensive irregular problems, where only a small number of expensive fitness evaluations is allowed. In the proposed algorithm, the reference vectors are adapted based on both the individuals in the current population and the non-dominated solutions that have been evaluated using the real objective functions. A surrogate management strategy is then designed to balance convergence and diversity according to the adaptive reference vectors as well as the non-dominated solutions that have been evaluated using the expensive objective functions so that the irregularity of the Pareto front can be taken into account. To reduce the computational cost for updating the Gaussian process based surrogates, a subset of training data near the adaptive reference vectors are prioritized. Experimental results on the DTLZ, WFG, DPF and MaF test suites demonstrate that the proposed algorithm is able to solve expensive many-objective optimization problems with both irregular and regular Pareto fronts. The proposed algorithm is also tested on a real-world application example to further confirm its effectiveness and competitiveness.

Data-Driven Surrogate-Assisted Evolutionary Optimization of Combined Economic/Emission Dispatch in Large-Scale Power Systems

Data-Driven Surrogate-Assisted Evolutionary Optimization of Combined Economic/Emission Dispatch in Large-Scale Power Systems

A surrogate-assisted optimization framework for microclimate-sensitive urban design practice

Simulations can often benefit microclimate-sensitive urban design by offering insightful abstractions of stochastic urban system behaviors, yet many of them are difficult to use and generally consume significant computational resources. We adapt an advanced surrogate-assisted evolutionary optimization algorithm instead of other empirical multi-objective evolutionary algorithms commonly used to search for optimal design alternatives to confront the challenge. Moreover, a parametric design module is hybridized with this surrogate-assisted evolutionary optimization algorithm to create a working scheme for mathematically modeling microclimate-sensitive urban design problems. This seven-step scheme is tested using a hypothetical case, a spatial planning problem in a residential block, to search for design proposals that maximize project development profits and facilitate the needed creation of a comfortable wind environment. Moreover, by utilizing three optimization solvers, we obtained a near-optimal site plan with a wind velocity ratio of 0.36, a wind velocity Gini index of 0.31, and a gross profit of 4.05 × 108 RMB. Also, the case study results show that the proposed optimization framework, which consists of a global surrogate (additive Gaussian process model) and a local surrogate (gradient boosted regression trees model), converges faster and provides better optimal solutions to a high-dimensional design problem compared to the algorithm which only uses a single surrogate. Built with a flexible structure, we believe the proposed framework can address the emerging demands for a wide range of microclimate-sensitive design tasks, especially those with costly simulations and small experimental datasets.

Test Data Generation for Path Coverage of MPI Programs Using SAEO

Message-passing interface (MPI) programs, a typical kind of parallel programs, have been commonly used in various applications. However, it generally takes exhaustive computation to run these programs when generating test data to test them. In this article, we propose a method of test data generation for path coverage of MPI programs using surrogate-assisted evolutionary optimization, which can efficiently generate test data with high quality. We first divide a sample set of a program into a number of clusters according to the multi-mode characteristic of the coverage problem, with each cluster training a surrogate model. Then, we estimate the fitness of each individual using one or more surrogate models when generating test data through evolving a population. Finally, a small number of representative individuals are selected to execute the program, with the purpose of obtaining their real fitness, to guide the subsequent evolution of the population. We apply the proposed method to seven benchmark MPI programs and compare it with several state-of-the-art approaches. The experimental results show that the proposed method can generate test data with reduced computation, thus improving the testing efficiency.

Truncation-learning-driven surrogate assisted social learning particle swarm optimization for computationally expensive problem

Surrogate-assisted evolutionary optimization greatly slashes the computational burden of evolutionary algorithms for computationally expensive problems. However, new issues arise concerning the compatibility and fault tolerance of surrogates, evolutionary learning operators, and problem property. To this end, this paper proposes a truncation-learning-driven surrogate assisted social learning particle swarm optimizer (TL-SSLPSO) to coordinate these three ingredients. For avoiding and correcting the deceptions induced by the low confidence exemplars due to the surrogate in behavior learning, TL-SSLPSO equally segments the iterative population into multiple sub-populations with different fitness levels and selects exemplars from the randomly selected high-level sub-populations for the behavior learning of low-level sub-population, while truncating the behavior learning of the highest-level sub-population composed of some of the best approximated or real evaluated particles and retaining the sub-population directly to the next generation. Besides, a greedy sampling strategy is employed to find promising solutions with better fitness versus the global best to complement the truncation learning. Extensive experiments on twenty-four widely used benchmark problems and a stepped cantilever beam design problem with 17 steps are conducted to assess the effectiveness of cooperation between truncation learning and greedy sampling and comparisons with several state-of-the-art algorithms demonstrate the superiority of the proposed method.

Evolution Control for parallel ANN-assisted simulation-based optimization application to Tuberculosis Transmission Control

In many optimal design searches, the function to optimize is a simulator that is computationally expensive. While current High Performance Computing (HPC) methods are not able to solve such problems efficiently, parallelism can be coupled with approximate models (surrogates or meta-models) that imitate the simulator in timely fashion to achieve better results. This combined approach reduces the number of simulations thanks to surrogate use whereas the remaining evaluations are handled by supercomputers. While the surrogates’ ability to limit computational times is very attractive, integrating them into the overarching optimization process can be challenging. Indeed, it is critical to address the major trade-off between the quality (precision) and the efficiency (execution time) of the resolution. In this article, we investigate Evolution Controls (ECs) which are strategies that define the alternation between the simulator and the surrogate within the optimization process. We propose a new EC based on the prediction uncertainty obtained from Monte Carlo Dropout (MCDropout), a technique originally dedicated to quantifying uncertainty in deep learning. Investigations of such uncertainty-aware ECs remain uncommon in surrogate-assisted evolutionary optimization. In addition, we use parallel computing in a complementary way to address the high computational burden. Our new strategy is implemented in the context of a pioneering application to Tuberculosis Transmission Control. The reported results show that the MCDropout-based EC coupled with massively parallel computing outperforms strategies previously proposed in the field of surrogate-assisted optimization.

An adaptive Bayesian approach to surrogate-assisted evolutionary multi-objective optimization

Surrogate models have been widely used for solving computationally expensive multi-objective optimization problems (MOPs). The efficient global optimization (EGO) algorithm, a Bayesian approach to surrogate-assisted optimization, has become very popular in surrogate-assisted evolutionary optimization. In this paper, we propose an adaptive Bayesian approach to surrogate-assisted evolutionary algorithm to solve expensive MOPs. The main idea is to tune the hyperparameter in the acquisition function according to the search dynamics to determine which candidate solutions are to be evaluated using the expensive real objective functions. In addition, the sampling selection criterion switches between an angle based distance and an angle-penalized distance over the course of optimization to achieve a better balance between exploration and exploitation. The performance of the proposed algorithm is examined on a set of benchmark problems and an airfoil design optimization problem using a maximum of 300 real fitness evaluations. Our experimental results show that the proposed algorithm is competitive compared to four popular multi-objective evolutionary algorithms.

Multiobjective Infill Criterion Driven Gaussian Process-Assisted Particle Swarm Optimization of High-Dimensional Expensive Problems

Model management plays an essential role in surrogate-assisted evolutionary optimization of expensive problems, since the strategy for selecting individuals for fitness evaluation using the real objective function has substantial influences on the final performance. Among many others, infill criterion driven Gaussian process (GP)-assisted evolutionary algorithms have been demonstrated competitive for optimization of problems with up to 50 decision variables. In this paper, a multiobjective infill criterion (MIC) that considers the approximated fitness and the approximation uncertainty as two objectives is proposed for a GP-assisted social learning particle swarm optimization algorithm. The MIC uses nondominated sorting for model management, thereby avoiding combining the approximated fitness and the approximation uncertainty into a scalar function, which is shown to be particularly important for high-dimensional problems, where the estimated uncertainty becomes less reliable. Empirical studies on 50-D and 100-D benchmark problems and a synthetic problem constructed from four real-world optimization problems demonstrate that the proposed MIC is more effective than existing scalar infill criteria for GP-assisted optimization given a limited computational budget.

Constraint-handling techniques in surrogate-assisted evolutionary optimization. An empirical study

Many real-world constrained numerical optimization problems are defined through expensive models. An alternative to solve them is by using evolutionary algorithms. However, they usually require a considerable number of evaluations to get a competitive solution, and such task needs a high computational effort. This disadvantage can be tackled with surrogate models. In this paper, a study of the relationship between a surrogate model and different constraint-handling techniques is presented. The surrogate model (k nearest neighbors regression, kNN regression) is assembled into differential evolution with combined variants (DECV), named surrogate-assisted DECV (SA-DECV) to approximate the objective function value and the sum of constraint violation. Additionally, four SA-DECV variants with different constraint-handling techniques are assessed. The best variant is compared against two state-of-the-art algorithms. Twenty-four well-known test problems are solved in the experiments. According to the results, the best variant is the one that uses a ranking process as constraint-handling technique. Furthermore, a case of study is included, where a set of engineering optimization problems is solved with the best variant. Finally, the surrogate model is tested in other DE-based algorithms.

Heterogeneous Ensemble-Based Infill Criterion for Evolutionary Multiobjective Optimization of Expensive Problems.

Gaussian processes (GPs) are the most popular model used in surrogate-assisted evolutionary optimization of computationally expensive problems, mainly because GPs are able to measure the uncertainty of the estimated fitness values, based on which certain infill sampling criteria can be used to guide the search and update the surrogate model. However, the computation time for constructing GPs may become excessively long when the number of training samples increases, which makes it inappropriate to use them as surrogates in evolutionary optimization. To address this issue, this paper proposes to use ensembles as surrogates and infill criteria for model management in evolutionary optimization. A heterogeneous ensemble consisting of a least square support vector machine and two radial basis function networks is constructed to enhance the reliability of ensembles for uncertainty estimation. In addition to the original decision variables, a selected subset of the decision variables and a set of transformed variables are used as inputs of the heterogeneous ensemble to further promote the diversity of the ensemble. The proposed heterogeneous ensemble is compared with a GP and a homogeneous ensemble for infill sampling criteria in evolutionary multiobjective optimization. Experimental results demonstrate that the heterogeneous ensemble is competitive in performance compared with GPs and much more scalable in computational complexity to the increase in search dimension.