Expensive Multiobjective Optimization Problems Research Articles

A surrogate-assisted expensive constrained multi-objective global optimization algorithm and application

Expensive multi-objective optimization problems (MOPs) have seen the successful applications of surrogate-assisted evolutionary algorithms (SAEAs). Nevertheless, the majority of SAEAs are developed for costly unconstrained optimization, and costly constrained MOPs (CMOPs) have received less attention. Therefore, this article proposes a surrogate-assisted global optimization algorithm (named CTEA) for solving CMOPs within a very limited number of fitness evaluations. The proposed algorithm combines two selection frameworks, a bi-level selection framework, and an adaptive sampling framework, to enhance optimization performance. Leveraging on a constraint-improving strategy and a Pareto-based three-indicator criterion (convergence, constraint, and diversity indicators) at the different levels, the proposed bi-level selection framework can select more promising solutions. Moreover, an adaptive sampling framework is developed to prioritize objective and constraint functions and select the candidate solutions for real function evaluations according to the priority. Experimental results demonstrate that the proposed CTEA exhibits superior performance when compared with five state-of-the-art algorithms, achieving the best results in 61.9 % out of the 64 test instances. Finally, CTEA is applied to the multidisciplinary design optimization of blended-wing-body underwater gliders, and an impressive solution set is obtained.

Batch subproblem coevolution with gaussian process-driven linear models for expensive multi-objective optimization

The efficacy of surrogate-assisted multi-objective evolutionary algorithms (SAMOEAs) in addressing expensive multi-objective optimization problems (MOPs) is contingent upon the modeling techniques and model-based infill sampling strategies. In addressing this pivotal aspect, this paper introduces a pioneering methodology known as batch subproblem coevolution with Gaussian process-driven linear models (BSCo-GPLM). Specifically, from a modeling perspective, BSCo-GPLM decomposes the MOP into single-objective subproblems. Following this decomposition, for each subproblem, a Gaussian process-driven linear model (GPLM) is collaboratively trained to prevent overfitting and improve prediction accuracy. Regarding infill sampling, collaborative optimization of all GPLMs yields optimal candidate solutions for each subproblem, organized into coherent clusters. Within each cluster, only the solution with the highest utility is evaluated. Relying on the heightened prediction accuracy of the GPLM model and an efficient batch sampling strategy, BSCo-GPLM exhibits clear superiority over state-of-the-art SAMOEAs in effectively addressing expensive MOPs. The source code of BSCo-GPLM is available at https://github.com/CIAM-Group/EvolutionaryAlgorithm_Codes/tree/main/BSCo-GPLM.

Self-organizing surrogate-assisted non-dominated sorting differential evolution

Multi-objective optimization problems (MOPs) involve optimizing multiple conflicting objectives simultaneously, resulting in a set of Pareto optimal solutions. Due to the high computational or financial cost associated with evaluating fitness, expensive multi-objective optimization problems (EMOPs) further complicate the optimization process. Surrogate-assisted evolutionary algorithms (SAEAs) have emerged as a promising approach to address EMOPs by substituting costly evaluations with computationally efficient surrogate models. This paper introduces the self-organizing surrogate-assisted non-dominated sorting differential evolution (SSDE), which uses surrogate model based on a self-organizing map (SOM) to approximate the fitness function. SSDE offers advantages such as reduced computational cost, improved accuracy, and the speed of enhanced convergence. The SOM-based surrogate models effectively capture the underlying structure of the Pareto optimal set and Pareto optimal front, leading to superior approximations of the fitness function. Experimental results on benchmark functions and real-world problems, including Model-Free Adaptive Control (MFAC) and the Yagi-Uda Antenna design, demonstrate the competitiveness and efficiency of SSDE compared to other algorithms.

A parallel variable-fidelity algorithm for efficient constrained multi-objective aerodynamic design optimization

Modern aerodynamic design optimization aims to discover optimal configurations using computational fluid dynamics under complex flow conditions, which is a typical expensive multi-objective optimization problem. The multi-objective evolutionary algorithm based on decomposition (MOEA/D) combined with efficient global optimization is a promising method but requires enhanced efficiency and faces limitations in its application to multi-objective aerodynamic design optimization (MOADO). To address the issues, an efficient parallel MOEA/D assisted with variable-fidelity optimization (VFO) is proposed for solving MOADO, called the MOEA/D-VFO algorithm. Variable-fidelity surrogates are built for objectives and constraints, achieving higher accuracy using fewer high-fidelity samples and a great number of low-fidelity samples. By retaining more good candidates, the sub-optimization problems defined by decomposing original objectives are capable of discovering more favorable samples using MOEA/D, which prompts optimization convergence. A constraint-handling strategy is developed by incorporating the probability of feasibility functions in the sub-optimizations. The selection of new samples for parallel evaluation is improved by filtering out poor candidates and selecting effective promising samples, which improves the feasibility and diversity of solved Pareto solutions. A Pareto front (PF) can be efficiently found in a single optimization run. The proposed approach is demonstrated by four analytical test functions and verified by two aerodynamic design optimizations of airfoils with and without constraints, respectively. The results indicate that the MOEA/D-VFO approach can greatly improve optimization efficiency and obtain the PF satisfying constraints within an affordable computational budget.

An ensemble dual model assisted MOEA/D for tackling medium scale expensive multiobjective optimization

Many surrogate-assisted evolutionary algorithms (SAEAs) have been developed to address expensive multi-objective optimization problems (EMOPs). However, existing research primarily focuses on low-dimensional EMOPs. In this article, we propose an ensemble dual model-assisted multi-objective evolutionary algorithm based on decomposition to tackle medium-scale EMOPs. The proposed approach includes two key insights. First, a new ensemble strategy is proposed to improve both the prediction ability and computation efficiency of surrogates in tackling medium-scale EMOPs. Second, we develop an improved computing resource allocation strategy and an operator pool to enhance convergence capabilities. The improved computing resource allocation strategy gives more computing resources to the subproblems with poor fitness values, while the operator pool is used to generate promising offspring set in higher dimensions. Experimental results from three sets of expensive multi-objective test suites demonstrate that our proposed algorithm significantly outperforms seven compared SAEAs.

Accelerating surrogate assisted evolutionary algorithms for expensive multi-objective optimization via explainable machine learning

A series of surrogate-assisted evolutionary algorithms (SAEAs) have been proposed to handle expensive multi-objective optimization problems (EMOPs). However, the surrogate of these SAEAs is underutilized to a large extent, which limits the search efficiency of these algorithms. To be specific, existing algorithms do not sufficiently exploit the estimated solution quality information from the surrogate models during offspring generation. To address this issue, this paper proposes an SAEA framework named EXO-SAEA (EXplanation Operator based Surrogate-Assisted Evolutionary Algorithm). First, it divides the current population into two populations according to the a priori knowledge from the surrogate model. Then, for each solution in the first population, EXO-SAEA employs the SHapley Additive exPlanations (SHAP) model to estimate the contribution of each decision variable to the fitness values. After that, the Shapley values are then normalized for the offspring generation of the first population, while the second population uses generic GA operators. Two representative surrogate-assisted evolutionary algorithms are used to instantiate the proposed framework. Experimental results on the synthetic benchmark problems and three real-world problems involving six state-of-the-art algorithms demonstrate the effectiveness of the proposed framework.

Surrogate-assisted multi-objective evolutionary feature selection of generation-based fixed evolution control for time series forecasting with LSTM networks

Surrogate-assisted multi-objective evolutionary algorithms are powerful techniques to solve computationally expensive multi-objective optimization problems. In this paper, we propose a direct fitness replacement method with generation-based fixed evolution control to implement a multi-objective evolutionary algorithm that uses a surrogate model for wrapper-type feature selection, where long short-term memory is established as the learning algorithm. The importance of the work and its benefits lie in the need to reduce the excessive computational time required by conventional wrapper-type feature selection methods based on multi-objective evolutionary algorithms and LSTM networks, maintaining or improving the predictive capacity of the models. We analyze the use of incremental learning to update the surrogate model, in comparison with the conventional non-incremental learning approach. We applied these methods in real-life time series forecasting of air quality, indoor temperature in a smart building and oil temperature in electricity transformers. Multi-step ahead predictions of the forecast models obtained with different meta-learners of the surrogate model were compared by using the Diebold–Mariano statistical test on a multi-criteria performance metric. The proposed method outperformed other approaches for feature selection including, among others, methods based on surrogate-assisted multi-objective evolutionary algorithms developed by the authors in previous research, other surrogate-assisted deterministic methods for feature selection and the conventional wrapper-type feature selection method based on LSTM, improving the prediction on test dataset by 23.98%, 34.61% and 13.77%, respectively.

A New Multitask Joint Learning Framework for Expensive Multi-Objective Optimization Problems

A New Multitask Joint Learning Framework for Expensive Multi-Objective Optimization Problems

Hypervolume-Guided Decomposition for Parallel Expensive Multiobjective Optimization

The hypervolume metric is widely used to guide the search in multiobjective optimization. However, in parallel expensive multiobjective optimization, the hypervolume-based multi-point expected improvement (EI) suffers from high computational overhead and scales poorly with the batch size. To address this issue, we integrate hypervolume-based EI with the MOEA/D framework and propose a novel EI, named the expected direction-based hypervolume improvement (DirHV-EI). The DirHV-EI only measures the hypervolume improvement within each axis-parallel box induced by the modified Tchebycheff scalarization. Thus, it has a simple analytical expression that can be easily computed. Theoretical analysis indicates that the maximization of our proposed improvement function can help to maximize both the weighted hypervolume and the Tchebycheff improvement metrics. Using DirHV-EI, we design a decomposition-based Bayesian optimization algorithm, called DirHV-EGO, for solving expensive multiobjective optimization problems. At each iteration, the MOEA/D is used to maximize the DirHV-EI values with respect to a number of direction vectors in a collaborative manner, and a number of candidate solutions can be obtained. Then, a submodularity-based greedy selection strategy is used to select multiple query points from the candidates. Experimental results on both benchmark instances and real-world problems show that our proposed algorithm is an efficient and effective method for parallel expensive multiobjective optimization.

Are You Concerned about Limited Function Evaluations: Data-Augmented Pareto Set Learning for Expensive Multi-Objective Optimization

Optimizing multiple conflicting black-box objectives simultaneously is a prevalent occurrence in many real-world applications, such as neural architecture search, and machine learning. These problems are known as expensive multi-objective optimization problems (EMOPs) when the function evaluations are computationally or financially costly. Multi-objective Bayesian optimization (MOBO) offers an efficient approach to discovering a set of Pareto optimal solutions. However, the data deficiency issue caused by limited function evaluations has posed a great challenge to current optimization methods. Moreover, most current methods tend to prioritize the quality of candidate solutions, while ignoring the quantity of promising samples. In order to tackle these issues, our paper proposes a novel multi-objective Bayesian optimization algorithm with a data augmentation strategy that provides ample high-quality samples for Pareto set learning (PSL). Specifically, it utilizes Generative Adversarial Networks (GANs) to enrich data and a dominance prediction model to screen out high-quality samples, mitigating the predicament of limited function evaluations in EMOPs. Additionally, we adopt the regularity model to expensive multi-objective Bayesian optimization for PSL. Experimental results on both synthetic and real-world problems demonstrate that our algorithm outperforms several state-of-the-art and classical algorithms.

An XGBoost-assisted evolutionary algorithm for expensive multiobjective optimization problems

Many expensive optimization problems exist in various real-world applications. However traditional evolutionary algorithms are inadequate for solving these problems directly. Surrogate-assisted evolutionary algorithm (SAEA) can effectively solve expensive optimization problems using computationally inexpensive surrogate models. However, both the Kriging and ensemble models most SAEAs adopted have limited uncertainty of prediction, especially for expensive multiobjective optimization problems (EMOPs). To enhance the optimization performance of SAEA for EMOPs, this paper proposes a new XGBoost-assisted evolutionary algorithm, calling XGBEA. Specifically, XGBoost is used as the surrogate model, and a neighborhood density selection strategy based on a mixed population and archive space (NDS-MPA) is proposed to measure the uncertainties of individuals. XGBoost helps to best fit objective functions with different fitness landscapes. NDS-MPA selects non-dominated individuals with minimal density for re-evaluation, incorporating considerations of convergence, diversity and uncertainty. Experimental results on two well-studied benchmarks demonstrated the superiority of XGBEA over seven state-of-the-art SAEAs.

Evolutionary multiobjective optimization assisted by scalarization function approximation for high-dimensional expensive problems

Surrogate-assisted evolutionary algorithms (SAEAs) are a promising approach for solving expensive multiobjective optimization problems, but they often cannot address high-dimensional problems. Although one common approach to overcoming this challenge is to construct reliable surrogates, their accuracy inevitably deteriorates in a high-dimensional search space. Thus, this paper presents a novel SAEA based on scalarization function approximation, which is designed to strengthen its robustness against this deterioration. The proposed algorithm constructs an approximation model for each scalarization function defined in a decomposition-based framework. Each decomposed problem is then solved using multiple independent models trained for its neighbor problems. The intent is to decrease the risk of search performance degradations caused by unreliable approximations and retain the redundancy of the surrogate-assisted search to hedge the risk of over-fitting. Furthermore, each approximation model is adapted to a promising region of its corresponding decomposed problem to reduce the complexity of model fitting given a limited number of training samples. Experimental results show that the proposed algorithm is competitive with state-of-the-art SAEAs adapted for high-dimensional problems.

Evolutionary Optimization with Simplified Helper Task for High-dimensional Expensive Multiobjective Problems

In recent years, surrogate-assisted evolutionary algorithms (SAEAs) have been sufficiently studied for tackling computationally expensive multiobjective optimization problems (EMOPs), as they can quickly estimate the qualities of solutions by using surrogate models to substitute for expensive evaluations. However, most existing SAEAs only show promising performance for solving EMOPs with no more than 10 dimensions, and become less efficient for tackling EMOPs with higher dimensionality. Thus, this article proposes a new SAEA with a simplified helper task for tackling high-dimensional EMOPs. In each generation, one simplified task will be generated artificially by using random dimension reduction on the target task (i.e., the target EMOPs). Then, two surrogate models are trained for the helper task and the target task, respectively. Based on the trained surrogate models, evolutionary multitasking optimization is run to solve these two tasks, so that the experiences of solving the helper task can be transferred to speed up the convergence of tackling the target task. Moreover, an effective model management strategy is designed to select new promising samples for training the surrogate models. When compared to five competitive SAEAs on four well-known benchmark suites, the experiments validate the advantages of the proposed algorithm on most test cases.

Bi-Level Model Management Strategy for Solving Expensive Multi-Objective Optimization Problems

Bi-Level Model Management Strategy for Solving Expensive Multi-Objective Optimization Problems

Solving High-Dimensional Expensive Multiobjective Optimization Problems by Adaptive Decision Variable Grouping

Solving High-Dimensional Expensive Multiobjective Optimization Problems by Adaptive Decision Variable Grouping

Inverse distance weighting and radial basis function based surrogate model for high-dimensional expensive multi-objective optimization

Radial basis function (RBF) models have attracted a lot of attention in assisting evolutionary algorithms for solving computationally expensive optimization problems. However, most RBFs cannot directly provide the uncertainty information of their predictions, making it difficult to adopt principled infill sampling criteria for model management. To overcome this limitation, an inverse distance weighting (IDW) and RBF based surrogate assisted evolutionary algorithm, named IR-SAEA, is proposed to address high-dimensional expensive multi-objective optimization problems. First, an RBF-IDW model is developed, which can provide both the predicted objective values and the uncertainty of the predictions. Moreover, a modified lower confidence bound infill criterion is proposed based on the RBF-IDW for the balance of exploration and exploitation. Extensive experiments have been conducted on widely used benchmark problems with up to 100 dimensions. The empirical results have validated that the proposed algorithm is able to achieve a competitive performance compared with state-of-the-art SAEAs.

Surrogate-assisted evolutionary algorithm with decomposition-based local learning for high-dimensional multi-objective optimization

When the evolutionary algorithm is applied to handle high-dimensional expensive multi-objective optimization problems (MOPs), population evolution is crucial since it controls exploration and exploitation and decides if promising candidate solutions could be generated. However, little attention has been paid to this issue, evolution operators based on Genetic Algorithm (GA) and Differential Evolution (DE) are still the two most common approaches, whose convergence on high-dimensional MOPs with a limited number of fitness evaluations remains challenging. In this paper, we propose a decomposition-based local learning strategy to accelerate convergence in the high-dimensional search space of MOPs. Specifically, an individual is updated by learning from one of the best solutions of its corresponding local area based on the multi-objective decomposition approach. Accordingly, a surrogate-assisted evolutionary algorithm is proposed for better solving expensive high-dimensional MOPs. Experimental studies on MOPs with up to 100 decision variables and with 300 fitness evaluations demonstrate the effectiveness of the proposed method. Furthermore, we use the proposed method to solve a 2-objective shape design problem of the blended wing-body underwater glider (BWBUG) with 39 decision variables, and an impressive solution set is obtained.

A trust-region-like algorithm for expensive multi-objective optimization

In solving expensive multi-objective optimization problems, surrogate models have been widely investigated. The existing multi-objective algorithms adopting surrogate models can be classified into two categories: surrogate-assisted evolutionary algorithms (SAEAs) and surrogate-based optimization algorithms (SBAs). However, their efficiency and convergence remain considerably inadequate. In this work, we propose a trust-region-like algorithm for dealing with expensive multi-objective problems, where only a small number of expensive objective function evaluations are allowed. The optimization process of the proposed algorithm mainly includes two stages. The first stage is the surrogate-assisted stage, in which a promising population with the consideration of convergence and diversity is obtained by reference vector-guided evolutionary algorithms (RVEA). The promising population is considered the preliminary trust-region. Moreover, a new adaptive sampling selection criterion switching between two different sampling strategies is used to further narrow the trust-region. The second stage is the surrogate-based stage, wherein the selection of the most promising individuals is carried out from the ultimate trust-region using the pseudo hypervolume-based expected improvement matrix (PEIM) criterion. Comparison results on the benchmark functions demonstrate that the proposed algorithm is competitive with five state-of-art algorithms in a limited computational budget. Finally, the proposed algorithm is used in the design optimization of a variable stiffness composite cylinder.

A Steady-State Algorithm for Solving Expensive Multiobjective Optimization Problems With Nonparallelizable Evaluations

Expensive multi-objective optimization problems (EMOPs) refer to those wherein evaluation of each candidate solution incurs a significant cost. To solve such problems within a limited number of solution evaluations, surrogate-assisted evolutionary algorithms (SAEAs) are often used. However, existing SAEAs typically operate in a generational framework wherein multiple solutions are identified for evaluation in each generation. There exist relatively few proposals in steady-state framework, wherein only a single solution is evaluated in each iteration. The development of such algorithms is crucial to efficiently solve EMOPs for which the evaluation of candidate designs cannot be parallelized. Furthermore, regardless of the framework used, the performance of current SAEAs tends to degrade when the Pareto front (PF) of the problem has irregularities, such as extremely concave/convex segments, even for 2/3-objective problems. To contextualize the motivation of this study, the performance of a few state-of-the-art SAEAs is first demonstrated on some such selected problems. Then, to address the above research gaps, we propose a surrogate-assisted steady-state EA (SASSEA), which incorporates a number of novel elements including (a) effective use of model uncertainty information to aid the search, including the use of probabilistic dominance and Mahalanobis distance, (b) two-step infill identification using non-dominance (ND) and distance based selection, and (c) a shadow ND mechanism to avoid repeated selection and evaluation of dominated solutions. The efficacy of the proposed approach is demonstrated through extensive benchmarking on a range of test problems. It shows competitive performance relative to many state-of-the-art SAEAs, including both steady-state and generational approaches.

A dual surrogate assisted evolutionary algorithm based on parallel search for expensive multi/many-objective optimization

Numerous optimization problems in the real world involve multi-objective and computationally expensive simulations (i.e., expensive multi-objective optimization problems). This paper purposes a dual surrogates-assisted evolutionary algorithm (SAEA) based on parallel search, termed DSAEA-PS, for this issue. Approximation and classification are two main implementation forms of surrogate models, but the existing methods of expensive multi-objective optimization only apply one kind of them, and scarce works have paid attention to combining approximation and classification to improve the optimization performance. In the proposed algorithm, to enhance the prediction accuracy and reliability, both the approximation model and classification model are applied to cooperate to provide the quality and uncertainty information of candidate solutions. Meanwhile, the parallel search based on heterogeneous multi-objective evolutionary algorithms is introduced for better exploration of the decision space. In addition, combined with the strengthened dominance relation (SDR), a sampling strategy that comprehensively considers the quality of candidate solutions and their uncertainty information is proposed. Experimental results with five peer competitors on a set of widely-used benchmark problems demonstrate the ability of DSAEA-PS. Furthermore, DSAEA-PS is adopted for a five-objective blended-wing-body underwater glider design problem that involves time-consuming simulations of fluid dynamics and structural strength. A series of high-performance solutions obtained from DSAEA-PS verifies its effectiveness on engineering applications.