Expensive Multiobjective Problems Research Articles

Evolutionary Optimization with Simplified Helper Task for High-dimensional Expensive Multiobjective Problems

In recent years, surrogate-assisted evolutionary algorithms (SAEAs) have been sufficiently studied for tackling computationally expensive multiobjective optimization problems (EMOPs), as they can quickly estimate the qualities of solutions by using surrogate models to substitute for expensive evaluations. However, most existing SAEAs only show promising performance for solving EMOPs with no more than 10 dimensions, and become less efficient for tackling EMOPs with higher dimensionality. Thus, this article proposes a new SAEA with a simplified helper task for tackling high-dimensional EMOPs. In each generation, one simplified task will be generated artificially by using random dimension reduction on the target task (i.e., the target EMOPs). Then, two surrogate models are trained for the helper task and the target task, respectively. Based on the trained surrogate models, evolutionary multitasking optimization is run to solve these two tasks, so that the experiences of solving the helper task can be transferred to speed up the convergence of tackling the target task. Moreover, an effective model management strategy is designed to select new promising samples for training the surrogate models. When compared to five competitive SAEAs on four well-known benchmark suites, the experiments validate the advantages of the proposed algorithm on most test cases.

High-Dimensional Multi-Objective Bayesian Optimization With Block Coordinate Updates: Case Studies in Intelligent Transportation System

Many transportation system problems can be formulated as high-dimensional expensive multi-objective problems. They are challenging for Gaussian process-based Bayesian optimization methods to find the Pareto fronts due to the curse of dimensionality and the boundary issue in the acquisition function optimization. This paper presents a multi-objective Bayesian optimization method with block coordinate updates, Block-MOBO, to solve high-dimensional expensive multi-objective problems. Block-MOBO first partitions the decision variable space into different blocks, each of which includes a low-dimensional multi-objective problem. At each iteration, one block is considered and the decision variables not in this block are approximated by context-vector generation embedded with the Pareto prior knowledge thus promoting convergence. To tackle the boundary issue, we present <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\epsilon$</tex-math> </inline-formula> -greedy acquisition function in a Bayesian and multi-objective fashion, which recommends candidates either from the exploitation-exploration trade-off perspective or with probability <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\epsilon$</tex-math> </inline-formula> from the Pareto dominance relationship perspective. We verify the effectiveness of Block-MOBO by comparing it with other multi-objective Bayesian methods on two real-world optimization problems in transportation system and three multi-objective synthetic test suites. The experimental results show that Block-MOBO can find more evenly distributed and non-dominated solutions in the whole search space with lower complexity compared with other state-of-the-art approaches. Our analyses illustrate that block coordinate updates and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\epsilon$</tex-math> </inline-formula> -greedy acquisition function contribute to computational complexity reduction and convergence-diversity trade-offs, respectively.

A trust-region-like algorithm for expensive multi-objective optimization

In solving expensive multi-objective optimization problems, surrogate models have been widely investigated. The existing multi-objective algorithms adopting surrogate models can be classified into two categories: surrogate-assisted evolutionary algorithms (SAEAs) and surrogate-based optimization algorithms (SBAs). However, their efficiency and convergence remain considerably inadequate. In this work, we propose a trust-region-like algorithm for dealing with expensive multi-objective problems, where only a small number of expensive objective function evaluations are allowed. The optimization process of the proposed algorithm mainly includes two stages. The first stage is the surrogate-assisted stage, in which a promising population with the consideration of convergence and diversity is obtained by reference vector-guided evolutionary algorithms (RVEA). The promising population is considered the preliminary trust-region. Moreover, a new adaptive sampling selection criterion switching between two different sampling strategies is used to further narrow the trust-region. The second stage is the surrogate-based stage, wherein the selection of the most promising individuals is carried out from the ultimate trust-region using the pseudo hypervolume-based expected improvement matrix (PEIM) criterion. Comparison results on the benchmark functions demonstrate that the proposed algorithm is competitive with five state-of-art algorithms in a limited computational budget. Finally, the proposed algorithm is used in the design optimization of a variable stiffness composite cylinder.

Surrogate-assisted operator-repeated evolutionary algorithm for computationally expensive multi-objective problems

Surrogate-assisted strategies work differently in surrogate-assisted multi-objective evolutionary algorithms. How to select suitable surrogate-assisted strategies to balance global and local search and diversity of Pareto optimal solutions is usually difficult for efficient optimization of computationally expensive multi-objective problems, which only allow limited time costs in the optimization process. Therefore, to further improve the optimization efficiency for expensive multi-objective problems, a surrogate-assisted operator-repeated multi-objective evolutionary algorithm is proposed by reasonably integrating several surrogate-assisted strategies in this study. Specifically, the proposed algorithm is based on a operator-repeated offspring creation strategy, which can produce many diverse candidate offspring individuals and thus make the proposed algorithm search globally and efficiently. In addition, a novel infill criterion termed as reference-vector-guided surrogate-assisted penalty-based boundary intersection is proposed and combined with a common Kriging-based expected improvement matrix infill criterion for complementary prescreening over the candidate offspring individuals. These two infill criteria can make a good balance between global search and diversity of Pareto optimal solutions. In addition, to fasten convergence speed, an improved surrogate-based multi-objective local search method with minimum distance-angle sampling is also proposed and embedded in the proposed algorithm. Several benchmark problems with dimensions varying from 8 to 50 and a practical two-objective airfoil design optimization problem with 14 design variables are tested to validate efficiency of the proposed algorithm. The experimental results demonstrate that the proposed algorithm significantly outperforms some state-of-the-art surrogate-assisted multi-objective optimization algorithms on most of problems.

Complementary surrogate-assisted differential evolution algorithm for expensive multi-objective problems under a limited computational budget

Different surrogate-assisted strategies can greatly influence the optimization efficiency of surrogate-assisted multi-objective evolutionary algorithms. By hybridizing two complementary surrogate-assisted strategies, this study proposed an efficient surrogate-assisted differential evolution algorithm to optimize expensive multi-objective problems under a limited computational budget. The two proposed surrogate-assisted strategies balance global and local search for multi-objective optimization. Specifically, one strategy is an improved surrogate-based multi-objective local search method that is based on maximin angle-distance sequential sampling. Compared with the previous local search method that is based on Euclidian distance-based sampling, the improved local search method is more efficient because it can mitigate the scale difference of different objectives. The other surrogate-assisted strategy is prescreening based on a diversity-enhanced expected improvement matrix infill criterion. The proposed infill criterion aims to improve the diversity of approximate Pareto optimal solutions by considering distribution of candidate individuals in the objective space in the infill function. Within a limited computational burden, the performance of the proposed algorithm is demonstrated on a large set of multi-objective benchmark problems, as well as a real-world airfoil design problem. Experimental results show that the proposed algorithm performs significantly better than some existing algorithms on most problems investigated in this study.

A performance approximation assisted expensive many-objective evolutionary algorithm

Surrogate-assisted multi-objective evolutionary algorithms have been paid much attention to solve expensive multi-objective problems in recent years. However, with the number of objectives increasing, an improper solution may be picked for expensive objective evaluation due to the accumulation error of approximated values on objective functions. Furthermore, the time to construct surrogate models for all objectives will significantly increase. Thus, in this paper, Gaussian process (GP) models are proposed for performance indicators instead of for objective functions. Furthermore, solutions are selected from either of two ways to be evaluated using the expensive objective function. When there are non-dominated solutions found so far that are approximated, they will be exactly evaluated using the objective function. Otherwise, the solution with the maximum approximation uncertainty among the current population will be evaluated using the real objective functions. The efficiency of the presented approach is validated on the DTLZ test suite with 3, 6, 10, 15, and 20 objectives, MaF benchmark problems with 3, 6, 10, 15, and 20 objectives, and a real-world optimization problem called filter design. The experimental results show that the method proposed in this paper is competitive compared to recently proposed peer algorithms for expensive many-objective problems.

An adaptive batch Bayesian optimization approach for expensive multi-objective problems

This paper presents Adaptive Batch-ParEGO, an adaptive batch Bayesian optimization method for expensive multi-objective problems. This method extends the classical multi-objective Bayesian optimization method, sequential ParEGO, to the batch mode. Specifically, the proposed method exploits a newly proposed bi-objective acquisition function to recommend and evaluate multiple solutions. The bi-objective acquisition function takes exploitation and exploration as two optimization objectives, which are traded off by a multi-objective evolutionary algorithm. Since there’s usually a certain number of limited hardware resources available in reality, we further propose an adaptive solution selection criterion to fix the number of candidate solutions in each iteration. This strategy dynamically balances exploitation and exploration by tuning the hyper-parameter in the exploitation-exploration fitness function. In addition, the expected improvement is exploited to select another candidate solution to ensure convergence and make the algorithm more robust. We verify the effectiveness of Adaptive Batch-ParEGO on three multi-objective benchmarks and a hyperparameter tuning task of neural networks compared with the state-of-the-art multi-objective approaches. Our analysis demonstrates that the bi-objective acquisition function with the adaptive recommendation strategy can balance exploitation and exploration well in batch mode for expensive multi-objective problems. All our source codes will be published at https://github.com/thuiar/Adaptive-Batch-ParEGO.

High-dimensional expensive multi-objective optimization via additive structure

• A high-dimensional expensive multi-objective optimization method is proposed. • A multi-point sampling strategy to recommend multiple candidate solutions is introduced. • Additive Gaussian Structure in expensive multi-objective optimization in highdimensional cases is addressed. Expensive multi-objective problems (MOPs) are extremely challenging due to the high evaluation cost to find satisfying solutions with adequate precision, especially in high-dimensional cases. However, most of the current EGO-based algorithms for expensive MOPs are limited to low decision dimensions because of the exponential difficulty in high dimensional circumstances. This paper presents High-Dimensional Expensive Multi-objective Optimization with Additive structure (ADD-HDEMO) to solve high-dimensional expensive MOPs via additive structural kernel and identifies two key challenges in this endeavor. First, we integrate multiple sub-objectives in high-dimensional expensive MOPs into a single objective with the decision space unchanged. Then, we infer the dependence correlation between the decision and objective space of the augmented EMOP via an additive GP kernel structure where Gibbs sampling is used to learn the latent additive structure. Furthermore, we parallel the proposed algorithm by introducing a multi-point sampling mechanism when recommending infill points. The effectiveness of the proposed method is evaluated on ZDT and DTLZ benchmarks compared with three other EGO-based multi-objective optimization approaches, ParEGO, SMS-EGO and MOEA/D-EGO. Our analyses demonstrate that ADD-HDEMO is effective in solving high-dimensional expensive MOPs.

Using following heroes operation in multi-objective differential evolution for fast convergence

In this study, a new hybrid variant of multi-objective differential evolution is developed to improve the extent and the speed of convergence by combining the ability of DE/rand/1 strategy (NSDE1) to retain only better solutions in the selection operation with the ability of Following Heroes operation from adaptive social evolution algorithm to rapidly propagate the effect of the best solutions to the entire population. Such an algorithm is quite useful for fast convergence, particularly in solving computationally intensive multi-objective optimization problems for which it is highly desirable to obtain the better quality optimal solutions in a limited number of function calculations. The performance of the algorithm is first analyzed on thirty multi-objective test problems from ZDT, DTLZ, and WFG test suites for a limited number of function calculations, and the results are compared with four well-established multi-objective optimization algorithms: real-coded elitist non-dominated sorting genetic algorithm (RNSGA-II), non-dominated sorting particle swarm optimization (NSPSO), adaptive social evolution (ASE) and non-dominated sorting differential evolution with DE/rand/1 strategy (NSDE1) and found to be superior. The analysis is further extended to model-based algorithms, Pareto efficient global optimization (ParEGO), kriging for expensive evaluation Pareto (KEEP), and surrogate optimization of computationally expensive multi-objective problems (SOCEMO) which are commonly used for computationally intensive problems. These involve an additional computational cost of evolving the surrogate models. The performance of the proposed algorithm is found to be better than these, though, it has a relatively simple algorithmic structure and does not involve any additional computational cost. The algorithm is then evaluated on computationally intensive multi-objective optimization of bulk polymerization of methyl methacrylate. The numerical results obtained for this problem show better convergence for a limited number of function calculations.

Surrogate-assisted multi-objective particle swarm optimization for the operation of CO2 capture using VPSA

Multi-objective evolutionary algorithms (MOEAs) have received increasing attention over the past few decades. However, when applying MOEAs to solve computationally expensive real-world applications, they are often criticized due to the large number of function evaluations required. To this end, we proposed a method, called MOPSONN-EGO, for dealing with expensive multi-objective problems. MOPSONN-EGO uses Kriging models to approximate the objective function of expensive problems. The recently proposed MOPSONN algorithm is used to search the landscape of the Kriging models. The cheap-to-evaluate expected improvement matrix is adopted to select infill samples to update the Kriging models. Empirical results on several benchmark problems show the competitive performance of MOPSONN-EGO. Moreover, MOPSONN-EGO is applied to solve the multi-objective optimization problem of the Vacuum pressure swing adsorption (VPSA) for carbon dioxide capture. The simulation results are validated against the expensive model using normalized root mean square error (NRMSE) and compared to the One-shot surrogate modeling strategy. The results reveal the significant efficiency of the proposed algorithm in approximating the real Pareto front of VPSA using only a limited number of detailed model evaluations. Moreover, the obtained best values for each objective with their corresponding decision variables values are reported.

Preference-driven Kriging-based multiobjective optimization method with a novel multipoint infill criterion and application to airfoil shape design

Multipoint infill sampling has emerged recently as a promising tool to enhance the efficiency of the adaptive Kriging-based optimization method for computational expensive multiobjective problems. However, the number of infill samples in each iteration in the open literature is up to 10. A novel multipoint infill criterion is proposed on the basis of Expected Angle-Penalized Length Improvement (EAPLI). One of its distinct features is that it makes the number of infill samples in each iteration increase tenfold along with a fair convergence speed by using multiple reference vectors to search for multiple infill points. Its implementation involves several major elements: firstly, angle-penalized length based improvement is established for each reference vector; then maximizing EAPLI is conducted for all vectors to obtain sufficient candidate infill points; finally, multiple infill points are selected from the candidate pool according to the candidate niche counts. Experimental results on benchmark test problems show that the proposed EAPLI criterion is highly competitive in comparison with other criteria. Then the performance of EAPLI using different numbers of multiple infill points, from several to tens and even a hundred, is studied. The other distinct feature of EAPLI is, the distribution of reference vectors can be manipulated to include the designer's preferences to perform preference-driven optimization, which can reduce searches towards solutions violating the preferences and thus further enhance the optimization efficiency. A preference-driven airfoil shape optimization method is established by integrating the EAPLI, free-form deformation method, and a flow solver. It is used to optimize the NACA0012 airfoil, by which the non-dominated solutions are fairly driven into the preferred subspaces with significant gains in both objectives of lift/drag coefficient ratio and drag coefficient simultaneously.

Evolutionary Optimization of Expensive Multiobjective Problems With Co-Sub-Pareto Front Gaussian Process Surrogates.

This paper proposes a Gaussian process (GP) based co-sub-Pareto front surrogate augmentation strategy for evolutionary optimization of computationally expensive multiobjective problems. In the proposed algorithm, a multiobjective problem is decomposed into a number of subproblems, the solution of each of which is used to approximate a portion or sector of the Pareto front (i.e., a subPF). Thereafter, a multitask GP model is incorporated to exploit the correlations across the subproblems via joint surrogate model learning. A novel criterion for the utility function is defined on the surrogate landscape to determine the next candidate solution for evaluation using the actual expensive objectives. In addition, a new management strategy for the evaluated solutions is presented for model building. The novel feature of our approach is that it infers multiple subproblems jointly by exploiting the possible dependencies between them, such that knowledge can be transferred across subPFs approximated by the subproblems. Experimental studies under several scenarios indicate that the proposed algorithm outperforms state-of-the-art multiobjective evolutionary algorithms for expensive problems. The parameter sensitivity and effectiveness of the proposed algorithm are analyzed in detail.

SOCEMO: Surrogate Optimization of Computationally Expensive Multiobjective Problems

We present the algorithm SOCEMO for optimization problems that have multiple conflicting computationally expensive black-box objective functions. The computational expense arising from the objective function evaluations considerably restricts the number of evaluations that can be done to find Pareto-optimal solutions. Frequently used multiobjective optimization methods are based on evolutionary strategies and generally require a prohibitively large number of function evaluations to find a good approximation of the Pareto front. SOCEMO, in contrast, employs surrogate models to approximate the expensive objective functions. These surrogate models are used in the iterative sampling process to decide at which points in the variable domain the next expensive evaluations should be done. Therefore, fewer expensive objective function evaluations are needed, and a good approximation of the Pareto front can be found efficiently. Previous algorithms have generally been tested on problems with few variables (up to 10) and few objective functions (up to 5). In our numerical study, we show that our algorithm performs well for benchmark problems with up to 35 dimensions and up to 10 objective functions, as well as two engineering application problems. We compared the performance of SOCEMO to a variant of NSGA-II and show that SOCEMO’s sophisticated search strategy is more efficient than NSGA-II when the number of allowable function evaluations is low. The online supplement is available at https://doi.org/10.1287/ijoc.2017.0749 .

A Multifractal-Guided Multilevel Surrogate Model-Based Evolutionary Algorithm for Expensive Multiobjective Problems

In applied engineering, there are tremendous optimization problems which are multiobjective problems. Meanwhile, a number of them require large amount of time to evaluate their expensive cost function during optimization procedures. This kind of problems can be either financially expensive due to significant computational resources being required or time expensive due to numerous computational complexity. Aiming to this kind of problems, this paper proposed a multilevel surrogate model-based evolutionary algorithm. The proposed method employs DACE modeling method at the beginning to obtain a global trend in the decision domain. When more and more samples are involved and the sample distribution presents a trend or a manifold, the SVR model is utilized as a second-level surrogate model to achieve a better local search. The model transition is determined by the multifractal analysis on the solution set. Experimental results on ZDT and DTLZ standard test cases demonstrate that the time for EGO modeling can be reduced, and the accuracy can be better balanced by comparing to existing SVR and EGO methods.