Expensive Multiobjective Optimization Research Articles

Expected Improvement of Penalty-Based Boundary Intersection for Expensive Multiobjective Optimization

Computationally expensive multiobjective optimization problems are difficult to solve using solely evolutionary algorithms (EAs) and require surrogate models, such as the Kriging model. To solve such problems efficiently, we propose infill criteria for appropriately selecting multiple additional sample points for updating the Kriging model. These criteria correspond to the expected improvement of the penalty-based boundary intersection (PBI) and the inverted PBI. These PBI-based measures are increasingly applied to EAs due to their ability to explore better nondominated solutions than those that are obtained by the Tchebycheff function. In order to add sample points uniformly in the multiobjective space, we assign territories and niche counts to uniformly distributed weight vectors for evaluating the proposed criteria. We investigate these criteria in various test problems and compare them with established infill criteria for multiobjective surrogate-based optimization. Both proposed criteria yield better diversity and convergence than those obtained with other criteria for most of the test problems.

Expected Improvement Matrix-Based Infill Criteria for Expensive Multiobjective Optimization

The existing multiobjective expected improvement (EI) criteria are often computationally expensive because they are calculated using multivariate piecewise integrations, the number of which increases exponentially with the number of objectives. In order to solve this problem, this paper proposes a new approach to develop cheap-to-evaluate multiobjective EI criteria based on the proposed EI matrix (EIM). The elements in the EIM are the single-objective EIs that the studying point has beyond each Pareto front approximation point in each objective. Three multiobjective criteria are developed by combining the elements in the EIM into scalar functions in three different ways. These proposed multiobjective criteria are calculated using only 1-D integrations, whose number increases linearly with respect to the number of objectives. Moreover, all the three criteria are derived in closed form expressions, thus are significantly cheaper to evaluate than the state-of-the-art multiobjective criteria. The efficiencies of the proposed criteria are validated through 12 test problems. Besides the computational advantage, the proposed multiobjective EI criteria also show competitive abilities in approximating the Pareto fronts of the chosen test problems compared against the state-of-the-art multiobjective EI criteria.

Linear programming-based directed local search for expensive multi-objective optimization problems: Application to drinking water production plants

Local search (LS) is an essential module of most hybrid meta-heuristic evolutionary algorithms which are a major approach aimed to solve efficiently multi-objective optimization (MOO) problems. Furthermore, LS is specifically useful in many real-world applications where there is a need only to improve a current state of a system locally with limited computational budget and/or relying on computationally expensive process simulators. In these contexts, this paper proposes a new neighborhood-based iterative LS method, relying on first derivatives approximation and linear programming (LP), aiming to steer the search along any desired direction in the objectives space. The paper also leverages the directed local search (DS) method to constrained MOO problems. These methods are applied to the bi-objective (cost versus life cycle assessment-based environmental impact) optimization of drinking water production plants. The results obtained show that the proposed method constitutes a promising local search method which clearly outperforms the directed search approach.

An archive-based multi-objective evolutionary algorithm with adaptive search space partitioning to deal with expensive optimization problems: Application to process eco-design

In eco-design, the integration of environmental aspects into the earliest stage of design is considered with the aim of reducing adverse environmental impacts throughout a product's life cycle. An eco-design problem is therefore multi-objective, where several objectives (environmental, economic, and technological) are to be simultaneously optimized.The optimization of industrial processes usually requires solving expensive multi-objective optimization problems (MOPs). Aiming to solve efficiently MOPs, with a limited computational budget, this paper proposes a new framework called AMOEA-MAP. The framework relies on the structure of the NSGAII algorithm and possesses two novel operators: a memory-based adaptive partitioning strategy, which provides an adaptive reticulation of the search space for a quick identification of optimal zones with less computational effort; and a bi-population evolutionary algorithm, tailored for expensive optimization problems.To ascertain its generality, the framework is first tested on several tough benchmarks. Its performance is subsequently validated on a real-world eco-design problem.

A Review of Surrogate Assisted Multiobjective Evolutionary Algorithms.

Multiobjective evolutionary algorithms have incorporated surrogate models in order to reduce the number of required evaluations to approximate the Pareto front of computationally expensive multiobjective optimization problems. Currently, few works have reviewed the state of the art in this topic. However, the existing reviews have focused on classifying the evolutionary multiobjective optimization algorithms with respect to the type of underlying surrogate model. In this paper, we center our focus on classifying multiobjective evolutionary algorithms with respect to their integration with surrogate models. This interaction has led us to classify similar approaches and identify advantages and disadvantages of each class.

A fuzzy based approach for fitness approximation in multi-objective evolutionary algorithms

Evolutionary Algorithm provides a framework that is largely applicable to particular problems including multiobjective optimization problems, basically for the ease of their implementation and their capability to perform efficient parallel search. Indeed, in some cases, expensive multiobjective optimization evaluations might be a challenge to restrict the number of explicit fitness evaluations in multiobjective evolutionary algorithms. Accordingly, this article presents a novel approach that tackles this problem so as to not only decrease the number of fitness evaluations but also to improve the performance. During evolution, our proposed approach selects fit individuals based on the knowledge acquired throughout the search, and performs explicit fitness evaluations on these individuals. A comprehensive comparative analysis of a wide range of well-established test problems, selected from both traditional and state-of-the-art benchmarks, has been presented. Afterward, the effectiveness of the obtained results is compared with some of the state-of-the-art methods using two well-known metricsi.e. Hypervolume and Inverted Generational Distance (IGD). The experiments of our implemented approach is performed to illustrate that our proposal seems to be promising and would prove more efficient than other approaches in terms of both the performance and the computational cost.

A survey on handling computationally expensive multiobjective optimization problems using surrogates

Computationally expensive multiobjective optimization problems arise, e.g. in many engineering applications, where several conflicting objectives are to be optimized simultaneously while satisfying...

E-NAUTILUS: A decision support system for complex multiobjective optimization problems based on the NAUTILUS method

Interactive multiobjective optimization methods cannot necessarily be easily used when (industrial) multiobjective optimization problems are involved. There are at least two important factors to be considered with any interactive method: computationally expensive functions and aspects of human behavior. In this paper, we propose a method based on the existing NAUTILUS method and call it the Enhanced NAUTILUS (E-NAUTILUS) method. This method borrows the motivation of NAUTILUS along with the human aspects related to avoiding trading-off and anchoring bias and extends its applicability for computationally expensive multiobjective optimization problems. In the E-NAUTILUS method, a set of Pareto optimal solutions is calculated in a pre-processing stage before the decision maker is involved. When the decision maker interacts with the solution process in the interactive decision making stage, no new optimization problem is solved, thus, avoiding the waiting time for the decision maker to obtain new solutions according to her/his preferences. In this stage, starting from the worst possible objective function values, the decision maker is shown a set of points in the objective space, from which (s)he chooses one as the preferable point. At successive iterations, (s)he always sees points which improve all the objective values achieved by the previously chosen point. In this way, the decision maker remains focused on the solution process, as there is no loss in any objective function value between successive iterations. The last post-processing stage ensures the Pareto optimality of the final solution. A real-life engineering problem is used to demonstrate how E-NAUTILUS works in practice.

A survey on handling computationally expensive multiobjective optimization problems using surrogates: non-nature inspired methods

Computationally expensive multiobjective optimization problems arise, e.g. in many engineering applications, where several conflicting objectives are to be optimized simultaneously while satisfying constraints. In many cases, the lack of explicit mathematical formulas of the objectives and constraints may necessitate conducting computationally expensive and time-consuming experiments and/or simulations. As another challenge, these problems may have either convex or nonconvex or even disconnected Pareto frontier consisting of Pareto optimal solutions. Because of the existence of many such solutions, typically, a decision maker is required to select the most preferred one. In order to deal with the high computational cost, surrogate-based methods are commonly used in the literature. This paper surveys surrogate-based methods proposed in the literature, where the methods are independent of the underlying optimization algorithm and mitigate the computational burden to capture different types of Pareto frontiers. The methods considered are classified, discussed and then compared. These methods are divided into two frameworks: the sequential and the adaptive frameworks. Based on the comparison, we recommend the adaptive framework to tackle the aforementioned challenges.

Solving Expensive Multi-Objective Optimization Problems by Kriging Model with Multi-Point Updating Strategy

A method which utilizes Kriging model and a multi-point updating strategy is put forward for solving expensive multi-objective optimization problems. Assisted by a defined cheaper multi-objective optimization problem and a maximum average distance criterion, multiple updating points can be found. The proposed method is tested on two numerical functions and a ten-bar truss problem, the results show that the proposed method is efficient in obtaining Pareto optimal solutions with good convergence and diversity when the same computation resource is used comparing with two other methods.

A solution process for simulation-based multiobjective design optimization with an application in the paper industry

In this paper, we address some computational challenges arising in complex simulation-based design optimization problems. High computational cost, black-box formulation and stochasticity are some of the challenges related to optimization of design problems involving the simulation of complex mathematical models. Solving becomes even more challenging in case of multiple conflicting objectives that must be optimized simultaneously. In such cases, application of multiobjective optimization methods is necessary in order to gain an understanding of which design offers the best possible trade-off. We apply a three-stage solution process to meet the challenges mentioned above. As our case study, we consider the integrated design and control problem in paper mill design where the aim is to decrease the investment cost and enhance the quality of paper on the design level and, at the same time, guarantee the smooth performance of the production system on the operational level. In the first stage of the three-stage solution process, a set of solutions involving different trade-offs is generated with a method suited for computationally expensive multiobjective optimization problems using parallel computing. Then, based on the generated solutions an approximation method is applied to create a computationally inexpensive surrogate problem for the design problem and the surrogate problem is solved in the second stage with an interactive multiobjective optimization method. This stage involves a decision maker and her/his preferences to find the most preferred solution to the surrogate problem. In the third stage, the solution best corresponding that of stage two is found for the original problem.

PAINT: Pareto front interpolation for nonlinear multiobjective optimization

A method called PAINT is introduced for computationally expensive multiobjective optimization problems. The method interpolates between a given set of Pareto optimal outcomes. The interpolation provided by the PAINT method implies a mixed integer linear surrogate problem for the original problem which can be optimized with any interactive method to make decisions concerning the original problem. When the scalarizations of the interactive method used do not introduce nonlinearity to the problem (which is true e.g., for the synchronous NIMBUS method), the scalarizations of the surrogate problem can be optimized with available mixed integer linear solvers. Thus, the use of the interactive method is fast with the surrogate problem even though the problem is computationally expensive. Numerical examples of applying the PAINT method for interpolation are included.

Multiobjective evolutionary algorithms: A survey of the state of the art

A multiobjective optimization problem involves several conflicting objectives and has a set of Pareto optimal solutions. By evolving a population of solutions, multiobjective evolutionary algorithms (MOEAs) are able to approximate the Pareto optimal set in a single run. MOEAs have attracted a lot of research effort during the last 20 years, and they are still one of the hottest research areas in the field of evolutionary computation. This paper surveys the development of MOEAs primarily during the last eight years. It covers algorithmic frameworks such as decomposition-based MOEAs (MOEA/Ds), memetic MOEAs, coevolutionary MOEAs, selection and offspring reproduction operators, MOEAs with specific search methods, MOEAs for multimodal problems, constraint handling and MOEAs, computationally expensive multiobjective optimization problems (MOPs), dynamic MOPs, noisy MOPs, combinatorial and discrete MOPs, benchmark problems, performance indicators, and applications. In addition, some future research issues are also presented.

Constructing a Pareto front approximation for decision making

An approach to constructing a Pareto front approximation to computationally expensive multiobjective optimization problems is developed. The approximation is constructed as a sub-complex of a Delaunay triangulation of a finite set of Pareto optimal outcomes to the problem. The approach is based on the concept of inherent nondominance. Rules for checking the inherent nondominance of complexes are developed and applying the rules is demonstrated with examples. The quality of the approximation is quantified with error estimates. Due to its properties, the Pareto front approximation works as a surrogate to the original problem for decision making with interactive methods.

Multi-objective memetic algorithm: comparing artificial neural networks and pattern search filter method approaches

In this work, two methodologies to reduce the computation time of expensive multi-objective optimization problems are compared. These methodologies consist of the hybridization of a multi-objective evolutionary algorithm (MOEA) with local search procedures. First, an inverse artificial neural network proposed previously, consisting of mapping the decision variables into the multiple objectives to be optimized in order to generate improved solutions on certain generations of the MOEA, is presented. Second, a new approach based on a pattern search filter method is proposed in order to perform a local search around certain solutions selected previously from the Pareto frontier. The results obtained, by the application of both methodologies to difficult test problems, indicate a good performance of the approaches proposed.

Expensive Multiobjective Optimization by MOEA/D With Gaussian Process Model

In some expensive multiobjective optimization problems (MOPs), several function evaluations can be carried out in a batch way. Therefore, it is very desirable to develop methods which can generate multipler test points simultaneously. This paper proposes such a method, called MOEA/D-EGO, for dealing with expensive multiobjective optimization. MOEA/D-EGO decomposes an MOP in question into a number of single-objective optimization subproblems. A predictive model is built for each subproblem based on the points evaluated so far. Effort has been made to reduce the overhead for modeling and to improve the prediction quality. At each generation, MOEA/D is used for maximizing the expected improvement metric values of all the subproblems, and then several test points are selected for evaluation. Extensive experimental studies have been carried out to investigate the ability of the proposed algorithm.

Clustering aided approach for decision making in computationally expensive multiobjective optimization

Typically, industrial optimization problems need to be solved in an efficient, multiobjective and global manner, because they are often computationally expensive (as function values are typically based on simulations), they may contain multiple conflicting objectives, and they may have several local optima. Solving such problems may be challenging and time consuming when the aim is to find the most preferred Pareto optimal solution. In this study, we propose a method where we use an advanced clustering technique to reveal essential characteristics of the approximation of the Pareto optimal set, which has been generated beforehand. Thus, the decision maker (DM) is involved only after the most time consuming computation is finished. After the initiation phase, a moderate number of cluster prototypes projected to the Pareto optimal set is presented to the DM to be studied. This allows him/her to rapidly gain an overall understanding of the main characteristics of the problem without placing too much cognitive load on the DM. Furthermore, we also suggest some ways of applying our approach to different types of problems and demonstrate it with an example related to internal combustion engine design.

A Scalarizing One-Stage Algorithm for Efficient Multi-Objective Optimization

A novel kriging-assisted algorithm is proposed for computationally expensive multi-objective optimization problems, such as those which arise in electromagnetic design. The algorithm combines the multiple objectives into a single objective, which it then optimizes using a one-stage method from single-objective optimization. Inequality and equality constraint handling techniques are included, as is a method for dealing with failed iterations. Its efficiency is demonstrated on the constrained optimization of a pair of Helmholtz coils.

A fast Pareto genetic algorithm approach for solving expensive multiobjective optimization problems

We present a new multiobjective evolutionary algorithm (MOEA), called fast Pareto genetic algorithm (FastPGA), for the simultaneous optimization of multiple objectives where each solution evaluation is computationally- and/or financially-expensive. This is often the case when there are time or resource constraints involved in finding a solution. FastPGA utilizes a new ranking strategy that utilizes more information about Pareto dominance among solutions and niching relations. New genetic operators are employed to enhance the proposed algorithm's performance in terms of convergence behavior and computational effort as rapid convergence is of utmost concern and highly desired when solving expensive multiobjective optimization problems (MOPs). Computational results for a number of test problems indicate that FastPGA is a promising approach. FastPGA yields similar performance to that of the improved nondominated sorting genetic algorithm (NSGA-II), a widely-accepted benchmark in the MOEA research community. However, FastPGA outperforms NSGA-II when only a small number of solution evaluations are permitted, as would be the case when solving expensive MOPs.

A computationally efficient metamodeling approach for expensive multiobjective optimization

This paper explores a new metamodeling framework that may collapse the computational explosion that characterizes the modeling of complex systems under a multiobjective and/or multidisciplinary setting. Under the new framework, a pseudo response surface is constructed for each design objective for each discipline. This pseudo response surface has the unique property of being highly accurate in Pareto optimal regions, while it is intentionally allowed to be inaccurate in other regions. In short, the response surface for each design objective is accurate only where it matters. Because the pseudo response surface is allowed to be inaccurate in other regions of the design space, the computational cost of constructing it is dramatically reduced. An important distinguishing feature of the new framework is that the response surfaces for all the design objectives are constructed simultaneously in a mutually dependent fashion, in a way that identifies Pareto regions for the multiobjective problem. The new framework supports the puzzling notion that it is possible to obtain more accuracy and radically more design space exploration capability, while actually reducing the computation effort. This counterintuitive metamodeling paradigm shift holds the potential for identifying highly competitive products and systems that are well beyond today’s state of the art.