Evolutionary Multi-objective Optimization Methods Research Articles

Exploring the Explainable Aspects and Performance of a Learnable Evolutionary Multiobjective Optimization Method

Multiobjective optimization problems have multiple conflicting objective functions to be optimized simultaneously. The solutions to these problems are known as Pareto optimal solutions, which are mathematically incomparable. Thus, a decision maker must be employed to provide preferences to find the most preferred solution. However, decision makers often lack support in providing preferences and insights in exploring the solutions available. We explore the combination of learnable evolutionary models with interactive indicator-based evolutionary multiobjective optimization to create a learnable evolutionary multiobjective optimization method. Furthermore, we leverage interpretable machine learning to provide decision makers with potential insights about the problem being solved in the form of rule-based explanations. In fact, we show that a learnable evolutionary multiobjective optimization method can offer advantages in the search for solutions to a multiobjective optimization problem. We also provide an open source software framework for other researchers to implement and explore our ideas in their own works. Our work is a step toward establishing a new paradigm in the field on multiobjective optimization: explainable and learnable multiobjective optimization . We take the first steps toward this new research direction and provide other researchers and practitioners with necessary tools and ideas to further contribute to this field.

A Performance Indicator for Interactive Evolutionary Multiobjective Optimization Methods

In recent years, interactive evolutionary multiobjective optimization methods have been getting more and more attention. In these methods, a decision maker, who is a domain expert, is iteratively involved in the solution process and guides the solution process toward her/his desired region with preference information. However, there have not been many studies regarding the performance evaluation of interactive evolutionary methods. On the other hand, indicators have been developed for a priori methods, where the DM provides preference information before optimization. In the literature, some studies treat interactive evolutionary methods as a series of a priori steps when assessing and comparing them. In such settings, indicators designed for a priori methods can be utilized. In this paper, we propose a novel performance indicator for interactive evolutionary multiobjective optimization methods and show how it can assess the performance of these interactive methods as a whole process and not as a series of separate steps. In addition, we demonstrate the shortcomings of using indicators designed for a priori methods for comparing interactive evolutionary methods.

A data-driven intelligent decision support system that combines predictive and prescriptive analytics for the design of new textile fabrics

In this paper, we propose an Intelligent Decision Support System (IDSS) for the design of new textile fabrics. The IDSS uses predictive analytics to estimate fabric properties (e.g., elasticity) and composition values (% cotton) and then prescriptive techniques to optimize the fabric design inputs that feed the predictive models (e.g., types of yarns used). Using thousands of data records from a Portuguese textile company, we compared two distinct Machine Learning (ML) predictive approaches: Single-Target Regression (STR), via an Automated ML (AutoML) tool, and Multi-target Regression, via a deep learning Artificial Neural Network. For the prescriptive analytics, we compared two Evolutionary Multi-objective Optimization (EMO) methods (NSGA-II and R-NSGA-II) when optimizing 100 new fabrics, aiming to simultaneously minimize the physical property predictive error and the distance of the optimized values when compared with the learned input space. The two EMO methods were applied to design of 100 new fabrics. Overall, the STR approach provided the best results for both prediction tasks, with Normalized Mean Absolute Error values that range from 4% (weft elasticity) to 11% (pilling) in terms of the fabric properties and a textile composition classification accuracy of 87% when adopting a small tolerance of 0.01 for predicting the percentages of six types of fibers (e.g., cotton). As for the prescriptive results, they favored the R-NSGA-II EMO method, which tends to select Pareto curves that are associated with an average 11% predictive error and 16% distance.

A Reference Point-Based Evolutionary Algorithm Solves Multi and Many-Objective Optimization Problems: Method and Validation.

The integration of a decision maker's preferences in evolutionary multi-objective optimization (EMO) has been a common research scope over the last decade. In the published literature, several preference-based evolutionary approaches have been proposed. The reference point-based non-dominated sorting genetic (R-NSGA-II) algorithm represents one of the well-known preference-based evolutionary approaches. This method mainly aims to find a set of the Pareto-optimal solutions in the region of interest (ROI) rather than obtaining the entire Pareto-optimal set. This approach uses Euclidean distance as a metric to calculate the distance between each candidate solution and the reference point. However, this metric may not produce desired solutions because the final minimal Euclidean distance value is unknown. Thus, determining whether the true Pareto-optimal solution is achieved at the end of optimization run becomes difficult. In this study, R-NSGA-II method is modified using the recently proposed simplified Karush-Kuhn-Tucker proximity measure (S-KKTPM) metric instead of the Euclidean distance metric, where S-KKTPM-based distance measure can predict the convergence behavior of a point from the Pareto-optimal front without prior knowledge of the optimum solution. Experimental results show that the algorithm proposed herein is highly competitive compared with several state-of-the-art preference-based EMO methods. Extensive experiments were conducted with 2 to 10 objectives on various standard problems. Results show the effectiveness of our algorithm in obtaining the preferred solutions in the ROI and its ability to control the size of each preferred region separately at the same time.

A decomposition-based many-objective evolutionary algorithm with adaptive weight vector strategy

Evolutionary multi-objective optimization methods have become increasingly popular in finding a representative set of Pareto optimal solutions for multi-objective optimization problems (MOPs). As an important multi-objective optimization method, MOEA/D has been successfully applied to resolve various of MOPs. However, there still exist two issues in MOEA/D. First, the distribution of result set is greatly limited in complex Pareto Front (PF) shapes. Second, the effectiveness is relatively poor for solving many-objective optimization problems (MaOPs). Although many researchers attempt to improve MOEA/D, these two issues have not been solved well. This paper proposes a modified MOEA/D method called MOEA/D-VW which involves three optimization strategies. First, an adaptive weight vector adjustment strategy is adopted to fine-tune the subproblems, thus changing the evolution direction of the subpopulation. Second, a weight vector initialization strategy with preference parameters is added into MOEA/D-VW. Third, a modified crossover operator is leveraged to generate new population with good diversity and overcome the problem of high computational cost. Experiments are carried out on the set of 23 continuous optimization problems. The experimental results show that the performance and the distribution of the Pareto Set (PS) of MOEA/D-VW are effectively improved. Meanwhile, the experimental results demonstrate that the modified crossover operator can effectively reduce the computational cost of MOEA/D-VW.

Adaptive Design of Experiments for Automotive Engine Applications Using Concurrent Bayesian Optimization

Abstract Most practical automotive problems require the design of experiments (DoEs) over a number of different operating conditions to deliver optimal calibration parameters. DoE is especially crucial for automotive engine calibration problems due to its increasing complexity and nonlinearity. As the complexity of the system increases, the DoE applications require a significant amount of expensive testing. However, only a limited number of testings are available and desired. The current work addresses this issue by presenting an adaptive DoE method based on Bayesian optimization to find optimal parameter settings with a significantly reduced number of physical testings (or function evaluations). To further improve optimization efficiency, this work presents a new approach: concurrent Bayesian optimization, which searches for optimal DoE under multiple operating conditions simultaneously. The method utilizes a surrogate model and a novel concurrent evolutionary multi-objective optimization method: concurrent non-dominated sorting genetic algorithm-II, to solve adaptive DoE in multiple operating conditions with a limited number of function evaluations. The experimental study is carried out on a gasoline engine calibration problem using a high-fidelity GT-SUITE™ engine model. The experimental results demonstrate the effectiveness of the algorithm by optimizing engine performance with a significantly reduced number of expensive testings to achieve accurate optimal solutions. The method simultaneously performs engine calibration at eight different operating conditions using only 500–600 testings, compared to the traditional approach, where each operating condition requires 300–500 testings independently to achieve optimal results.

Comparing interactive evolutionary multiobjective optimization methods with an artificial decision maker

Solving multiobjective optimization problems with interactive methods enables a decision maker with domain expertise to direct the search for the most preferred trade-offs with preference information and learn about the problem. There are different interactive methods, and it is important to compare them and find the best-suited one for solving the problem in question. Comparisons with real decision makers are expensive, and artificial decision makers (ADMs) have been proposed to simulate humans in basic testing before involving real decision makers. Existing ADMs only consider one type of preference information. In this paper, we propose ADM-II, which is tailored to assess several interactive evolutionary methods and is able to handle different types of preference information. We consider two phases of interactive solution processes, i.e., learning and decision phases separately, so that the proposed ADM-II generates preference information in different ways in each of them to reflect the nature of the phases. We demonstrate how ADM-II can be applied with different methods and problems. We also propose an indicator to assess and compare the performance of interactive evolutionary methods.

Multiobjective Sine Cosine Algorithm for Remote Sensing Image Spatial-Spectral Clustering.

Remote sensing image data clustering is a tough task, which involves classifying the image without any prior information. Remote sensing image clustering, in essence, belongs to a complex optimization problem, due to the high dimensionality and complexity of remote sensing imagery. Therefore, it can be easily affected by the initial values and trapped in locally optimal solutions. Meanwhile, remote sensing images contain complex and diverse spatial-spectral information, which makes them difficult to model with only a single objective function. Although evolutionary multiobjective optimization methods have been presented for the clustering task, the tradeoff between the global and local search abilities is not well adjusted in the evolutionary process. In this article, in order to address these problems, a multiobjective sine cosine algorithm for remote sensing image data spatial-spectral clustering (MOSCA_SSC) is proposed. In the proposed method, the clustering task is converted into a multiobjective optimization problem, and the Xie-Beni (XB) index and Jeffries-Matusita (Jm) distance combined with the spatial information term (SI_Jm measure) are utilized as the objective functions. In addition, for the first time, the sine cosine algorithm (SCA), which can effectively adjust the local and global search capabilities, is introduced into the framework of multiobjective clustering for continuous optimization. Furthermore, the destination solution in the SCA is automatically selected and updated from the current Pareto front through employing the knee-point-based selection approach. The benefits of the proposed method were demonstrated by clustering experiments with ten UCI datasets and four real remote sensing image datasets.

DMA Segmentation and Multiobjective Optimization for Trading Off Water Age, Excess Pressure, and Pump Operational Cost in Water Distribution Systems

This study presents a heuristic multiobjective approach for segmenting and operating water distribution systems (WDS). The methodology employs a two-pronged strategy: the first is a heuristic method for dividing the network into clusters (i.e., district metering areas) based on connectivity analysis. The second is the application of the evolutionary multiobjective optimization method non-dominated sorting genetic algorithm (NSGA)-II for trading off the operational cost, excess pressure (serving as a proxy to leakage reduction), and water age (acting as a surrogate to water quality) in the WDS. Three example applications of increasing complexities with various cluster partitioning are explored, showing a clear trade-off among the objectives. This study introduces an unprecedented heuristic approach for jointly solving the multiobjective problem under a given system partitioning. However, by enforcing a priori clustering formation (rather than including it in the optimization), optimality, completeness, and precision are compromised in favor of computational speed and effort. Thus, additional sensitivities need to be conducted outside of the optimization for the clusters’ impact. Challenges of extending this study are in embedding the clusters’ formations in the optimization considering other objectives such as residual capacity, developments of other optimization frameworks outside of the generic link of simulation-optimization, and uncertainty inclusion (e.g., in demands). All data and codes are included for allowing full replications and comparisons.

Evolutionary Multiobjective Optimization for Automatic Agent-Based Model Calibration: A Comparative Study

Complex problems can be analyzed by using model simulation but its use is not straight-forward since modelers must carefully calibrate and validate their models before using them. This is specially relevant for models considering multiple outputs as its calibration requires handling different criteria jointly. This can be achieved using automated calibration and evolutionary multiobjective optimization methods which are the state of the art in multiobjective optimization as they can find a set of representative Pareto solutions under these restrictions and in a single run. However, selecting the best algorithm for performing automated calibration can be overwhelming. We propose to deal with this issue by conducting an exhaustive analysis of the performance of several evolutionary multiobjective optimization algorithms when calibrating several instances of an agent-based model for marketing with multiple outputs. We analyze the calibration results using multiobjective performance indicators and attainment surfaces, including a statistical test for studying the significance of the indicator values, and benchmarking their performance with respect to a classical mathematical method. The results of our experimentation reflect that those algorithms based on decomposition perform significantly better than the remaining methods in most instances. Besides, we also identify how different properties of the problem instances (i.e., the shape of the feasible region, the shape of the Pareto front, and the increased dimensionality) erode the behavior of the algorithms to different degrees.

A multi-objective optimization method for identification of module biomarkers for disease diagnosis.

Biomarker identification aims at finding a set of biological indicators that best discriminate biological samples of different phenotypes. In this paper, we take the module containing the significant disease-related genes and their interactions from biological networks as a module biomarker, and propose an evolutionary multi-objective optimization method to identify module biomarkers for disease diagnosis. To be specific, we take the classification accuracy on control and disease samples, the association with disease and the intra-link density in the module as the optimization objectives. To achieve the best performance, a novel population initiation strategy is tailored to generate dense-connected initial solutions, and a specific population update strategy is employed to direct the evolution towards the global optimums with abundant diversity. Experimental results show that our method outperforms the previous state-of-the-art disease diagnosis methods. Meantime, the detected biomarker module can reflect the basic and significant biological functions and has a great correlation with a disease phenotype.

Sparse Feature Learning of Hyperspectral Imagery via Multiobjective-Based Extreme Learning Machine.

Hyperspectral image (HSI) consists of hundreds of narrow spectral band components with rich spectral and spatial information. Extreme Learning Machine (ELM) has been widely used for HSI analysis. However, the classical ELM is difficult to use for sparse feature leaning due to its randomly generated hidden layer. In this paper, we propose a novel unsupervised sparse feature learning approach, called Evolutionary Multiobjective-based ELM (EMO-ELM), and apply it to HSI feature extraction. Specifically, we represent the task of constructing the ELM Autoencoder (ELM-AE) as a multiobjective optimization problem that takes the sparsity of hidden layer outputs and the reconstruction error as two conflicting objectives. Then, we adopt an Evolutionary Multiobjective Optimization (EMO) method to solve the two objectives, simultaneously. To find the best solution from the Pareto solution set and construct the best trade-off feature extractor, a curvature-based method is proposed to focus on the knee area of the Pareto solutions. Benefited from the EMO, the proposed EMO-ELM is less prone to fall into a local minimum and has fewer trainable parameters than gradient-based AEs. Experiments on two real HSIs demonstrate that the features learned by EMO-ELM not only preserve better sparsity but also achieve superior separability than many existing feature learning methods.

Ra-dominance: A new dominance relationship for preference-based evolutionary multiobjective optimization

While traditional Pareto-based evolutionary multi-objective optimization (EMO) algorithms have shown an excellent balance between convergence and diversity on a wide range of practical problems with two or three objectives in real applications, the decision maker (DM) is interested in a unique set of solutions rather than the whole population on Pareto optimal front (POF). In addition, Pareto-based EMO algorithms have some shortcomings in dealing with many-objective problems because of insufficient selection pressure toward trade-off solutions. Due to the above, it is crucial to incorporate DM preference information into EMO and seek a representative subset of Pareto optimal solutions with an increase in the number of objectives. This paper proposes a new dominance relationship, called Ra-dominance, which can improve diversity among the Pareto-equivalent solutions increase the selection pressure in evolutionary process. It has the ability to guide the population toward areas more responsive to the needs of the DM according to a reference point and preference angle. We use the new dominance relationship in the NSGA-II algorithm, and the efficacy and usefulness of the modified procedure are assessed through two- to ten-objective problems. Experimental results show that the algorithm applying this new dominance relationship is highly competitive when compared with four state-of-the-art preference-based EMO methods.

Multiobjective Hyperspectral Feature Selection Based on Discrete Sine Cosine Algorithm

Feature selection is an effective way to reduce the data dimensionality of hyperspectral imagery and obtain a better performance in the subsequent applications, such as classification. The ideal approach is to obtain the optimal tradeoff between two criteria for hyperspectral image feature selection: 1) information preservation and 2) redundancy reduction. However, constructing a hyperspectral feature selection model for the above two criteria is difficult due to the complexity of hyperspectral imagery. Although evolutionary multiobjective optimization methods have been recently presented to simultaneously optimize the above criteria, they cannot control the global exploration versus local exploitation capabilities in the search space for the hyperspectral feature selection problem. Thus, in this article, a novel discrete sine cosine algorithm (SCA)-based multiobjective feature selection (MOSCA_FS) approach is proposed for hyperspectral imagery. In the proposed method, a novel and effective framework of multiobjective hyperspectral feature selection is designed. In the framework, the ratio between the Jeffries–Matusita (JM) distance and mutual information (MI) is modeled to minimize the redundancy and maximize the relevance of the selected feature subset. In addition, another measurement—the variance (Var)—is applied for maximizing the information amount. Furthermore, to resolve the discrete hyperspectral feature selection problem, a novel discrete SCA is first proposed, which enhances the selection of the ideal feature subset. The effectiveness and universality of the proposed method was verified by experiments with ten University of California at Irvine (UCI) data sets, five hyperspectral image data sets, and one spectral data set of typical surface features.

Route Generation for Drone Delivery Service Using an Evolutionary Multi-Objective Optimization Method

Route Generation for Drone Delivery Service Using an Evolutionary Multi-Objective Optimization Method

Extractive Multi-Document Arabic Text Summarization Using Evolutionary Multi-Objective Optimization With K-Medoid Clustering

The increasing usage of the Internet and social networks has produced a significant amount of online textual data. These online textual data led to information overload and redundancy. It is important to eliminate the information redundancy and preserve the time required for reading these online textual data. Thus, there is a persistent need for an automatic text summarization system, which extract the relevant and salient information from a collection of documents, that sharing the same or related topics. Then, presenting this extracted information in a condensed form to preserve the main topics. This paper proposes an automatic, generic, and extractive Arabic multi-document summarization system. The proposed system employs the clustering-based and evolutionary multi-objective optimization methods. The clustering-based method discovers the main topics in the text, while the evolutionary multi-objective optimization method optimizes three objectives based on coverage, diversity/redundancy, and relevancy. The performance of the proposed system is evaluated using TAC 2011 and DUC 2002 datasets. The experimental results are compared using ROUGE evaluation measure. The obtained results showed the effectiveness of the proposed system compared to other peer systems. The proposed system outperformed other peer systems for all ROUGE metrics using TAC 2011. We achieved an F-measure of 38.9%, 17.7%, 35.4%, and 15.8% for Rouge-1, Rouge-2, Rouge-L, and Rouge-SU4, respectively. In addition, the proposed system with DUC 2002 dataset achieved an F-measure of 47.1%, 23.7%, 47.1%, 20.4% for Rouge-1, Rouge-2, Rouge-L, and Rouge-SU4, respectively.

Explicit Control of Implicit Parallelism in Decomposition-Based Evolutionary Many-Objective Optimization Algorithms [Research Frontier

Over the past two decades, Evolutionary Multi-objective Optimization (EMO) algorithms have demonstrated their ability to find and maintain multiple trade-off solutions in two and three-objective problems, making EMO as one of the most emergent and exciting fields of research and application within Computational Intelligence (CI) area. The main reason for EMO's success is that the population-based EMO operators are able to establish an implicit parallel search within an evolving population to find multiple Pareto-optimal regions of the search space parallelly. For many-objective optimization problems involving a largedimensional objective space, the extent of implicit parallelism is argued here to be too generic, compared to the same in a lower-dimensional objective space. Decomposition-based EMO algorithms - a recent trend in EMO literature - which divide the overall computing task into a number of sub-tasks of focusing within a region of the search space have found to be successful in solving many-objective problems. In this paper, we study the effect of explicit control of an algorithm's implicit parallelism mechanism for achieving an enhanced performance of decomposition-based EMO algorithms. We consider three decomposition-based many-objective evolutionary algorithms (EAs) - MOEA/D, MOEA/D-M2M, and NSGA-III - for this purpose. We also investigate another explicit control strategy of suitably choosing a normalization method of objectives for improving the performance of MOEA/D and MOEA/ D-M2M methods, and report much improved performance than their original counterparts. The principles of this study are valid for any population-based search and optimization algorithms and can be extended to improve the performance other single-objective EA, EMO, and other relevant CI methods.

Integration of Preferences in Decomposition Multiobjective Optimization

Rather than a whole Pareto-optimal front, which demands too many points (especially in a high-dimensional space), the decision maker (DM) may only be interested in a partial region, called the region of interest (ROI). In this case, solutions outside this region can be noisy to the decision-making procedure. Even worse, there is no guarantee that we can find the preferred solutions when tackling problems with complicated properties or many objectives. In this paper, we develop a systematic way to incorporate the DM's preference information into the decomposition-based evolutionary multiobjective optimization methods. Generally speaking, our basic idea is a nonuniform mapping scheme by which the originally evenly distributed reference points on a canonical simplex can be mapped to new positions close to the aspiration-level vector supplied by the DM. By this means, we are able to steer the search process toward the ROI either directly or interactively and also handle many objectives. Meanwhile, solutions lying on the boundary can be approximated as well given the DM's requirements. Furthermore, the extent of the ROI is intuitively understandable and controllable in a closed form. Extensive experiments on a variety of benchmark problems with 2 to 10 objectives, fully demonstrate the effectiveness of our proposed method for approximating the preferred solutions in the ROI.

Provisional-Ideal-Point-Based Multi-objective Optimization Method for Drone Delivery Problem

In this paper, we proposed a new evolutionary multi-objective optimization method for solving drone delivery problems (DDP). It can be formulated as a constrained multi-objective optimization problem. In our previous research, we proposed the “aspiration-point-based method” to solve multi-objective optimization problems. However, this method needs to calculate the optimal values of each objective function value in advance. Moreover, it does not consider the constraint conditions except for the objective functions. Therefore, it cannot apply to DDP which has many constraint conditions. To solve these issues, we proposed “provisional-ideal-point-based method.” The proposed method defines a “penalty value” to search for feasible solutions. It also defines a new reference solution named “provisional-ideal point” to search for the preferred solution for a decision maker. In this way, we can eliminate the preliminary calculations and its limited application scope. The results of the benchmark test problems show that the proposed method can generate the preferred solution efficiently. The usefulness of the proposed method is also demonstrated by applying it to DDP. As a result, the delivery path when combining one drone and one truck drastically reduces the traveling distance and the delivery time compared with the case of using only one truck.

Design of a Permanent Magnet Synchronous Generator Using Interactive Multiobjective Optimization

We consider an analytical model of a permanent magnet synchronous generator (PMSG) and formulate a mixed-integer constrained multiobjective optimization problem with six objective functions. We demonstrate the usefulness of solving such a problem by applying an interactive multiobjective optimization method called NIMBUS. In the NIMBUS method, a decision is iteratively involved in the optimization process and directs the solution process in order to find the most preferred Pareto optimal solution for the problem. We also employ a commonly used noninteractive evolutionary multiobjective optimization method called NSGA-II to generate a set of solutions that approximate the Pareto set and demonstrate the advantages of using an interactive method. This study is the first one to consider an interactive approach for the design of a PMSG. Thus, we promote the further usage of interactive multiobjective optimization methods in the design. Further, we see that these methods could also be very useful in the teaching of electrical machines.