Evolutionary Multi-objective Method Research Articles

A Multiobjective Genetic Algorithm to Evolving Local Interpretable Model-Agnostic Explanations for Deep Neural Networks in Image Classification

Deep convolutional neural networks have become a dominant solution for numerous image classification tasks. However, a main criticism is the poor explainability due to the black-box characteristic, which hurdles the extensive usage of deep convolutional neural networks. To address this issue, this paper proposes a new evolutionary multi-objective based method, which aims to explain the behaviours of deep convolutional neural networks by evolving local explanations on specific images. To the best of our knowledge, this is the first evolutionary multi-objective method to evolve local explanations. The proposed method is model-agnostic, i.e. it is applicable to explain any deep convolutional neural networks. ImageNet is used to examine the effectiveness of the proposed method. Three well-known deep convolutional neural networks -VGGNet, ResNet, and MobileNet, are chosen to demonstrate the modelagnostic characteristic. Based on the experimental results, it can be observed that the local explanations are understandable to end-users, who need to check the sensibility of the evolved explanations to decide whether to trust the predictions made by the deep convolutional neural networks. Furthermore, the local explanations evolved by the proposed method improves the confidence of deep convolutional neural networks making the predictions. Lastly, the pareto front and convergence analyses indicate that the proposed method can form a good set of nondominated solutions.

Terminal Trajectory Planning for Synthetic Aperture Radar Imaging Guidance Based on Chronological Iterative Search Framework.

Synthetic aperture radar (SAR) is capable of obtaining the high-resolution 2-D image of the interested target scene, which enables advanced remote sensing and military applications, such as missile terminal guidance. In this article, the terminal trajectory planning for SAR imaging guidance is first investigated. It is found that the guidance performance of an attack platform is determined by the adopted terminal trajectory. Therefore, the aim of the terminal trajectory planning is to generate a set of feasible flight paths to guide the attack platform toward the target and meanwhile obtain the optimized SAR imaging performance for enhanced guidance precision. The trajectory planning is then modeled as a constrained multiobjective optimization problem given a high-dimensional search space, where the trajectory control and SAR imaging performance are comprehensively considered. By utilizing the temporal-order-dependent property of the trajectory planning problem, a chronological iterative search framework (CISF) is proposed. The problem is decomposed into a series of subproblems, where the search space, objective functions, and constraints are reformulated in chronological order. The difficulty of solving the trajectory planning problem is thus significantly alleviated. Then, the search strategy of CISF is devised to solve the subproblems successively. The optimization results of the preceding subproblem can be utilized as the initial input of the subsequent subproblems to enhance the convergence and search performance. Finally, a trajectory planning method is put forward based on CISF. Experimental studies demonstrate the effectiveness and superiority of the proposed CISF compared with the state-of-the-art multiobjective evolutionary methods. The proposed trajectory planning method can generate a set of feasible terminal trajectories with optimized mission performance.

Improvement of signal communication for a foraging task using evolutionary robotics

Communication systems represent an evolutionary advantage for a group of robots solving coordinated tasks. In the field of evolutionary robotics, the emergence and establishment of communication are regulated by different variables. These systems are a tool produced by artificial evolution for the exchange of personal and environmental information. Since evolutionary processes involve multi-objective evolutionary methods, it is important to study all the mechanisms that affect the emergence and establishment of communication systems. One of these variables is the evolutionary advantage of emerging signals. In this article it is assumed that the signals appear because they are evolutionarily useful in solving a task. Emitter and receiver generate a process of conceptualization, which makes the signals associated with a meaning for the community. In this way an experiment is adapted to the FARSA simulator and the MARXBOT robot. This experiment consists of a group of robots that spend as much time as possible in a food zone and avoid a poisoned zone. Under normal conditions, robots tend to point to the food zone to attract the rest of the population. When the evolutionary importance of pointing out the food zone is reduced, the signals arise in different situations such as the presence of nearby objects and the poisoned zone. Communication is configured with the LED rings and linear cameras of the robots. In addition, an ethological method is adapted to quantify the behavioral effects of experimental manipulation.

Enhancing multi-objective evolutionary algorithms with machine learning for scheduling problems: recent advances and survey

Multi-objective scheduling problems in workshops are commonly encountered challenges in the increasingly competitive market economy. These scheduling problems require a trade-off among multiple objectives such as time, energy consumption, and product quality. The importance of each optimization objective typically varies in different time periods or contexts, necessitating decision-makers to devise optimal scheduling plans accordingly. In actual production, decision-makers confront intricate multi-objective scheduling problems that demand balancing clients’ requirements and corporate interests while concurrently striving to reduce production cycles and costs. In solving various problems, multi-objective evolutionary algorithms have attracted the attention of researchers and gradually become one of the mainstream methods to solve these problems. In recent years, research combining multi-objective evolutionary algorithms with machine learning technology has shown great potential, opening up new prospects for improving the performance of multi-objective evolutionary methods. This article comprehensively reviews the latest application progress of machine learning in multi-objective evolutionary algorithms for scheduling problems. We review various machine learning techniques employed for enhancing multi-objective evolutionary algorithms, particularly focusing on different types of reinforcement learning methods. Different categories of scheduling problems addressed using these methods were also discussed, including flow-shop scheduling issues, job-shop scheduling challenges, and more. Finally, we highlighted the challenges faced by the field and outlined future research directions.

An evolutionary multiobjective method based on dominance and decomposition for feature selection in classification

An evolutionary multiobjective method based on dominance and decomposition for feature selection in classification

Multiobjective Evolutionary Learning for Multitask Quality Prediction Problems in Continuous Annealing Process.

In industrial production processes, the mechanical properties of materials will directly determine the stability and consistency of product quality. However, detecting the current mechanical property is time-consuming and labor-intensive, and the material quality cannot be controlled in time. To achieve high-quality steel materials, developing a novel intelligent manufacturing technology that can satisfy multitask predictions for material properties has become a new research trend. This article proposes a multiobjective evolutionary learning method based on a two-stage model with topological sparse autoencoder (TSAE) and ensemble learning. For the structure characteristics of a typical autoencoder (AE), a topology-related constraint is incorporated into the loss function of the AE, thus maintaining the global relationship among multistage input data to improve the data reconstruction quality. Then, a sparse representation of the data is added to the AE to achieve dimensionality reduction. Moreover, the extreme gradient boosting (XGBoost) method is applied to predict the mechanical properties of steel materials through collaboration learning mechanisms. To enhance the model accuracy, a multiobjective evolutionary algorithm (MOEA) with a knee solution strategy is used to optimize the network structure and hyperparameters of the two-stage model. Experiments are conducted using real steel production data from a continuous annealing process (CAP). The results verify that the proposed method obtains a higher prediction accuracy than other state-of-the-art methods and can guide practical production and new material design.

Evolutionary auto-design for aircraft engine cycle

Traditional engine cycle innovation is limited by human experiences, imagination, and currently available engine component performance expectations. Thus, the engine cycle innovation process is quite slow for the past 90 years. In this work, we propose a mixed variable multi-objective evolutionary optimization method for automatic engine cycle design. In the first, a simulation toolkit is developed for performance evaluation of potentially viable engine cycle solutions. Then, the engine cycle solutions are mixed encoded by the pins and the parameters of different engine components. The new engine cycle solutions are generated through the mutation operator. Finally, we construct two optimization objectives to drive the optimization process. Through the experimental research, new engine cycle solutions are discovered that exceed the performance of known turbojet and turbofan engines.

Efficient multi-objective evolutionary neural architecture search for U-Nets with diamond atrous convolution and Transformer for medical image segmentation

Deep encoder–decoder neural networks like U-Nets have made significant contributions to the development of computer vision applications such as image segmentation. Neural architecture search (NAS) has the potential to further automatically adjust the architectures of U-Nets for various medical image segmentation tasks. Most of the NAS techniques focus on optimizing segmentation accuracies of network architectures. In real-world medical image segmentation scenarios, two main challenges are poor medical image quality and diverse deployment devices with different computing capabilities. A large architecture designed only for the high segmentation accuracy is difficult to run on various deployment devices. To address these challenges, this paper proposes a multi-objective evolutionary neural architecture search method (CTU-NAS) for U-Nets with diamond atrous convolution and Transformer for medical image segmentation. A hybrid U-Net architecture (CTU-Net) with diamond atrous convolution and Transformer modules is designed as the supernet of CTU-NAS. Then a channel search strategy based on sorting and selection is applied to speed up the search for subnets by precisely selecting and training the most important channels more times. In addition, CTU-NAS employs a dual acceleration mechanism based on weight sharing and surrogate model to lower the cost of evaluations of subnets. CTU-NAS applies a multi-objective evolutionary algorithm to balance between the segmentation accuracy and the number of parameters. Experimental results on two medical segmentation datasets show that CTU-NAS is capable of quickly generating a group of excellent network architectures with different sizes and their performances also outperform or come close to those of the manually designed networks.

Feature library-assisted surrogate model for evolutionary wrapper-based feature selection and classification

In recent years, wrapper-based feature selection (FS) using evolutionary algorithms has been widely studied due to its ability to search for and evaluate subsets of features based on populations. However, these methods often suffer from a high computational cost and a long computation time, mainly due to the process of evaluating the feature subsets according to the classification performance. In order to tackle this problem, this paper presents a feature library-assisted surrogate model (FL-SM), which aims to reduce the computational cost but maintain a good prediction accuracy. Unlike the existing surrogate models used in FS, the proposed method focuses on the feature level instead of the sample level: an FL is built by collecting the scores of all the features during the evolutionary search. Specifically, each solution (subset candidate) is pre-evaluated based on the FL using only simple operations to decide whether or not it deserves to be evaluated by the classifier, improving the efficiency of the FS algorithm. Meanwhile, because not evaluating a certain number of solutions may lead to inaccurate solution selection during the evolutionary search, dynamic individual selection criteria are proposed. In addition, an adaptive FL update operator is proposed to handle the dynamics of the evolved population; it ensures the real-time validity of the FL. Furthermore, we incorporate the proposed FL-SM into some state-of-the-art single- and multi-objective evolutionary FS methods. The experimental results on benchmark datasets show that with good flexibility and extendibility, FL-SM can effectively reduce the computational cost of wrapper-based FS and still obtain high-quality feature subsets. Among the five algorithms tested, the average computation time reduction was 34.87%; at the same time, there was no significant difference in the classification accuracy for 80% of the tests, and our method even improved the classification accuracy for 6% of the tests.

Feature Selection Using Diversity-Based Multi-objective Binary Differential Evolution

By identifying relevant features from the original data, feature selection methods can maintain or improve the classification accuracy and reduce the dimensionality. Recently, many multi-objective evolutionary methods have been proposed for feature selection. However, effectively handling the trade-offs between convergence and diversity of the non-dominated solutions remains a major challenge, especially for high-dimensional datasets. To cover this issue, this work studies a diversity-based multi-objective differential evolution approach to feature selection. During the environmental selection process, each of the solutions in the candidate pool will have a diversity score, and solutions with large diversity score values will be preferred so as to improve the population diversity. To reduce the search space, irrelevant and weakly relevant features are detected and removed in the proposed method. A new binary mutation operator using the neighborhood information of individuals is also proposed, aiming to produce better feature subsets. Experimental results on 14 datasets with varying difficulties show that the proposed feature selection method can obtain significantly better feature selection performance than current popular multi-objective feature selection methods.

Explainable Regression Via Prototypes

Model interpretability/explainability is increasingly a concern when applying machine learning to real-world problems. In this article, we are interested in explaining regression models by exploiting prototypes, which are exemplar cases in the problem domain. Previous works focused on finding prototypes that are representative of all training data but ignore the model predictions, i.e., they explain the data distribution but not necessarily the predictions. We propose a two-level model-agnostic method that considers prototypes to provide global and local explanations for regression problems and that account for both the input features and the model output. M-PEER (Multiobjective Prototype-basEd Explanation for Regression) is based on a multi-objective evolutionary method that optimizes both the error of the explainable model and two other “semantics”-based measures of interpretability adapted from the context of classification, namely, model fidelity and stability. We compare the proposed method with the state-of-the-art method based on prototypes for explanation—ProtoDash—and with other methods widely used in correlated areas of machine learning, such as instance selection and clustering. We conduct experiments on 25 datasets, and results demonstrate significant gains of M-PEER over other strategies, with an average of 12% improvement in the proposed metrics (i.e., model fidelity and stability) and 17% in root mean squared error (RMSE) when compared to ProtoDash.

Sophisticated methods for noise filtering, subgroup discovery, and classification in big data analysis

Big Data is a popular research area where a vast amount of data is created, replicated, and consumed by society. The quality of the data used directly influences big data knowledge discovery. The existence of noise is the most prevalent problem influencing data quality. The following techniques were developed to reduce noise in data with a distributed setting: Homogenous Ensemble for Big Data (HME-BD) and Heterogeneous Ensemble for Big Data (HTE-BD). In this article, the performance of HTE-BD is improved further by developing Enhanced HTE-BD (EHTE-BD), which combines Logistic Regression based Support Vector Machine (LR-SVM) in conjunction with RF, LR, and KNN to reduce noisy data. Furthermore, the Multi-Objective Evolutionary Fuzzy Method for Subgroup Discovery throughout Big Data (MEFASD-BD) was used to resolve the multi-objective optimization challenge, and the Non-Dominated Sorting Genetic Algorithm-II (NSGA-II) was utilized to handle the rising dimensionality issue through subgroup discovery. To address the NSGA-II’s slow convergence rate, an Improved Multi-Objective Meta-Heuristic Fuzzy approach for discovering subgroups in big data is described, that contains a meta-heuristic method for subgroup discovery known as the Multi-Objective Differential Search Algorithm (MODSA). It selects the most relevant subgroups from vast amounts of data, reducing the data’s dimensionality. The Fuzzy Deep Neural Network (FDNN) classifier assesses the main subgroups. By removing noisy data and selecting the most relevant subgroups, the performance of FDNN in classifying vast amounts of data is improved.

An environment-driven hybrid evolutionary algorithm for dynamic multi-objective optimization problems

In dynamic multi-objective optimization problems, the environmental parameters may change over time, which makes the Pareto fronts shifting. To address the issue, a common idea is to track the moving Pareto front once an environmental change occurs. However, it might be hard to obtain the Pareto optimal solutions if the environment changes rapidly. Moreover, it may be costly to implement a new solution. By contrast, robust Pareto optimization over time provides a novel framework to find the robust solutions whose performance is acceptable for more than one environment, which not only saves the computational costs for tracking solutions, but also minimizes the cost for switching solutions. However, neither of the above two approaches can balance between the quality of the obtained non-dominated solutions and the computation cost. To address this issue, environment-driven hybrid dynamic multi-objective evolutionary optimization method is proposed, aiming to fully use strengths of TMO and RPOOT under various characteristics of environmental changes. Two indexes, i.e., the frequency and intensity of environmental changes, are first defined. Then, a criterion is presented based on the characteristics of dynamic environments and the switching cost of solutions, to select an appropriate optimization method in a given environment. The experimental results on a set of dynamic benchmark functions indicate that the proposed hybrid dynamic multi-objective evolutionary optimization method can choose the most rational method that meets the requirements of decision makers, and balance the convergence and robustness of the obtained non-dominated solutions.

Application Analysis of Multi-Intelligence Optimization Decision-Making Method in College Students’ Ideological and Political Education System

Today’s society is a society with diversified information. The rapid change of information also affects the ideological and political education in universities. Therefore, the fixed thinking of ideological and political education in universities tells students, which is an educational way in line with the development of the times. We should follow up with the rapidly changing times at any time, change with the times, and update the thinking and systematic way of the ideological and political education system at any time. In the ideological and political education of college students, diversified teaching methods and multiobjective recommendation systems are implemented and then combined with traditional ideological and political teaching methods. The two ways complement each other and promote each other so as to achieve higher learning efficiency and better learning effect. The optimization algorithm of multiobjective recommendation should be used to further improve the ideological and political education system. By analyzing the optimization results and performance comparison of various algorithms, we find the most suitable algorithm model for optimizing the ideological and political education system. The multiobjective-recommended ideological and political education system for college students needs to fully improve the teaching tasks of teachers and students in two stages. A reasonable and scientific system recommendation mechanism should take into account students’ own learning preferences, subject types, ideological and political teacher information, curriculum information, curriculum evaluation, curriculum relevance, and other multiobjective data. This paper achieves the highest performance and the lowest time cost of the ideological and political education system through multiobjective evolutionary optimization method.

Multi-objective evolutionary optimization of unsupervised latent variables of turning process

Manufacturing process modeling and optimization is a challenging task due to the numerous objectives to be considered in the optimization. Generally, the optimization of these processes requires many objective optimization methods to deal with four or more objective functions. However, the correlation structure of the outputs cannot be disregarded. In this work, it is proposed the unsupervised learning of the outputs together with multi-objective evolutionary optimization of the turning process of AISI 4340 steel considering three scenarios varying the tool nose radius. A central composite design varying the process parameters is used to conduct the experimental tests. After tests and measurements of quality and productivity outputs the p correlated observed outputs are firstly transformed in m unobserved latent variables through factor analysis using principal axis extraction method and varimax rotation, with m<p. Subsequently, the relation between the process parameters and the scores of latent variables is modeled through response surface methodology. Multi-objective evolutionary optimization methods are applied in the reduced and uncorrelated set of response models of the transformed outputs. The multi-objective algorithms are compared through hypervolume metric and the pseudo-weights approach is used to decision making. The proposed method can also be applied in other multi-response processes with correlated outputs.

Pareto optimal directional drilling advisory for improved real-time decision making

Directional drilling decision making is complicated by the presence of multiple objectives and constraints. Existing directional drilling advisory methods find a balanced solution through optimization over a weighted sum of the objectives, but the weights are often chosen arbitrarily or based on rough estimates. In addition, existing methods only return a single solution to a multi-objective problem that likely does not have a single, best solution. The work presented here overcomes these limitations by identifying a set of feasible, diversely performing (i.e., Pareto optimal) solutions for the directional driller to choose from. In this novel approach to directional drilling steering optimization, the directional drilling steering problem is framed as a constrained optimal control problem over a finite horizon. A wellbore propagation model that is calibrated at set interval using field data was used to estimate system responses for a bent-sub mud motor drilling assembly. Pareto front approximation was used to solve the problem and a multi-objective evolutionary search method was leveraged to solve this optimization problem and present the directional driller with a set of Pareto optimal solutions that satisfy the problem constraints. The proposed Pareto front approximation approach was validated against a set of field test cases. These demonstrate that the proposed method is capable of finding feasible, high-performing solutions to these real-world problems sufficiently fast to be of use for real-time directional drilling advisory in the field. • First application of Pareto front approximation to the directional drilling steering optimization problem.

Modular unmanned aerial vehicle platform design: Multi-objective evolutionary system method

For the air quality data collection, a great potential is offered by unmanned aerial vehicles (UAV) with spatial and temporal resolutions. The fast scientific development has enabled the class production of UAVs cost effective in which civilian applications are serviced. A modular UAV-based platform design and development are the objectives of this study. The system comprises of five modules like the UAV, the ground station, sensors, data acquisition (DA) module, and the data fusion (DF) module. With adequate flight time, UAV platform can fly on pathways which are predetermined as shown by the results of data collection missions. The high precision data and air quality are collected through the system simultaneously. In a real manner, they are integrated and visualized. The UAV system power consumption is measured on ground and flight. The consumed current reached about 30 A when the UAV was in the air regardless the speed.

Negative Pressure Wave-Based Method for Abnormal Signal Location in Energy Transportation System

Abnormal signal location is an indispensable part to handle unexpected events such as pipeline leak in energy transportation system. However, unclear signal characteristics would make traditional location methods such as wavelet analysis failure in location process. To address it, a data-driven method based on negative pressure wave (NPW) is proposed in this article. First, a signal enhancement model based on Gaussian process regression (GPR) is proposed to highlight the changed signal characteristics. Then, a multi-objective evolutionary computation method is proposed to obtain the location result, which overcomes the shortcoming of unclear signal characteristics. Moreover, the signal transmission attenuation rules are added into the location process, which ensures the rationality and the reliability of the location result. Without introducing extra signal numbers and types, the proposed method could achieve higher accuracy and more robustness than traditional location methods such as wavelet analysis, which has the less 1.2% location deviation of the pipeline. Finally, different experiment results show that the proposed method is a realistic way to provide the position of abnormal signal source.

Multi-objective evolutionary optimization based on online perceiving Pareto front characteristics

When solving multi-objective optimization problems with complex Pareto fronts characteristics, previous work generally ignores the information related to Pareto fronts provided by the population during the evolution, which is detrimental to efficiently tackle them. In order to taking full advantage of information associated with a population evolution, a multi-objective evolutionary optimization method based on online perceiving Pareto front characteristics is proposed in this study. To this end, the information associated with the Pareto front of an optimization problem is first extracted from the population. Following that, the characteristics of the Pareto front in concavity/convexity and continuity are perceived online. For the purpose of each sub-front containing only one characteristic, a Pareto front is divided based on the concavity/convexity and continuity. According to the characteristic of each sub-front, different reference points are selected to refine the distribution of reference vectors. Finally, a multi-objective evolutionary algorithm is designed targeting the characteristics of the Pareto front. The performance of the proposed method is evaluated by comparing it with 8 state-of-the-art optimizers on 31 test problems. Further, the experimental results demonstrate that the proposed method is competitive in handling multi-objective optimization problems with irregular Pareto fronts.

Multiobjective Particle Swarm Optimization for Feature Selection With Fuzzy Cost.

Feature selection (FS) is an important data processing technique in the field of machine learning. There have been various FS methods, but all assume that the cost associated with a feature is precise, which restricts their real applications. Focusing on the FS problem with fuzzy cost, a fuzzy multiobjective FS method with particle swarm optimization, called PSOMOFS, is studied in this article. The proposed method develops a fuzzy dominance relationship to compare the goodness of candidate particles and defines a fuzzy crowding distance measure to prune the elitist archive and determine the global leader of particles. Also, a tolerance coefficient is introduced into the proposed method to ensure that the Pareto-optimal solutions obtained satisfy decision makers' preferences. The developed method is used to tackle a series of the UCI datasets and is compared with three fuzzy multiobjective evolutionary methods and three typical multiobjective FS methods. Experimental results show that the proposed method can achieve feature sets with superior performances in approximation, diversity, and feature cost.