Multi-objective Evolutionary Framework Research Articles

Multi-Objective Evolutionary Neural Architecture Search with Weight-Sharing Supernet

Deep neural networks have played a crucial role in the field of deep learning, achieving significant success in practical applications. The architecture of neural networks is key to their performance. In the past few years, these architectures have been manually designed by experts with rich domain knowledge. Additionally, the optimal neural network architecture can vary depending on specific tasks and data distributions. Neural Architecture Search (NAS) is a class of techniques aimed at automatically searching for and designing neural network architectures according to the given tasks and data. Specifically, evolutionary-computation-based NAS methods are known for their strong global search capability and have aroused widespread interest in recent years. Although evolutionary-computation-based NAS has achieved success in a wide range of research and applications, it still faces bottlenecks in training and evaluating a large number of individuals during optimization. In this study, we first devise a multi-objective evolutionary NAS framework based on a weight-sharing supernet to improve the search efficiency of traditional evolutionary-computation-based NAS. This framework combines the population optimization characteristic of evolutionary algorithms with the weight-sharing ideas in one-shot models. We then design a bi-population MOEA/D algorithm based on the proposed framework to effectively solve the NAS problem. By constructing two sub-populations with different optimization objectives, the algorithm can effectively explore network architectures of various sizes in complex search spaces. An inter-population communication mechanism further enhances the algorithm’s exploratory capability, enabling it to find network architectures with uniform distribution and high diversity. Finally, we conduct performance comparison experiments on image classification datasets of different scales and complexities. Experimental results demonstrate the effectiveness of the proposed multi-objective evolutionary NAS framework and the practicality and transferability of the introduced bi-population MOEA/D-based NAS method compared to existing state-of-the-art NAS methods.

Subset Selection for Evolutionary Multiobjective Optimization

Subset selection, which selects a subset of solutions according to certain criterion/indicator, is a topic closely related to evolutionary multi-objective optimization (EMO). The critical component of a multi-objective evolutionary algorithm (MOEA), environmental selection, is essentially a subset selection problem, i.e., selecting N solutions as the next-generation population from usually 2N candidates (where N denotes the size of the population). Another use of subset selection is the solution pre-processing procedure for decision-making, in which typically a few representatives are selected from the final population or a large-size archive, in order not to overwhelm the decision maker. Existing work for subset selection in EMO is focused on developing greedy algorithms, but may suffer from being trapped in local optima. In this paper, we approach the problem by providing a multi-objective evolutionary algorithmic framework. We consider several popular quality indicators for subset evaluation and present accelerated variants of a well-studied MOEA in the theoretical study area, Global Simple Evolutionary Multi-objective Optimizer (GSEMO), for each indicator. We conduct rigorous theoretical analyses of the acceleration procedure. Moreover, we prove that our algorithms can achieve the best-so-far approximation guarantee by the submodularity of the indicators. We finally empirically show the effectiveness and scalability of the proposed algorithms, in addition to the potentials to be further improved by introducing popular MOEAs (e.g., using NSGA-II and MOEA/D to replace GSEMO).

Higher-order Directed Community Detection by A Multi-objective Evolutionary Framework

Higher-order Directed Community Detection by A Multi-objective Evolutionary Framework

Distributed multi-objective optimization for SNP-SNP interaction detection

The detection of complex interactions between single nucleotide polymorphisms (SNPs) plays a vital role in genome-wide association analysis (GWAS). The multi-objective evolutionary algorithm is a promising technique for SNP-SNP interaction detection. However, as the scale of SNP data further increases, the exponentially growing search space gradually becomes the dominant factor, causing evolutionary algorithm (EA)-based approaches to fall into local optima. In addition, multi-objective genetic operations consume significant amounts of time and computational resources. To this end, this study proposes a distributed multi-objective evolutionary framework (DM-EF) to identify SNP-SNP interactions on large-scale datasets. DM-EF first partitions the entire search space into several subspaces based on a space-partitioning strategy, which is nondestructive because it guarantees that each feasible solution is assigned to a specific subspace. Thereafter, each subspace is optimized using a multi-objective EA optimizer, and all subspaces are optimized in parallel. A decomposition-based multi-objective firework optimizer (DCFWA) with several problem-guided operators was designed. Finally, the final output is selected from the Pareto-optimal solutions in the historical search of each subspace. DM-EF avoids the preference for a single objective function, handles the heavy computational burden, and enhances the diversity of the population to avoid local optima. Notably, DM-EF is load-balanced and scalable because it can flexibly partition the space according to the number of available computational nodes and problem size. Experiments on both artificial and real-world datasets demonstrate that the proposed method significantly improves the search speed and accuracy.

Evolutionary multi-objective design of autoencoders for compact representation of histopathology whole slide images

The recent success of deep learning coincides with the surge of digital pathology as an effect of the pandemic. However, processing a massive volume of gigapixel histopathology images is daunting. Hence, establishing compact image representations for efficient computational pathology becomes paramount. Deep autoencoders have been frequently used to compress feature vectors extracted from pre-trained networks. However, the topology of autoencoders is generally set heuristically. In this paper, we propose a multi-objective evolutionary framework as a bi-level optimization scheme in which the outer level evolves autoencoders for the goal of dimension reduction while the inner level is to optimize their weights for training. Throughout the evolution process, multiple minimization objectives such as complexity, classification error, and code size are considered. These objectives can obtain compressed models to generate a very small set of deep features with the highest classification accuracy. We use images from The Cancer Genome Atlas (TCGA) repository provided by the National Cancer Institute to train and validate our optimization framework. We increased the classification accuracy by 8% while the image representation is 46,000 times shorter. We apply the framework to demonstrate higher accuracy, substantial memory reduction and faster processing for image retrieval compared to SOTA.

Mitigating Unfairness via Evolutionary Multiobjective Ensemble Learning

In the literature of mitigating unfairness in machine learning, many fairness measures are designed to evaluate predictions of learning models and also utilised to guide the training of fair models. It has been theoretically and empirically shown that there exist conflicts and inconsistencies among accuracy and multiple fairness measures. Optimising one or several fairness measures may sacrifice or deteriorate other measures. Two key questions should be considered, how to simultaneously optimise accuracy and multiple fairness measures, and how to optimise all the considered fairness measures more effectively. In this paper, we view the mitigating unfairness problem as a multi-objective learning problem considering the conflicts among fairness measures. A multi-objective evolutionary learning framework is used to simultaneously optimise several metrics (including accuracy and multiple fairness measures) of machine learning models. Then, ensembles are constructed based on the learning models in order to automatically balance different metrics. Empirical results on eight well-known datasets demonstrate that compared with the state-of-the-art approaches for mitigating unfairness, our proposed algorithm can provide decision-makers with better tradeoffs among accuracy and multiple fairness metrics. Furthermore, the high-quality models generated by the framework can be used to construct an ensemble to automatically achieve a better tradeoff among all the considered fairness metrics than other ensemble methods. Our code is publicly available at https://github.com/qingquan63/FairEMOL

A knowledge-based constructive estimation of distribution algorithm for bi-objective portfolio optimization with cardinality constraints

Portfolio optimization is an essential and practical model for financial decision making. With the consideration of some real-world constraints, especially the cardinality constraints, the problem becomes much more challenging as it converts to a mixed-integer quadratic multi-objective optimization problem. To solve this problem, we propose a knowledge-based constructive estimation of distribution algorithm (KC-EDA) with the following three features. First, a hybrid design of Ant colony optimization (ACO) and Estimation distribution algorithm (EDA) is used to solve this mixed-variable optimization problem based on knowledge information. Second, a knowledge accumulation mechanism is designed to discover the internal relationship among the assets. The mechanism can not only guide the selection of assets effectively but also enable the use of historical information during evolution to direct the allocation of investment proportion. Third, a constructive approach is applied to construct portfolios under the constraints. This hybrid and constructive approach is incorporated into the multi-objective evolutionary framework and the experiment has been performed on the SZ50, SZ180, and SZ380 datasets (from January 2014 to December 2018). The experimental results demonstrate the effectiveness of KC-EDA in solving the portfolio optimization problem with cardinality constraints.

MSC-CSMC: A multi-objective semi-supervised clustering algorithm based on constraints selection and multi-source constraints for gene expression data.

Many clustering techniques have been proposed to group genes based on gene expression data. Among these methods, semi-supervised clustering techniques aim to improve clustering performance by incorporating supervisory information in the form of pairwise constraints. However, noisy constraints inevitably exist in the constraint set obtained on the practical unlabeled dataset, which degenerates the performance of semi-supervised clustering. Moreover, multiple information sources are not integrated into multi-source constraints to improve clustering quality. To this end, the research proposes a new multi-objective semi-supervised clustering algorithm based on constraints selection and multi-source constraints (MSC-CSMC) for unlabeled gene expression data. The proposed method first uses the gene expression data and the gene ontology (GO) that describes gene annotation information to form multi-source constraints. Then, the multi-source constraints are applied to the clustering by improving the constraint violation penalty weight in the semi-supervised clustering objective function. Furthermore, the constraints selection and cluster prototypes are put into the multi-objective evolutionary framework by adopting a mixed chromosome encoding strategy, which can select pairwise constraints suitable for clustering tasks through synergistic optimization to reduce the negative influence of noisy constraints. The proposed MSC-CSMC algorithm is testified using five benchmark gene expression datasets, and the results show that the proposed algorithm achieves superior performance.

Novel multi objective evolutionary framework for solving next release problem

Software developers want to meet the requirements of customers in next versions. Choosing which set of requirements can be done according to cost and time is an NP-hard problem known as Next Release Problem (NRP). In this article, a multi objective evolutionary algorithm (MOEA) framework is proposed to solve NRP. The framework applies the non-repetitive population, integrates solutions and external repository. Furthermore, a novel approach is implemented to satisfy the constraints of the problem. In this framework, six evolutionary algorithms are implemented and using seven quality indicators, the achieved results of that algorithms are compared with the original versions of same algorithms. Through using HV (the ratio of the region covered by Pareto Front) and NDS (the number of solutions in the Pareto Front) metrics, the effects of the proposed algorithms are compared with other works’ results. The efficacy of the proposed MOEA framework is measured using three real world datasets. The gained results represent that the implemented algorithms perform better than other related algorithms previously published.

Multiobjective Multiple Mobile Sink Scheduling via Evolutionary Fuzzy Rough Neural Network for Wireless Sensor Networks

The sensor nodes in wireless sensor networks have the deficiency of limited energy, and the multihop transmission of information will lead to a premature paralysis of nodes near the sink. The use of the mobile sink can balance the energy consumption and greatly prolong the lifetime. Therefore, this article studies the scheduling strategy of multiple mobile sinks and proposes a heuristic strategy based on interval type-2 fuzzy rough neural network. The energy and lifetime of sensor nodes, as well as location information of the mobile sink and special nodes are taken as input features. Through neural network learning, the outputs determine whether to move, moving direction, moving distance, and residence time, which can complete the scheduling task. The scheduling problem is regarded as a multiobjective optimization problem, and the network lifetime, the moving path length, and the network interpretability are optimized at the same time, so as to obtain a lightweight network with good interpretability and performance. Based on the parallel multiobjective evolutionary algorithm, a multiobjective neural evolutionary framework is constructed. This framework can balance multiple objectives and complete complex scheduling tasks. Compared with static sinks, random-moving sinks, sinks with manually designed strategy, gene expression programming-based sinks, as well as the other state-of-the-art multiobjective evolutionary algorithms, the proposed framework can achieve superior results.

Multiobjective Evolution of the Explainable Fuzzy Rough Neural Network With Gene Expression Programming

The fuzzy logic-based neural network usually forms fuzzy rules via multiplying the input membership degrees, which lacks expressiveness and flexibility. In this article, a novel neural network model is designed by integrating the gene expression programming into the interval type-2 fuzzy rough neural network, aiming to generate fuzzy rules with more expressiveness utilizing various logical operators. The network training is regarded as a multiobjective optimization problem through simultaneously considering network precision, explainability, and generalization. Specifically, the network complexity can be minimized to generate concise and few fuzzy rules for improving the network explainability. Inspired by the extreme learning machine and the broad learning system, an enhanced distributed parallel multiobjective evolutionary algorithm is proposed. This evolutionary algorithm can flexibly explore the forms of fuzzy rules, and the weight refinement of the final layer can significantly improve precision and convergence by solving the pseudoinverse. Experimental results show that the proposed multiobjective evolutionary network framework is superior in both effectiveness and explainability.

An optimized recommendation framework exploiting textual review based opinion mining for generating pleasantly surprising, novel yet relevant recommendations

Serendipity is a critical factor in the Recommender Systems (RS) in delivering pleasantly surprising, novel, yet contextually relevant recommendations. Most existing methods improve serendipity in RS by learning user preferences based on item popularity or similarity. However, the effectiveness of these methods in mitigating popularity bias and generating novel and unexpected item recommendations remains poorly understood. Recent studies suggest improvement in user preference by incorporating textual opinion provided by the user on an item. Additionally, the trade-off relationship between serendipity's conflicting components, including relevance, novelty, and unexpectedness, warrants further investigation to improve the quality of top-n recommendations. Hence, this research proposes an opinion mining-based approach to learn the users' personalized preferences from the textual reviews and incorporate both rating and reviews preferences to improve the quality of the recommendation list. Next, we design a new Two-Fold Algorithmic (TFA) approach-based objective function for serendipity to mitigate the popularity bias by aggregating uncertainty based on item popularity and item similarity to user preferences. Lastly, a multi-objective evolutionary algorithm-based Serendipity Objective Optimization-based Recommendation Framework(SOORF) is designed to optimize the serendipity's conflicting components. Extensive simulations are conducted over four benchmark datasets. The Mean Absolute Precision(MAP)@n and [email protected] based evaluation findings of SOORF and TFA demonstrate an improvement of at least 8.10% and 58.48%, respectively. The [email protected] and [email protected] based evaluations on different dataset sparsity conditions are observed with an improvement of at least 13.59% and 27.73% over baseline models. The Pareto front shows the models' ability to generate surprising, novel, yet relevant recommendations.

Adaptive Weighted Strategy Based Integrated Surrogate Models for Multiobjective Evolutionary Algorithm

Although the integrated model has good convergence ability, it is difficult to solve the multimodal problem and noisy problem due to the lack of uncertainty evaluation. Radial basis function model performs best for different degrees of nonlinear problems with small-scale and noisy training datasets but is insensitive to the increase of decision-space dimension, while Gaussian process regression model can provide prediction fitness and uncertainty evaluation. Therefore, an adaptive weighted strategy based integrated surrogate models is proposed to solve noisy multiobjective evolutionary problems. Based on the indicator-based multiobjective evolutionary framework, our proposed algorithm introduces the weighted combination of radial basis function and Gaussian process regression, and U-learning sampling scheme is adopted to improve the performance of population in convergence and diversity and judge the improvement of convergence and diversity. Finally, the effectiveness of the proposed algorithm is verified by 12 benchmark test problems, which are applied to the hybrid optimization problem on the construction of samples and the determination of parameters. The experimental results show that our proposed method is feasible and effective.

Broad learning approach to Surrogate-Assisted Multi-Objective evolutionary fuzzy clustering algorithm based on reference points for color image segmentation

For some real-world problems, the objective function evaluation is time-consuming and computationally expensive in multi-objective evolutionary algorithms. Surrogate-assistance approaches can be used to estimate the objective function and solve the computationally expensive problem. Taking advantages of the flatted structure and incremental learning, broad learning system (BLS) has shown a great improvement of time efficiency on the premise of ensuring the performance on the classification and regression problems. In this paper, BLS is adopted to design a BLS surrogate-assisted evolutionary framework to boost the time efficiency of clustering-based image segmentation. A BLS-assisted multi-objective evolutionary fuzzy clustering algorithm with reference points (BLS-MOEFC) is proposed. Firstly, two fuzzy clustering objective functions integrating image region information are constructed as the optimized fitness functions for obtaining satisfactory segmentation quality. Secondly, in order to improve the optimization efficiency and reduce the computational time cost, a BLS surrogate model-assisted multi-objective evolutionary framework with reference points is designed to optimize the constructed objective functions. Thirdly, an adaptively updating strategy of parameters and a model update mechanism based on the non-dominated sorting are adopted to improve the prediction accuracy of the BLS surrogate model. Finally, a novel fuzzy clustering validity index integrating region information is designed to select an optimal solution from the final set of non-dominated solutions. Experimental results on Berkeley and Weizmann images show that the proposed algorithm behaves well in the segmentation performance and time cost.

Rate–distortion optimal evolutionary algorithm for JPEG quantization with multiple rates

The JPEG standard is used extensively in image-related applications. The optimization on a JPEG-standard compressor is of high importance for image storage and transmission. In this paper, we propose a versatile and efficient quantization optimization method for image compression. It is based on multi-objective optimization and is JPEG standard-compatible. Unlike the existing optimization methods, our method fully considers the rate–distortion optimal principle and provides several optimal solutions to address the multiple-rate requirement in applications. In the multi-objective evolutionary optimization framework, the fitness of each quantization table for JPEG compression is evaluated efficiently by the searching in a look-up table, which is constructed based on the statistics of each DCT band in a pre-defined manner. Then, the population update strategy based on the rate–distortion optimal principle is recommended to guide the evolution toward the best rate–distortion performance. Furthermore, to maintain the population’s diversity and uniformity, convex-hull based environmental selection is recommended to identify the solutions at the first Pareto front, and the identified solutions are enriched further with scaling operation to fill the blank rate range. The experimental results for several classic datasets demonstrate the superiority of our method in terms of solution distribution, coding efficiency, and computational complexity.

A multipopulation evolutionary framework with Steffensen\u2019s method for dynamic multiobjective optimization problems

Dynamic multiobjective optimization problems (DMOPs) require the evolutionary algorithms that can track the moving Pareto-optimal fronts efficiently. This paper presents a dynamic multiobjective evolutionary framework (DMOEF-MS), which adopts a novel multipopulation structure and Steffensen’s method to solve DMOPs. In DMOEF-MS, only one population deals with the original DMOP, while the others focus on single-objective problems that are generated by the weighted summation of the original DMOP. Then, Steffensen’s method is used to control the evolving process in two ways: prediction and diversity-maintenance. Particularly, the prediction strategy is devised to predict the next promising positions for the individuals that handle single-objective problems, and the diversity-maintenance strategy is used to increase population diversity before the environment changes and reinitialize the multiple populations after the environment changes. This paper gives a comprehensive comparison of DMOEF-MS with some state-of-the-art DMOEAs on 14 DMOPs and the experimental results demonstrate the effectiveness of the proposed algorithm.

Payoff-based Dynamic Segment Replication and Graph Classification Method with Attribute Vectors Adapted to Urban VANET

Due to the number of constraints and the dynamic nature of vehicular ad hoc networks (VANET), effective video broadcasting always remains a difficult task. In this work, we proposed a quality of video visualization guarantee model based on a feedback loop and an efficient algorithm for segmenting and replicating video segments using the Payoff-based Dynamic Segment Replication Policy (P-DSR). In the urban VANET environment, P-DSR is defined by taking into account the position of the vehicles, the speed, the direction, the number of neighboring vehicles, and the reputation of each node to stabilize the urban VANET topology. However, the management of various load control parameters between the different components of the urban VANET network remains a problem to be studied. This work uses a multi-objective problem that takes the parameters of our algorithm based on the Graph Classification Method with Attribute Vectors (GCMAV) as input. This algorithm aims to provide an improved class lifetime, an improved video segment delivery rate, a reduced inter-class overload, and an optimization of a global criterion. A scalable algorithm is used to optimize the parameters of the GCMAV. The simulations were carried out using the NetSim simulator and Multi-Objective Evolutionary Algorithms framework to optimize parameters. Experiments were carried out with realistic maps of Open Street Maps and its results were compared with other algorithms such as Seamless and Authorized Multimedia Streaming and P-DSR. The survey suggests that the proposed methodology works well concerning the average lifetime of the inter-classes and the delivery rate of video segments.

A multi-objective evolutionary optimization framework for a natural gas liquids recovery unit

A simulation-based multi-objective optimization scheme is proposed for determining the optimal operating conditions of a natural gas liquids (NGL) recovery unit. Two objective functions are considered, the annualized profitability of the unit and the concentration of methane in the NGL product stream. Two problem formulations are studied including a deterministic model and a stochastic model which incorporates market uncertainty. The techno-economic framework combines the process simulation package PRO/II and a Python environment, in which the simulation status is tracked through the optimization. An evolutionary optimization algorithm simultaneously optimizes eight decision variables for constructing a 2-D Pareto front. Results provide insightful guidance on determining the most adequate conditions of a gas subcooled process (GSP) unit and portray an operational back-off which aims to reduce the impact introduced by market uncertainties.

A multi-objective sparse evolutionary framework for large-scale weapon target assignment based on a reward strategy

The asset-based weapon target assignment (ABWTA) problem is one of the important branches of the weapon target assignment (WTA) problem. Due to the current large-scale battlefield environment, the ABWTA problem is a multi-objective optimization problem (MOP) with strong constraints, large-scale and sparse properties. The novel model of the ABWTA problem with the operation error parameter is established. An evolutionary algorithm for large-scale sparse problems (SparseEA) is introduced as the main framework for solving large-scale sparse ABWTA problem. The proposed framework (SparseEA-ABWTA) mainly addresses the issue that problem-specific initialization method and genetic operators with a reward strategy can generate solutions efficiently considering the sparsity of variables and an improved non-dominated solution selection method is presented to handle the constraints. Under the premise of constructing large-scale cases by the specific case generator, two numerical experiments on four outstanding multi-objective evolutionary algorithms (MOEAs) show Runtime of SparseEA-ABWTA is faster nearly 50% than others under the same convergence and the gap between MOEAs improved by the mechanism of SparseEA-ABWTA and SparseEA-ABWTA is reduced to nearly 20% in the convergence and distribution.

Genetic-based web regression testing: an ontology-based multi-objective evolutionary framework to auto-regression testing of web applications

Genetic-based web regression testing: an ontology-based multi-objective evolutionary framework to auto-regression testing of web applications