Multimodal Multi-objective Optimization Research Articles

Decomposition in decision and objective space for multi-modal multi-objective optimization

Multi-modal multi-objective optimization problems (MMMOPs) have multiple subsets within the Pareto-optimal Set, each independently mapping to the same Pareto-Front. Prevalent multi-objective evolutionary algorithms are not purely designed to search for multiple solution subsets, whereas, algorithms designed for MMMOPs demonstrate degraded performance in the objective space. This motivates the design of better algorithms for addressing MMMOPs. The present work identifies the crowding illusion problem originating from using crowding distance globally over the entire decision space. Subsequently, an evolutionary framework, called graph Laplacian based Optimization using Reference vector assisted Decomposition (LORD), is proposed, which uses decomposition in both objective and decision space for dealing with MMMOPs. Its filtering step is further extended to present LORD-II algorithm, which demonstrates its dynamics on multi-modal many-objective problems. The efficacies of the frameworks are established by comparing their performance on test instances from the CEC 2019 multi-modal multi-objective test suite and polygon problems with the state-of-the-art algorithms for MMMOPs and other multi- and many-objective evolutionary algorithms. The manuscript is concluded by mentioning the limitations of the proposed frameworks and future directions to design still better algorithms for MMMOPs. The source code is available at https://worksupplements.droppages.com/lord.

An Evolutionary Algorithm with Clustering-Based Assisted Selection Strategy for Multimodal Multiobjective Optimization

In multimodal multiobjective optimization problems (MMOPs), multiple Pareto optimal sets, even some good local Pareto optimal sets, should be reserved, which can provide more choices for decision-makers. To solve MMOPs, this paper proposes an evolutionary algorithm with clustering-based assisted selection strategy for multimodal multiobjective optimization, in which the addition operator and deletion operator are proposed to comprehensively consider the diversity in both decision and objective spaces. Specifically, in decision space, the union population is partitioned into multiple clusters by using a density-based clustering method, aiming to assist the addition operator to strengthen the population diversity. Then, a number of weight vectors are adopted to divide population into N subregions in objective space (N is population size). Moreover, in the deletion operator, the solutions in the most crowded subregion are first collected into previous clusters, and then the worst solution in the most crowded cluster is deleted until there are N solutions left. Our algorithm is compared with other multimodal multiobjective evolutionary algorithms on the well-known benchmark MMOPs. Numerical experiments report the effectiveness and advantages of our proposed algorithm.

A Spatial Learning Based SGSO with Emotional Tracking for Multimodal Multi-objective Optimization

A Spatial Learning Based SGSO with Emotional Tracking for Multimodal Multi-objective Optimization

A Multipopulation Evolutionary Algorithm for Solving Large-Scale Multimodal Multiobjective Optimization Problems

Multimodal multiobjective optimization problems (MMOPs) widely exist in real-world applications, which have multiple equivalent Pareto-optimal solutions that are similar in the objective space but totally different in the decision space. While some evolutionary algorithms (EAs) have been developed to find the equivalent Pareto-optimal solutions in recent years, they are ineffective to handle large-scale MMOPs having a large number of variables. This article thus proposes an EA for solving large-scale MMOPs with sparse Pareto-optimal solutions, i.e., most variables in the optimal solutions are 0. The proposed algorithm explores different regions of the decision space via multiple subpopulations and guides the search behavior of the subpopulations via adaptively updated guiding vectors. The guiding vector for each subpopulation not only provides efficient convergence in the huge search space but also differentiates its search direction from others to handle the multimodality. While most existing EAs solve MMOPs with 2-7 decision variables, the proposed algorithm is shown to be effective for benchmark MMOPs with up to 500 decision variables. Moreover, the proposed algorithm also produces a better result than state-of-the-art methods for the neural architecture search.

Multi-objective evolution strategy for multimodal multi-objective optimization

In the past decades, various effective and efficient multi-objective evolutionary algorithms (MOEAs) have been proposed for solving multi-objective optimization problems. However, existing MOEAs cannot satisfactorily address multimodal multi-objective optimization problems that demand to find multiple groups of optimal solutions simultaneously. In this paper, we propose an evolution strategy to solve multimodal multi-objective optimization problems, named MMO-MOES. This paper focus on searching for well-converged and well-distributed solutions in the decision space. Firstly, a novel niching strategy in the decision space, which imitates the repulsive force among isotropic magnetic particles, is adopted to drive the individuals to preserve uniform distances from each other and spread to the whole Pareto set automatically. This strategy is effective in finding multiple groups of optimal solutions simultaneously. Secondly, MMO-MOES requires only a very small population size to obtain a well-distributed and well-converged set of Pareto optimal solutions in the decision space. The greater the population size, the clearer contour of the approximate Pareto sets and Pareto front will be. Finally, the MMO-MOES is compared against some chosen leading-edge MMOEAs. The experimental results demonstrate that MMO-MOES provides exceptional performance in searching for the complete Pareto subsets and Pareto front on Omni-test problem, Symmetrical Parts (SYM-PART) problems, and CEC 2019 Multimodal Multi-Objective Optimization Problems (MMOPs) test suite.

Incorporation of multimodal multiobjective optimization in designing a filter based feature selection technique

Current work reports about the development of a new feature selection technique fusing two concepts, multimodal multiobjective optimization and filter-based feature selection. Use of the concept of multimodality in multiobjective based feature selection helps in generating diverse set of feature subsets on final Pareto front. This approach evaluates the quality of the reduced feature set by utilizing different quality measures. This process of feature selection focuses on achieving two discrete objectives: (1) identification of a large number of Pareto-optimal solutions along with achieving a good distribution in both objective and decision spaces; (2) making a selection of feature subset with minimal redundancy and high correlation with classes. To achieve the second objective, a variety of objective functions based on information-theoretic measures like normalized mutual information and correlation with class attributes are utilized. A multiobjective ring-based Particle swarm Optimization (PSO) and non-dominated sorting with special crowding distance are employed to cover the aspects corresponding to the first objective. An evaluation is carried out on seven publicly available datasets concerning different classifiers. The results of these experiments illustrate that the multimodal PSO based feature selection approach finds more feature subsets than its simple PSO counterpart in multiobjective environment. And, the results are also compared with those of existing wrapper based multimodal multiobjective feature selection methods.

A clustering-based differential evolution algorithm for solving multimodal multi-objective optimization problems

Abstract Multimodal Multi-objective Optimization Problems (MMOPs) refer to the problems that have multiple Pareto-optimal solution sets in decision space corresponding to the same or similar Pareto-optimal front in objective space. These problems require the optimization algorithm to locate multiple Pareto Sets (PSs). This paper proposes a differential evolution algorithm based on the clustering technique and an elite selection mechanism to solve MMOPs. In this algorithm, a Clustering-based Special Crowding Distance (CSCD) method is designed to calculate the comprehensive crowding degree in decision and objective spaces. Subsequently, a distance-based elite selection mechanism (DBESM) is introduced to determine the learning exemplars of various individuals. New individuals are generated around the exemplars to obtain a well-distributed population in both decision and objective spaces. To test the performance of the proposed algorithm, extensive experiments on the suit of CEC'2019 benchmark functions have been conducted. The results indicate that the proposed method has superior performance compared with other commonly used algorithms.

A modified particle swarm optimization for multimodal multi-objective optimization

As an effective evolutionary algorithm, particle swarm optimization (PSO) has been widely used to solve single or multi-objective optimization problems. However, the performance of PSO in solving multi-objective problems is unsatisfactory, so a variety of PSO has been proposed to enhance the performance of PSO on multi-objective optimization problems. In this paper, a modified particle swarm optimization (AMPSO) is proposed to solve the multimodal multi-objective problems. Firstly, a dynamic neighborhood-based learning strategy is introduced to replace the global learning strategy, which enhances the diversity of the population. Meanwhile, to enhance the performance of PSO, the offering competition mechanism is utilized. 11 multimodal multi-objective optimization functions are utilized to verify the feasibility and effectiveness of the proposed AMPSO. Experimental results and statistical analysis indicate that AMPSO has competitive performance compared with 5 state-of-the-art multimodal multi-objective algorithms.

Knee based multimodal multi-objective evolutionary algorithm for decision making

Recently, many advanced multimodal multi-objective evolutionary algorithms (MMOEAs) have been presented to solve multimodal multi-objective optimization problems (MMOPs). However, most of these approaches struggle not only to find the entire Pareto sets and Pareto front, but also ignoring the problem of Multi-Criteria Decision Making (MCDM) to be handled at a later stage. As a matter of fact, it is a very eclectic and challenging burden for decision makers to choose some trade-off optimal solutions from an enormous collection of non-dominated Pareto-optimal individuals. In this paper, we propose a knee based evolutionary algorithm, named MMO-EvoKnee, which incorporates MCDM strategy into solving MMOPs. The proposed algorithm is designed to search for a complete set of global knee solutions instead of an entire Pareto front and Pareto sets. Firstly, the EvoKnee Selection is adopted to find target global knee solutions and boundary individuals. Secondly, the Knee Multimodal Solutions Selection focuses on decision space exploitation to obtain well-converged multiple Pareto-optimal knee solutions. Thirdly, the Maximum Spanning Subset Selection is designed to accurately identify a complete number of knee solutions from overcrowded Pareto regions. Fourthly, the Boundary Multimodal Solutions Selection can find and maintain well-diversified and well-converged boundary solutions. As a result, MMO-EvoKnee can consistently find all knee solutions of interest quickly, and it relieves the burden for decision makers. Finally, the performance of the proposed algorithm is evaluated on thirteen benchmark MMOPs. The experiment results clearly show that the proposed MMO-EvoKnee provides a competitive edge over the chosen state-of-the-art MMOEAs.

Multimodal Multiobjective Evolutionary Optimization With Dual Clustering in Decision and Objective Spaces

This article suggests a multimodal multiobjective evolutionary algorithm with dual clustering in decision and objective spaces. One clustering is run in decision space to gather nearby solutions, which will classify solutions into multiple local clusters. Nondominated solutions within each local cluster are first selected to maintain local Pareto sets, and the remaining ones with good convergence in objective space are also selected, which will form a temporary population with more than <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${N}$ </tex-math></inline-formula> solutions ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${N}$ </tex-math></inline-formula> is the population size). After that, a second clustering is run in objective space for this temporary population to get <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${N}$ </tex-math></inline-formula> final clusters with good diversity in objective space. Finally, a pruning process is repeatedly run on the above clusters until each cluster has only one solution, which removes the most crowded solution in decision space from the most crowded cluster in objective space each time. This way, the clustering in decision space can distinguish all Pareto sets and avoid the loss of local Pareto sets, while that in objective space can maintain diversity in objective space. When solving all the benchmark problems from the competition of multimodal multiobjective optimization in the IEEE Congress on Evolutionary Computation 2019, the experiments validate our advantages to maintain diversity in both objective and decision spaces.

A Framework to Handle Multimodal Multiobjective Optimization in Decomposition-Based Evolutionary Algorithms

Multi-modal multi-objective optimization is to locate (almost) equivalent Pareto optimal solutions as many as possible. While decomposition-based evolutionary algorithms have good performance for multi-objective optimization, they are likely to perform poorly for multi-modal multi-objective optimization due to the lack of mechanisms to maintain the solution space diversity. To address this issue, this paper proposes a framework to improve the performance of decomposition-based evolutionary algorithms for multi-modal multi-objective optimization. Our framework is based on three operations: assignment, deletion, and addition operations. One or more individuals can be assigned to the same subproblem to handle multiple equivalent solutions. In each iteration, a child is assigned to a subproblem based on its objective vector, i.e., its location in the objective space. The child is compared with its neighbors in the solution space assigned to the same subproblem. The performance of improved versions of six decomposition-based evolutionary algorithms by our framework is evaluated on various test problems regarding the number of objectives, decision variables, and equivalent Pareto optimal solution sets. Results show that the improved versions perform clearly better than their original algorithms.

A cluster based PSO with leader updating mechanism and ring-topology for multimodal multi-objective optimization

Abstract In the multimodal multi-objective optimization problems (MMOPs), there exists more than one Pareto optimal solutions in the decision space corresponding to the same location on the Pareto front in the objective space. To solve the MMOPs, the designed algorithm is supposed to converge to the accurate and well-distributed Pareto front, and at the same time to search for the multiple Pareto optimal solutions in the decision space. This paper presents a new cluster based particle swarm optimization algorithm (PSO) with leader updating mechanism and ring-topology for solving MMOPs. Multiple subpopulations are formed by a new decision variable clustering method with the aim of searching for the multiple Pareto optima solutions and maintaining the diversity. Global-best PSO is employed for independent evolution of subpopulations, while local-best PSO with ring topology is used to enhance the information interaction among subpopulations. Seamlessly integrated, the proposed algorithm provides a good balance between exploration and exploitation. In addition, leader updating strategy is introduced to identify the best leaders in PSO. The performance of the proposed algorithm is compared with six state-of-the-art designs over 11 multimodal multi-objective optimization test functions. Experimental results demonstrate the effectiveness of the proposed algorithm.

A Multimodal Multiobjective Evolutionary Algorithm Using Two-Archive and Recombination Strategies

There have been few researches on solving multimodal multiobjective optimization problems, whereas they are commonly seen in real-world applications but difficult for the existing evolutionary optimizers. In this paper, we propose a novel multimodal multiobjective evolutionary algorithm using two-archive and recombination strategies. In the proposed algorithm, the properties of decision variables and the relationships among them are analyzed at first to guide the evolutionary search. Then, a general framework using two archives, i.e., the convergence and the diversity archives, is adopted to cooperatively solve these problems. Moreover, the diversity archive simultaneously employs a clustering strategy to guarantee diversity in the objective space and a niche-based clearing strategy to promote the same in the decision space. At the end of evolution process, solutions in the convergence and the diversity archives are recombined to obtain a large number of multiple Pareto optimal solutions. In addition, a set of benchmark test functions and a performance metric are designed for multimodal multiobjective optimization. The proposed algorithm is empirically compared with two state-of-the-art evolutionary algorithms on these test functions. The comparative results demonstrate that the overall performance of the proposed algorithm is significantly superior to the competing algorithms.

Differential evolution based on reinforcement learning with fitness ranking for solving multimodal multiobjective problems

In multimodal multiobjective optimization problems (MMOOPs), there is more than one Pareto-optimal Set (PS) in the decision space corresponding to the same Pareto Front(PF). How to dynamically adjust the evolution direction of the population adaptively is a key problem, to ensure approaching the PF in the global sense with good convergence while finding out more PSs. In this paper, a novel Differential Evolution algorithm based on Reinforcement Learning with Fitness Ranking (DE-RLFR) is proposed. The DE-RLFR is based on the Q-learning framework, and each individual in the population is considered an agent. The fitness ranking values of each agent are used to encode hierarchical state variables. Three typical DE mutation operations are employed as optional actions for the agent. Based on the analysis of the distribution characteristics of the population in objective space, decision space and fitness-ranking space, we design a reward function of the 〉 state , action 〈 pairs to guide the population to move to the PF asymptotically. According to its reinforcement learning experience represented by the corresponding Q table value, each agent could adaptively select a mutation strategy to generate offspring individuals. The evaluation results on eleven MMOOP test functions show that DE-RLFR could quickly and effectively find multiple PSs in the decision space, and approach PF in the global sense.

A niching indicator-based multi-modal many-objective optimizer

Multi-modal multi-objective optimization is to locate (almost) equivalent Pareto optimal solutions as many as possible. Some evolutionary algorithms for multi-modal multi-objective optimization have been proposed in the literature. However, there is no efficient method for multi-modal many-objective optimization, where the number of objectives is more than three. To address this issue, this paper proposes a niching indicator-based multi-modal multi- and many-objective optimization algorithm. In the proposed method, the fitness calculation is performed among a child and its closest individuals in the solution space to maintain the diversity. The performance of the proposed method is evaluated on multi-modal multi-objective test problems with up to 15 objectives. Results show that the proposed method can handle a large number of objectives and find a good approximation of multiple equivalent Pareto optimal solutions. The results also show that the proposed method performs significantly better than eight multi-objective evolutionary algorithms.

A Review of Evolutionary Multimodal Multiobjective Optimization

Multi-modal multi-objective optimization aims to find all Pareto optimal solutions including overlapping solutions in the objective space. Multi-modal multi-objective optimization has been investigated in the evolutionary computation community since 2005. However, it is difficult to survey existing studies in this field because they have been independently conducted and do not explicitly use the term "multi-modal multi-objective optimization". To address this issue, this paper reviews existing studies of evolutionary multi-modal multi-objective optimization, including studies published under names that are different from "multi-modal multi-objective optimization". Our review also clarifies open issues in this research area.

A self-organizing multimodal multi-objective pigeon-inspired optimization algorithm

Multi-objective optimization algorithms have recently attracted much attention as they can solve problems involving two or more conflicting objectives effectively and efficiently. However, most existing studies focus on improving the performance of the solutions in the objective spaces. This paper proposes a novel multimodal multi-objective pigeon-inspired optimization (MMOPIO) algorithm where some mechanisms are designed for the distribution of the solutions in the decision spaces. First, MMOPIO employs an improved pigeon-inspired optimization (PIO) based on consolidation parameters for simplifying the structure of the standard PIO. Second, the self-organizing map (SOM) is combined with the improved PIO for better control of the decision spaces, and thus, contributes to building a good neighborhood relation for the improved PIO. Finally, the elite learning strategy and the special crowding distance calculation mechanisms are used to prevent premature convergence and obtain solutions with uniform distribution, respectively. We evaluate the performance of the proposed MMOPIO in comparison to five state-of-the-art multi-objective optimization algorithms on some test instances, and demonstrate the superiority of MMOPIO in solving multimodal multi-objective optimization problems.

A novel scalable test problem suite for multimodal multiobjective optimization

Abstract This paper proposes a novel scalable multimodal multiobjective test problem suite. The proposed test problems have various properties, such as presence of local Pareto optimal set (PS), scalable number of PSs, nonuniformly distributed PSs, discrete Pareto front (PF), and scalable number of variables and objectives. All of the test problems proposed in this paper are continuous optimization problems. Therefore, they can be used to measure different capacities of multimodal multiobjective continuous optimization algorithms. Moreover, a landscape visualization method for multiobjective problems is proposed to show the properties of the multimodal multiobjective test problems. Based on the landscapes, the characteristics of these problems are analyzed and characterized. Furthermore, the existing multimodal multiobjective optimization algorithms and several popular multiobjective algorithms are tested and compared with the novel test problem suite. Then, a discussion on the desired properties of multimodal multiobjective optimization algorithms and future works on multimodal multiobjective optimization are presented.

Handling Imbalance Between Convergence and Diversity in the Decision Space in Evolutionary Multi-Modal Multi-Objective Optimization

There may exist more than one Pareto optimal solution with the same objective vector to a multimodal multiobjective optimization problem (MMOP). The difficulties in finding such solutions can be different. Although a number of evolutionary multimodal multiobjective algorithms (EMMAs) have been proposed, they are unable to solve such an MMOP due to their convergence-first selection criteria. They quickly converge to the Pareto optimal solutions which are easy to find and therefore lose diversity in the decision space. That is, such an MMOP features an imbalance between achieving convergence and preserving diversity in the decision space. In this article, we first present a set of imbalanced distance minimization benchmark problems. Then we propose an evolutionary algorithm using a convergence-penalized density method (CPDEA). In CPDEA, the distances among solutions in the decision space are transformed based on their local convergence quality. Their density values are estimated based on the transformed distances and used as the selection criterion. We compare CPDEA with five state-of-the-art EMMAs on the proposed benchmarks. Our experimental results show that CPDEA is clearly superior in solving these problems.

Multimodal multiobjective optimization with differential evolution

Abstract This paper proposes a multimodal multiobjective Differential Evolution optimization algorithm (MMODE). The technique is conceived for deployment on problems with a Pareto multimodality, where the Pareto set comprises multiple disjoint subsets, all of which map to the same Pareto front. A new contribution is the formulation of a decision-variable preselection scheme that promotes diversity of solutions in both the decision and objective space. A new mutation-bound process is also introduced as a supplement to a classical mutation scheme in Differential Evolution methods, where offspring that lie outside the search bounds are given a second opportunity to mutate, hence reducing the density of individuals on the boundaries of the search space. New multimodal multiobjective test functions are designed, along with analytical expressions for their Pareto sets and fronts. Some test functions introduce more complicated Pareto-front shapes and allow for decision-space dimensions greater than two. The performance of the MMODE algorithm is compared with five other state-of-the-art methods. The results show that MMODE realizes superior performance by finding more and better distributed Pareto solutions.