Large-scale Multi-objective Optimization Problems Research Articles

A two-space-decomposition-based evolutionary algorithm for large-scale multiobjective optimization

Large-scale multiobjective optimization problems (LSMOPs) pose a great challenge to maintaining the diversity of solutions. However, existing large-scale multiobjective optimization algorithms (MOEAs) prefer to directly use environmental selection methods designed for small-scale optimization problems. These methods are not effective in solving complex LSMOPs. To address this issue, this paper proposes a two-space (decision space and objective space) decomposition (TSD)-based diversity maintenance mechanism. Its main idea is to explicitly decompose the decision space and objective space into a number of subspaces, each of which may contain some Pareto-optimal solutions. Searching for Pareto-optimal solutions in these subspaces may help maintain the diversity of solutions. To this end, a diversity design subspace (DDS) is constructed to decompose the decision space. Then, a large-scale MOEA (MOEA/TSD) is designed by using the proposed TSD-based diversity maintenance mechanism. Experimental studies validate the effectiveness of the proposed TSD mechanism. Compared with nine state-of-the-art large-scale MOEAs on 112 benchmark LSMOPs, our proposed algorithm offers considerable advantages in overall optimization performance. The source code of MOEA/TSD is available at https://github.com/yizhizhede/MOEATSD.

A learning and potential area-mining evolutionary algorithm for large-scale multi-objective optimization

For large-scale multi-objective optimization problems, it is not easy for existing multi-objective evolutionary algorithms to search the entire decision variable space with limited computation resources. To alleviate this problem, we propose a learning and potential area-mining evolutionary algorithm to explore and exploit key regions for accelerating optimization. First, we mine promising areas by clustering the population-gathered regions. Then, potential directions are determined by a multi-guiding point scheme in these promising areas. Subsequently, we design a local search and global search scheme to enhance population convergence while ensuring diversity. Finally, a mutation strategy is used to improve diversity. We execute numerical experiments on two widely-used LSMOP benchmarks and compare the proposed algorithm with three state-of-the-art algorithms. The statistical results indicate that the proposed algorithm has significant performance.

A Population Cooperation based Particle Swarm Optimization algorithm for large-scale multi-objective optimization

There are many multi-objective optimization problems (MOPs) in real life that contain a large number of decision variables, such as auto body parts design, financial investment, engineering design, adversarial textual attack and so on. These problems are known as large-scale multi-objective optimization problems (LSMOPs). Due to the curse of dimensionality, existing multi-objective evolutionary algorithm encounter difficulties in balancing convergence and diversity on LSMOPs. In this paper, a Population Cooperation based Particle Swarm Optimization algorithm (PCPSO) is proposed for tackling LSMOPs. To be specific, PCPSO is a two-stage optimizer with two key components: (1) In the first stage, an inter-population collaboration component named Auxiliary Population Cooperation (APC) is used to improve the convergence speed. (2) In the second stage, an intra-subpopulation collaboration component called SubPopulation Cooperation (SPC) is applied to balance convergence and diversity. Experimental results on benchmark problems with up to 5000 decision variables and 2, 3, 5, 10 objectives demonstrate that the proposed PCPSO achieves better performance than several state-of-the-art large-scale multi-objective evolutionary algorithms (LSMOEAs) on most test problems.

Evolutionary Multitasking for Large-Scale Multiobjective Optimization

Evolutionary transfer optimization (ETO) has been becoming a hot research topic in the field of evolutionary computation, which is based on the fact that knowledge learning and transfer across the related optimization exercises can improve the efficiency of others. However, rare studies employ ETO to solve large-scale multiobjective optimization problems (LMOPs). To fill this research gap, this paper proposes a new multitasking ETO algorithm via a powerful transfer learning model to simultaneously solve multiple LMOPs. In particular, inspired by adversarial domain adaptation in transfer learning, a discriminative reconstruction network (DRN) model (containing an encoder, a decoder, and a classifier) is created for each LMOP. At each generation, the DRN is trained by the currently obtained nondominated solutions for all LMOPs via backpropagation with gradient descent. With this well-trained DRN model, the proposed algorithm can transfer the solutions of source LMOPs directly to the target LMOP for assisting its optimization, can evaluate the correlation between the source and target LMOPs to control the transfer of solutions, and can learn a dimensional-reduced Pareto-optimal subspace of the target LMOP to improve the efficiency of transfer optimization in the large-scale search space. Moreover, we propose a real-world multitasking LMOP suite to simulate the training of deep neural networks on multiple different classification tasks. Finally, the effectiveness of the proposed algorithm has been validated on this real-world problem suite and other two synthetic problem suites.

Manifold Interpolation for Large-Scale Multiobjective Optimization via Generative Adversarial Networks.

Large-scale multiobjective optimization problems (LSMOPs) are characterized as optimization problems involving hundreds or even thousands of decision variables and multiple conflicting objectives. To solve LSMOPs, some algorithms designed a variety of strategies to track Pareto-optimal solutions (POSs) by assuming that the distribution of POSs follows a low-dimensional manifold. However, traditional genetic operators for solving LSMOPs have some deficiencies in dealing with the manifold, which often results in poor diversity, local optima, and inefficient searches. In this work, a generative adversarial network (GAN)-based manifold interpolation framework is proposed to learn the manifold and generate high-quality solutions on the manifold, thereby improving the optimization performance of evolutionary algorithms. We compare the proposed approach with several state-of-the-art algorithms on various large-scale multiobjective benchmark functions. The experimental results demonstrate that significant improvements have been achieved by the proposed framework in solving LSMOPs.

A fast and efficient teaching-learning-based optimization algorithm for large-scale multi-objective optimization problems

ABSTRACT Multi-objective optimization problems with large-scale decision variables, known as LSMOPs, are characterized by their large-scale search space and multiple conflicting objectives to be optimized. They have been involved in many emergent real-world applications, such as feature and instance selection, data clustering, adversarial attack, vehicle routing problem, and others. In this work, we propose a new teaching-learning-based optimization algorithm to effectively tackle large-scale multi-objective optimization problems. The proposed approach ensures both the diversity of solutions, requested for high dimensionality of LSMOPs, and the balance between the exploitation and exploration during the optimization process. The experimental studies conducted on 54 instances of a large-scale multi-objective optimization problems test suite and the comparisons against well-known and state-of-the-art algorithms have shown the superiority of the proposed algorithm in terms of Pareto front approximation and computation time by using limited evaluation budget.

Guest Editorial Special Issue on Large-Scale Evolutionary Multiobjective Optimization and Its Practical Applications

Complex optimization problems with hundreds or even thousands of decision variables and dozens of conflicting objectives are not uncommon in the real world. In the past five years, increased research efforts have been dedicated to large-scale multiobjective optimization problems (LSMOPs) by using a variety of search strategies, including variable grouping, variable analysis, problem transformation, dimensionality reduction, and novel recombination operators. Despite the success of these efforts in solving some general LSMOPs, there still remains a big gap between the LSMOPs that have been addressed and those encountered in real life, such as sparse, highly constrained, dynamic, and expensive LMOPs, as well as very large-scale and many-objective optimization problems that are widely seen and of paramount importance for solving scientific and engineering problems. Due to the significant practical importance of large-scale multiobjective optimization, there is a high demand for computationally efficient and effective evolutionary algorithms for solving LSMOPs.

An extended fuzzy decision variables framework for solving large-scale multiobjective optimization problems

In large-scale multiobjective optimization, the huge search space poses a great challenge to the convergence search of existing evolutionary algorithms. A fuzzy decision variables (FDV) framework for large-scale multiobjective optimization involves a complex parameter tuning process, and the convergence efficiency is moderate. Therefore, we propose an extended fuzzy decision variables (EFDV) framework based on linear search and the non-dominated rate (the proportion of non-dominated individuals in the whole population) to solve large-scale multiobjective optimization problems. First, guiding solutions are sampled from the central region to provide more possible search directions in large-scale decision space. In addition, fuzzy evolution or precise evolution is determined according to the non-dominated rate. When the non-dominated rate is low, the convergence of the whole population is relatively poor. Then a higher degree of fuzzy search is carried out, and the individuals with poor convergence are reversed to increase the population's diversity and enhance the effectiveness of searches. Finally, by comparing experiments on several large-scale multiobjective test suites with 500–5000 decision variables, the efficiency of the EFDV is confirmed. According to experimental results, the EFDV can significantly enhance the performance and computing effectiveness of multiobjective optimizers when used for large-scale multiobjective optimization.

Machine Learning Assisted Evolutionary Multi-Objective Optimization [Guest Editorial

Optimization and learning are two main paradigms of artificial intelligence in addressing complex real-world problems, with their respective focuses but frequently enhanced by each other. Evolutionary multi-objective optimization (EMO) algorithms are a family of nature-inspired algorithms widely used for solving multi-objective optimization problems (MOPs). Despite the great success achieved by the existing EMO algorithms, most of these algorithms have also encountered many challenges in terms of optimization performance and efficiency for solving complex MOPs such as large-scale MOPs, expensive MOPs, dynamic MOPs, and real-world MOPs. To cope with these challenges, there has been increasing interest in applying machine learning (ML) techniques to enhance the EMO algorithms. Specifically, ML techniques can be adopted to extract useful knowledge hidden in the data generated by EMO algorithms in the search process, which can be leveraged to assist different components in EMO algorithms in different ways, e.g., problem formulation, offspring generation, fitness evaluation, and/or environmental selection. These machine learning assisted components have substantially enhanced the ability of EMO algorithms in handling complex MOPs.

A chaotic differential evolution and symmetric direction sampling for large-scale multiobjective optimization

As large-scale multiobjective optimization problems (LSMOPs) contain many decision variables, most existing large-scale multiobjective optimization algorithms (LSMOEAs) require millions or even tens of millions of function evaluations to obtain high-quality solutions, which is unaffordable for many real-world problems. Moreover, few studies are dedicated to solving LSMOPs under limited function evaluations. This article proposes a Chaotic Differential Evolution and Symmetric Direction Sampling for large-scale multiobjective optimization (CDE-SDS). First, based on the decision variable information of nondominated solutions, a chaotic differential evolution strategy is adopted to accelerate the convergence of the population towards the Pareto front. Second, a symmetric direction sampling strategy is employed, which generates sampling solutions by constructing symmetric search directions in a large-scale decision space. Thus, the high-dimensional decision space can be explored more fully. The two strategies optimize the population alternately to balance exploitation and exploration. The performance of CDE-SDS is compared with seven state-of-the-art large-scale multiobjective evolution algorithms on an artificial test suit LSMOP with 500 to 5000 decision variables and a real-world time-varying ratio error estimation problem with 3000 decision variables. Experimental results show that the performance of CDE-SDS significantly outperforms the seven compared algorithms in terms of both diversity and convergence of solutions under limited function evaluations.

Cooperative coevolutionary multi-guide particle swarm optimization algorithm for large-scale multi-objective optimization problems

Cooperative coevolutionary multi-guide particle swarm optimization algorithm for large-scale multi-objective optimization problems

A Stable Large-Scale Multiobjective Optimization Algorithm with Two Alternative Optimization Methods

For large-scale multiobjective evolutionary algorithms based on the grouping of decision variables, the challenge is to design a stable grouping strategy to balance convergence and population diversity. This paper proposes a large-scale multiobjective optimization algorithm with two alternative optimization methods (LSMOEA-TM). In LSMOEA-TM, two alternative optimization methods, which adopt two grouping strategies to divide decision variables, are introduced to efficiently solve large-scale multiobjective optimization problems. Furthermore, this paper introduces a Bayesian-based parameter-adjusting strategy to reduce computational costs by optimizing the parameters in the proposed two alternative optimization methods. The proposed LSMOEA-TM and four efficient large-scale multiobjective evolutionary algorithms have been tested on a set of benchmark large-scale multiobjective problems, and the statistical results demonstrate the effectiveness of the proposed algorithm.

An evolutionary algorithm based on dynamic sparse grouping for sparse large scale multiobjective optimization

Sparse large scale multiobjective optimization problems (sparse LSMOPs) contain numerous decision variables, and their Pareto optimal solutions' decision variables are very sparse (i.e., the majority of these solutions' decision variables are zero-valued). This poses grand challenges to an algorithm in converging to the Pareto set. Numerous evolutionary algorithms (EAs) tailored for sparse LSMOPs have been proposed in recent years. However, the final population generated by these EAs is not sparse enough because the location of the nonzero decision variables is difficult to locate accurately and there is insufficient interaction between the nonzero decision variables' locating process and the nonzero decision variables' optimizing process. To address this issue, we propose a dynamic sparse grouping evolutionary algorithm (DSGEA) that dynamically groups decision variables in the population that have a comparable amount of nonzero decision variables. Improved evolutionary operators are introduced to optimize the decision variables in groups. As a result, the population obtained by DSGEA can stably evolve towards the sparser Pareto optimal that has a precise location of nonzero decision variables. The proposed algorithm outperforms existing up-to-date EAs for sparse LSMOPs in experiments on three real-world problems and eight benchmark problems.

Evolutionary multiobjective optimization via efficient sampling-based offspring generation

With the rising number of large-scale multiobjective optimization problems from academia and industries, some evolutionary algorithms (EAs) with different decision variable handling strategies have been proposed in recent years. They mainly emphasize the balance between convergence enhancement and diversity maintenance for multiobjective optimization but ignore the local search tailored for large-scale optimization. Consequently, most existing EAs can hardly obtain the global or local optima. To address this issue, we propose an efficient sampling-based offspring generation method for large-scale multiobjective optimization, where convergence enhancement and diversity maintenance, together with ad hoc local search, are considered. First, the decision variables are dynamically classified into two types for solving large-scale decision space in a divide-and-conquer manner. Then, a convergence-related sampling strategy is designed to handle those decision variables related to convergence enhancement. Two additional sampling strategies are proposed for diversity maintenance and local search, respectively. Experimental results on problems with up to 5000 decision variables have indicated the effectiveness of the algorithm in large-scale multiobjective optimization.

Learning to Accelerate Evolutionary Search for Large-Scale Multiobjective Optimization

Most existing evolutionary search strategies are not so efficient when directly handling the decision space of large-scale multiobjective optimization problems (LMOPs). To enhance the efficiency of tackling LMOPs, this article proposes an accelerated evolutionary search (AES) strategy. Its main idea is to learn a gradient-descent-like direction vector (GDV) for each solution via the specially trained feedforward neural network, which may be the learnt possibly fastest convergent direction to reproduce new solutions efficiently. To be specific, a multilayer perceptron (MLP) with only one hidden layer is constructed, in which the number of neurons in the input and output layers is equal to the dimension of the decision space. Then, to get appropriate training data for the model, the current population is divided into two subsets based on the nondominated sorting, and each poor solution in one subset with worse convergence will be paired to an elitist solution in another subset with the minimum angle to it, which is considered most likely to guide it with rapid convergence. Next, this MLP is updated via backpropagation with gradient descent by using the above elaborately prepared dataset. Finally, an accelerated large-scale multiobjective evolutionary algorithm (ALMOEA) is designed by using AES as a reproduction operator. Experimental studies validate the effectiveness of the proposed AES when handling the search space of LMOPs with dimensionality ranging from 1000 to 10000. When compared with six state-of-the-art evolutionary algorithms, the experimental results also show the better efficiency and performance of the proposed optimizer in solving various LMOPs.

Multiple sparse detection-based evolutionary algorithm for large-scale sparse multiobjective optimization problems

AbstractSparse multiobjective optimization problems are common in practical applications. Such problems are characterized by large-scale decision variables and sparse optimal solutions. General large-scale multiobjective optimization problems (LSMOPs) have been extensively studied for many years. They can be well solved by many excellent custom algorithms. However, when these algorithms are used to deal with sparse LSMOPs, they often encounter difficulties because the sparse nature of the problem is not considered. Therefore, aiming at sparse LSMOPs, an algorithm based on multiple sparse detection is proposed in this paper. The algorithm applies an adaptive sparse genetic operator that can generate sparse solutions by detecting the sparsity of individuals. To improve the deficiency of sparse detection caused by local detection, an enhanced sparse detection (ESD) strategy is proposed in this paper. The strategy uses binary coefficient vectors to integrate the masks of nondominated solutions. Essentially, the mask is globally and deeply optimized by coefficient vectors to enhance the sparsity of the solutions. In addition, the algorithm adopts an improved weighted optimization strategy to fully optimize the key nonzero variables to balance exploration and optimization. Finally, the proposed algorithm is named MOEA-ESD and is compared to the current state-of-the-art algorithm to verify its effectiveness.

A Variable Importance-Based Differential Evolution for Large-Scale Multiobjective Optimization.

Large-scale multiobjective optimization problems (LMOPs) bring significant challenges for traditional evolutionary operators, as their search capability cannot efficiently handle the huge decision space. Some newly designed search methods for LMOPs usually classify all variables into different groups and then optimize the variables in the same group with the same manner, which can speed up the population's convergence. Following this research direction, this article suggests a differential evolution (DE) algorithm that favors searching the variables with higher importance to the solving of LMOPs. The importance of each variable to the target LMOP is quantized and then all variables are categorized into different groups based on their importance. The variable groups with higher importance are allocated with more computational resources using DE. In this way, the proposed method can efficiently generate offspring in a low-dimensional search subspace formed by more important variables, which can significantly speed up the convergence. During the evolutionary process, this search subspace for DE will be expanded gradually, which can strike a good balance between exploration and exploitation in tackling LMOPs. Finally, the experiments validate that our proposed algorithm can perform better than several state-of-the-art evolutionary algorithms for solving various benchmark LMOPs.

A fast sampling based evolutionary algorithm for million-dimensional multiobjective optimization

With their complexity and vast search space, large-scale multiobjective optimization problems (LSMOPs) challenge existing multiobjective evolutionary algorithms (MOEAs). Recently, several large-scale multiobjective evolutionary algorithms have been developed to tackle LSMOPs. Unlike conventional MOEAs that concentrate on selection operations in the objective space, large-scale MOEAs emphasize operations in the decision space, such as offspring generation, to tackle the large number of decision variables. Nevertheless, most present large-scale MOEAs experience difficulty effectively and efficiently solving LSMOPs with tens of thousands or more decision variables or exhibit poor versatility in solving different LSMOPs. We propose a fast large-scale MOEA framework with reference-guided offspring generation, named FLEA, aiming at these issues. Generally, FLEA constructs several reference vectors in the decision space to steer the sampling of promising solutions during offspring generation. A parameter is used to allocate computation resources between the convergence and diversity of the offspring population adaptively. Without computationally expensive problem reformulation or decision variable analysis techniques, the proposed method can significantly accelerate the search speed of conventional MOEAs in solving LSMOPs. FLEA is examined on various LSMOPs with up to 1.6 million decision variables, demonstrating its superior effectiveness, efficiency, and versatility in large-scale multiobjective optimization.

Goal programming and genetic algorithm in multiple objective optimization model for project portfolio selection: a review

Many scientific fields, such as engineering, data analytics, and deep learning, focus on optimization. Optimization problems are classified into two types based on the number of optimized objective functions: single objective and multi objective optimization problems. In this paper, a comparative review since 2000 using one of the deterministic and stochastic modelling approaches called goal programming (GP) and genetic algorithm (GA) in multi-objective optimization problem is discussed. This study gives a prime review of the application of GP and GA in various criteria of project portfolio selection problem. GP is a method for solving large-scale multi-objective optimization problems to assist decision makers in finding solutions that satisfy several competing goals. GA on the other hand are global meta-heuristic search algorithms that are used to provide approximation or optimal solutions to large-scale optimization problems. Of the 23 articles considered in this review showed that, from more than 100 projects, GA proved near optimal, feasible solution and efficient frontier in projects ranking, projects interaction and a preferred decision support tool of project portfolio selection. In addition, the two models select projects on risk-based approach, but GA proved to be more effective in terms of number of projects proposed, central processing unit (CPU) time and accuracy. The review concludes that, in multi-objective optimization model for project portfolio selection problems on a large-scale, very large or complex problems and less CPU time, GA is more effective than GP in multi-objective optimization problems. The review also showed gaps in previous studies of GP and GA application on project portfolio selection problem (PPSP). This review will aid scholars and demanding practitioners in gaining a broader understanding of goal programming and genetic algorithms in the context of project portfolio selection problems.

A Large Scale Evolutionary Algorithm Based on Determinantal Point Processes for Large Scale Multi-Objective Optimization Problems

Global optimization challenges are frequent in scientific and engineering areas where loads of evolutionary computation methods i.e., differential evolution (DE) and particle-swarm optimization (PSO) are employed to handle these problems. However, the performance of these algorithms declines due to expansion in the problem dimension. The evolutionary algorithms are obstructed to congregate with the Pareto front rapidly while using the large-scale optimization algorithm. This work intends a large-scale multi-objective evolutionary optimization scheme aided by the determinantal point process (LSMOEA-DPPs) to handle this problem. The proposed DPP model introduces a mechanism consisting of a kernel matrix and a probability model to achieve convergence and population variety in high dimensional relationship balance to keep the population diverse. We have also employed elitist non-dominated sorting for environmental selection. Moreover, the projected algorithm also demonstrates and distinguishes four cutting-edge algorithms, each with two and three objectives, respectively, and up to 2500 decision variables. The experimental results show that LSMOEA-DPPs outperform four cutting-edge multi-objective evolutionary algorithms by a large margin.