Large-scale Multi-objective Optimization Problems Research Articles

A two-stage direction-guided evolutionary algorithm for large-scale multiobjective optimization

Large-scale multiobjective optimization problems (LSMOPs) have exponential growth in the search space as the decision variables increase, and the vast search space poses a challenge to the performance of multiobjective evolutionary algorithms (MOEAs). Many current large-scale MOEAs need to consume a large amount of computational resources to get good performance. This paper proposes a two-stage direction-guided evolutionary algorithm for large-scale multiobjective optimization (LMOEA-S2D) to balance the performance and computational resource overhead. The algorithm exploits the Pareto-optimality property of domination and the diversity-preserving property of decomposition to optimize the performance in the two stages, respectively, and designs a corresponding direction-guided mechanism to improve search efficiency. LMOEA-S2D designs global direction search and local direction search in the domination-based stage for efficient exploitation to accelerate population convergence. To promote greater population diversity, a hybrid direction search was devised to aid diversity exploration in the decomposition-based stage, and this facilitates even distribution of candidate solutions across the Pareto optimal frontier. LMOEA-S2D is compared with five state-of-the-art large-scale MOEAs on some large-scale multiobjective test suites with 100 to 5,000 decision variables. The experimental results show that LMOEA-S2D significantly outperformed all compared algorithms under limited computational resources.

Boosting scalability for large-scale multiobjective optimization via transfer weights

Large-scale multiobjective optimization problems (LSMOPs), which optimize multiple conflicting objectives with hundreds or even thousands of decision variables, demand increasing computational resources to assure satisfactory performance as the decision variables increase. Multiobjective evolutionary algorithms are naturally scalable, but as the dimension increases, the conflict between analyzing enough solutions and the limited number of function evaluations has hindered further improvements in scalability. In this paper, we first define the scalability of multiobjective evolutionary algorithms and design an indicator to quantitatively measure the scalability. Second, to boost scalability when solving LSMOPs, we propose a scalable multiobjective optimization algorithm by transferring weights between solutions to reduce dependency on ever-increasing computational resources as the problem dimension increases. The proposed framework entails constructing a latent decision space to determine evolutionary weights for chosen representative solutions. These computed weights are then transferred to the remaining solutions, enabling evolutionary optimization to proceed without requiring additional evaluations, even as the dimensionality increases. By utilizing the knowledge gained from the source solutions, each solution is customized with an evolutionary weight scheme that not only preserves computational resources but also enhances optimization performance, thereby boosting scalability. We have conducted experiments on LSMOPs to verify the effectiveness and scalability of the proposed algorithm. The proposed method outperforms selected state-of-the-art algorithms and gains scalability boosts in situations where the dimensions increase while the function evaluation remains constant.

Improved Evolutionary Operators for Sparse Large-Scale Multiobjective Optimization Problems

Many critical science, societal, and engineering fields contain large-scale multiobjective optimization problems (LSMOPs), comprised of many decision variables. However, as the number of decision variables increases, optimization algorithms face exponentially large search spaces, thereby exhibiting a degraded performance. Nonetheless, LSMOPs whose optimal solutions correspond to sparse variable vectors can be solved more efficiently by evolutionary multiobjective optimization (EMO) algorithms. Despite the great recent strides in developing generic EMO algorithms for sparse LSMOPs, there is still room for improvement. Specifically, algorithms still struggle to find convergent and diverse Pareto fronts in an acceptable amount of time when solving sparse LSMOPs with thousands of decision variables. To better solve sparse LSMOPs, we propose a novel set of evolutionary operators to adapt small-scale EMO algorithms for sparse LSMOPs. These simple, novel, and effective operators include varied striped sparse population sampling (VSSPS), sparse simulated binary crossover (S-SBX), and sparse polynomial mutation (S-PM). These operators, combined with NSGA-II, make the proposed S-NSGA-II algorithm. S-NSGA-II runs near-universally faster than existing methods for problems containing up to 6,400 decision variables, while performing as well as or better than contemporary sparse LSMOP algorithms with respect to hypervolume, especially with problems with larger than 5,000 decision variables.

A co-evolutionary algorithm based on sparsity clustering for sparse large-scale multi-objective optimization

Sparse large-scale multi-objective optimization problems (LSMOPs), which are characterized by high dimensional search space and sparse Pareto optimal solutions, have a widespread existence in academic research and practical applications. While the high dimensional decision space poses challenges to multi-objective evolutionary algorithms (MOEAs), the difficulty of solving sparse LSMOPs can be alleviated by utilizing the prior knowledge that the optimal solutions are sparse. In this paper, a co-evolutionary algorithm based on sparsity clustering, namely SCEA, is proposed, where the prior knowledge of sparse optimal solutions is utilized explicitly. At each generation, SCEA first calculates the current optimal sparsity by sparsity clustering. Then, SCEA divides the population into a winner subpopulation and two loser subpopulations. While the winner subpopulation reproduces offspring solutions by conventional genetic operators, the loser subpopulations generate offspring solutions along two competitive directions under the guidance of current optimal sparsity and variable importance. In the experiments, four state-of-the-art MOEAs are selected as the comparative algorithms. Experimental results show that the proposed algorithm is superior to the four competitors on both benchmark problems and practical applications, which include the sparse signal reconstruction problem, the community detection problem, and the instance selection problem.

Improving Performance Insensitivity of Large-Scale Multiobjective Optimization via Monte Carlo Tree Search.

The large-scale multiobjective optimization problem (LSMOP) is characterized by simultaneously optimizing multiple conflicting objectives and involving hundreds of decision variables. Many real-world applications in engineering can be modeled as LSMOPs; simultaneously, engineering applications require insensitivity in performance. This requirement typically means that the algorithm should not only produce good results in terms of performance for every run but also the performance of multiple runs should not fluctuate too much. However, existing large-scale multiobjective optimization algorithms often focus on improving algorithm performance, but pay little attention to improving the insensitivity characteristic of algorithms. This directly leads to substantial limitations when solving practical problems. In this work, we propose an evolutionary algorithm called large-scale multiobjective optimization algorithm via Monte Carlo tree search, which is based on the Monte Carlo tree search and aims to improve the performance and insensitivity of solving LSMOPs. The proposed method samples decision variables to construct new nodes on the Monte Carlo tree for optimization and evaluation, and it selects nodes with good evaluations for further searches in order to reduce the performance sensitivity caused by large-scale decision variables. We propose two metrics to measure the sensitivity of the algorithm and compare the proposed algorithm with several state-of-the-art designs on different benchmark functions and metrics. The experimental results confirm the effectiveness and performance insensitivity of the proposed design for solving LSMOPs.

A Large-Scale Multiobjective Particle Swarm Optimizer With Enhanced Balance of Convergence and Diversity.

Large-scale multiobjective optimization problems (LSMOPs) continue to be challenging for existing multiobjective evolutionary algorithms (MOEAs). The main difficulties are that: 1) the diversity preservation in both the objective space and the decision space needs to be taken into account when solving LSMOPs and 2) the existing learning structures in current MOEAs usually make the learning operators only coincidentally serve convergence and diversity, leading to difficulties in balancing these two factors. Therefore, balancing convergence and diversity in current MOEAs is difficult. To address these issues, this article proposes a multiobjective particle swarm optimizer with enhanced balance of convergence and diversity (MPSO-EBCD). In MPSO-EBCD, a novel velocity update structure for multiobjective particle swarm optimization is put forward, dividing the convergence, and diversity preservation operations into independent components. Following the proposed update structure, a weighted convergence factor is introduced to serve the convergence strategy, whilst a diversity preservation strategy is built to uniformly distribute the particles in the searched space based on a proposed multidimensional local sparseness degree indicator. By this means, MPSO-EBCD is able to balance convergence and diversity with specific parameters in independent operators. Experimental results on LSMOP benchmarks and a voltage transformer optimization problem demonstrate the competitiveness of the proposed algorithm compared to several state-of-the-art MOEAs.

Large-Scale Multiobjective Optimization via Reformulated Decision Variable Analysis

With the rising number of large-scale multiobjective optimization problems (LSMOPs) from academia and industries, some multiobjective evolutionary algorithms (MOEAs) with different decision variable handling strategies have been proposed. Decision variable analysis (DVA) is widely used in large-scale optimization, aiming at identifying the connection between each decision variable and the objectives, and grouping those interacting decision variables to reduce the complexity of LSMOPs. Despite their effectiveness, existing DVA techniques require the unbearable cost of function evaluations for solving LSMOPs. We propose a reformulation based approach for efficient DVA to address this deficiency. Then a large-scale MOEA is proposed based on reformulated DVA, namely LERD. Specifically, the DVA process is reformulated into an optimization problem with binary decision variables, aiming to approximate different grouping results. Afterwards, each group of decision variables is used for convergence-related or diversity-related optimization. The effectiveness and efficiency of the reformulation based DVA are validated by replacing the corresponding DVA techniques in two large-scale MOEAs. Experiments in comparison with six state-of-the-art large-scale MOEAs on LSMOPs with up to 2000 decision variables have shown the promising performance of LERD.

Neural Net-Enhanced Competitive Swarm Optimizer for Large-Scale Multiobjective Optimization.

The competitive swarm optimizer (CSO) classifies swarm particles into loser and winner particles and then uses the winner particles to efficiently guide the search of the loser particles. This approach has very promising performance in solving large-scale multiobjective optimization problems (LMOPs). However, most studies of CSOs ignore the evolution of the winner particles, although their quality is very important for the final optimization performance. Aiming to fill this research gap, this article proposes a new neural net-enhanced CSO for solving LMOPs, called NN-CSO, which not only guides the loser particles via the original CSO strategy, but also applies our trained neural network (NN) model to evolve winner particles. First, the swarm particles are classified into winner and loser particles by the pairwise competition. Then, the loser particles and winner particles are, respectively, treated as the input and desired output to train the NN model, which tries to learn promising evolutionary dynamics by driving the loser particles toward the winners. Finally, when model training is complete, the winner particles are evolved by the well-trained NN model, while the loser particles are still guided by the winner particles to maintain the search pattern of CSOs. To evaluate the performance of our designed NN-CSO, several LMOPs with up to ten objectives and 1000 decision variables are adopted, and the experimental results show that our designed NN model can significantly improve the performance of CSOs and shows some advantages over several state-of-the-art large-scale multiobjective evolutionary algorithms as well as over model-based evolutionary algorithms.

A fast interpolation-based multi-objective evolutionary algorithm for large-scale multi-objective optimization problems

A fast interpolation-based multi-objective evolutionary algorithm for large-scale multi-objective optimization problems

A problem transformation-based and decomposition-based evolutionary algorithm for large-scale multiobjective optimization

For large-scale multi-objective optimization problems, the search space becomes exceptionally vast, resulting in increased complexity in the search process. The search space usually contains several local optimal individuals, and the difficulty of finding the global optimal individuals increases greatly. A problem transformation-based and decomposition-based large-scale multi-objective evolutionary algorithm is proposed to solve the problem of reducing the dimensionality of the search space and the optimization of populations. Reducing the number of optimization problems in the decision space through problem transformation has lowered the complexity of multi-objective optimization problems and enhanced computational efficiency. In the decision space, employing two direction vectors adaptively guides the generation of promising individuals, thereby preventing the population from falling into local optima. Due to the dimensionality reduction of the decision space and the optimization of the individuals in the objective space, a set of optimal solutions are effectively obtained and uniformly distributed on the approximate Pareto optimal front. Experimental results show that the algorithm is highly competitive with five large-scale multi-objective evolutionary algorithms for large-scale multi-objective optimization test problems with up to 2000 decision variables.

Directed quick search guided evolutionary framework for large-scale multi-objective optimization problems

For large-scale multi-objective evolutionary algorithms (LSMOEAs), obtaining efficient evolutionary directions in an ultrahigh-dimensional decision space to produce high-quality offspring is a major challenge. This study proposes a novel framework called the directed quick search-guided large-scale evolutionary framework (QSLMOF) to address multi-objective optimization problems on a large scale. The framework contains two innovative strategies: the bidirectional vector-based sampling strategy (BDVS) and the quick search-guided directed reproduction strategy (QS-DRS). BDVS is introduced as an approach to swiftly discern promising solutions that can steer the exploration in the large-scale decision space. This is achieved by formulating two distinct types of sampling directions to rapidly reduce the search space and strike a delicate balance between convergence and diversity. In QS-DRS, we introduced the concept of the potential convergence gradient (PCG), incorporating directional information from historical searches and convergence directions indicated byelite solutions in the current population. With this property, inferior solutions can obtain excellent search directions to explore the decision space, which can improve the convergence rate and prevent the search from falling into a local optimum. The proposed large-scale evolutionary framework incorporates an existing environmental selection mechanism. Comprehensive experiments show that the two novel strategies improve the search efficiency and evolutionary quality of LSMOEAs in ultrahigh-dimensional decision spaces. Moreover, the proposed framework outperformed seven state-of-the-art LSMOEAs for nine large-scale multi-objective benchmark problems LSMOP1-LSMOP9 with up to three objectives and 10000 decision variables.

Competitive Decomposition-Based Multiobjective Architecture Search for the Dendritic Neural Model.

The dendritic neural model (DNM) is computationally faster than other machine-learning techniques, because its architecture can be implemented by using logic circuits and its calculations can be performed entirely in binary form. To further improve the computational speed, a straightforward approach is to generate a more concise architecture for the DNM. Actually, the architecture search is a large-scale multiobjective optimization problem (LSMOP), where a large number of parameters need to be set with the aim of optimizing accuracy and structural complexity simultaneously. However, the issues of irregular Pareto front, objective discontinuity, and population degeneration strongly limit the performances of conventional multiobjective evolutionary algorithms (MOEAs) on the specific problem. Therefore, a novel competitive decomposition-based MOEA is proposed in this study, which decomposes the original problem into several constrained subproblems, with neighboring subproblems sharing overlapping regions in the objective space. The solutions in the overlapping regions participate in environmental selection for the neighboring subproblems and then propagate the selection pressure throughout the entire population. Experimental results demonstrate that the proposed algorithm can possess a more powerful optimization ability than the state-of-the-art MOEAs. Furthermore, both the DNM itself and its hardware implementation can achieve very competitive classification performances when trained by the proposed algorithm, compared with numerous widely used machine-learning approaches.

MOCPSO: A multi-objective cooperative particle swarm optimization algorithm with dual search strategies

Particle swarm optimization (PSO) is a widely embraced meta-heuristic approach to tackling the complexities of multi-objective optimization problems (MOPs), renowned for its simplicity and swift convergence. However, when faced with large-scale multi-objective optimization problems (LSMOPs), most PSOs suffer from limited local search capabilities and insufficient randomness. This can result in suboptimal results, particularly in high-dimensional spaces. To address these issues, this paper introduces MOCPSO, a Multi-Objective Cooperative Particle Swarm Optimization Algorithm with Dual Search Strategies. MOCPSO incorporates a diversity search strategy (DSS) to augment perturbation and enhance the local search scope of particles, alongside a more convergent search strategy (CSS) to expedite particle convergence. Moreover, MOCPSO utilizes a three-category framework to effectively leverage the benefits of both DSS and CSS. Experimental results on benchmark LSMOPs with 500, 1000, and 2000 decision variables demonstrate that MOCPSO outperforms existing state-of-the-art large-scale multi-objective evolutionary algorithms on most test instances.

A two-space-decomposition-based evolutionary algorithm for large-scale multiobjective optimization

Large-scale multiobjective optimization problems (LSMOPs) pose a great challenge to maintaining the diversity of solutions. However, existing large-scale multiobjective optimization algorithms (MOEAs) prefer to directly use environmental selection methods designed for small-scale optimization problems. These methods are not effective in solving complex LSMOPs. To address this issue, this paper proposes a two-space (decision space and objective space) decomposition (TSD)-based diversity maintenance mechanism. Its main idea is to explicitly decompose the decision space and objective space into a number of subspaces, each of which may contain some Pareto-optimal solutions. Searching for Pareto-optimal solutions in these subspaces may help maintain the diversity of solutions. To this end, a diversity design subspace (DDS) is constructed to decompose the decision space. Then, a large-scale MOEA (MOEA/TSD) is designed by using the proposed TSD-based diversity maintenance mechanism. Experimental studies validate the effectiveness of the proposed TSD mechanism. Compared with nine state-of-the-art large-scale MOEAs on 112 benchmark LSMOPs, our proposed algorithm offers considerable advantages in overall optimization performance. The source code of MOEA/TSD is available at https://github.com/yizhizhede/MOEATSD.

A learning and potential area-mining evolutionary algorithm for large-scale multi-objective optimization

For large-scale multi-objective optimization problems, it is not easy for existing multi-objective evolutionary algorithms to search the entire decision variable space with limited computation resources. To alleviate this problem, we propose a learning and potential area-mining evolutionary algorithm to explore and exploit key regions for accelerating optimization. First, we mine promising areas by clustering the population-gathered regions. Then, potential directions are determined by a multi-guiding point scheme in these promising areas. Subsequently, we design a local search and global search scheme to enhance population convergence while ensuring diversity. Finally, a mutation strategy is used to improve diversity. We execute numerical experiments on two widely-used LSMOP benchmarks and compare the proposed algorithm with three state-of-the-art algorithms. The statistical results indicate that the proposed algorithm has significant performance.

A Population Cooperation based Particle Swarm Optimization algorithm for large-scale multi-objective optimization

There are many multi-objective optimization problems (MOPs) in real life that contain a large number of decision variables, such as auto body parts design, financial investment, engineering design, adversarial textual attack and so on. These problems are known as large-scale multi-objective optimization problems (LSMOPs). Due to the curse of dimensionality, existing multi-objective evolutionary algorithm encounter difficulties in balancing convergence and diversity on LSMOPs. In this paper, a Population Cooperation based Particle Swarm Optimization algorithm (PCPSO) is proposed for tackling LSMOPs. To be specific, PCPSO is a two-stage optimizer with two key components: (1) In the first stage, an inter-population collaboration component named Auxiliary Population Cooperation (APC) is used to improve the convergence speed. (2) In the second stage, an intra-subpopulation collaboration component called SubPopulation Cooperation (SPC) is applied to balance convergence and diversity. Experimental results on benchmark problems with up to 5000 decision variables and 2, 3, 5, 10 objectives demonstrate that the proposed PCPSO achieves better performance than several state-of-the-art large-scale multi-objective evolutionary algorithms (LSMOEAs) on most test problems.

Evolutionary Multitasking for Large-Scale Multiobjective Optimization

Evolutionary transfer optimization (ETO) has been becoming a hot research topic in the field of evolutionary computation, which is based on the fact that knowledge learning and transfer across the related optimization exercises can improve the efficiency of others. However, rare studies employ ETO to solve large-scale multiobjective optimization problems (LMOPs). To fill this research gap, this paper proposes a new multitasking ETO algorithm via a powerful transfer learning model to simultaneously solve multiple LMOPs. In particular, inspired by adversarial domain adaptation in transfer learning, a discriminative reconstruction network (DRN) model (containing an encoder, a decoder, and a classifier) is created for each LMOP. At each generation, the DRN is trained by the currently obtained nondominated solutions for all LMOPs via backpropagation with gradient descent. With this well-trained DRN model, the proposed algorithm can transfer the solutions of source LMOPs directly to the target LMOP for assisting its optimization, can evaluate the correlation between the source and target LMOPs to control the transfer of solutions, and can learn a dimensional-reduced Pareto-optimal subspace of the target LMOP to improve the efficiency of transfer optimization in the large-scale search space. Moreover, we propose a real-world multitasking LMOP suite to simulate the training of deep neural networks on multiple different classification tasks. Finally, the effectiveness of the proposed algorithm has been validated on this real-world problem suite and other two synthetic problem suites.

Manifold Interpolation for Large-Scale Multiobjective Optimization via Generative Adversarial Networks.

Large-scale multiobjective optimization problems (LSMOPs) are characterized as optimization problems involving hundreds or even thousands of decision variables and multiple conflicting objectives. To solve LSMOPs, some algorithms designed a variety of strategies to track Pareto-optimal solutions (POSs) by assuming that the distribution of POSs follows a low-dimensional manifold. However, traditional genetic operators for solving LSMOPs have some deficiencies in dealing with the manifold, which often results in poor diversity, local optima, and inefficient searches. In this work, a generative adversarial network (GAN)-based manifold interpolation framework is proposed to learn the manifold and generate high-quality solutions on the manifold, thereby improving the optimization performance of evolutionary algorithms. We compare the proposed approach with several state-of-the-art algorithms on various large-scale multiobjective benchmark functions. The experimental results demonstrate that significant improvements have been achieved by the proposed framework in solving LSMOPs.

A fast and efficient teaching-learning-based optimization algorithm for large-scale multi-objective optimization problems

ABSTRACT Multi-objective optimization problems with large-scale decision variables, known as LSMOPs, are characterized by their large-scale search space and multiple conflicting objectives to be optimized. They have been involved in many emergent real-world applications, such as feature and instance selection, data clustering, adversarial attack, vehicle routing problem, and others. In this work, we propose a new teaching-learning-based optimization algorithm to effectively tackle large-scale multi-objective optimization problems. The proposed approach ensures both the diversity of solutions, requested for high dimensionality of LSMOPs, and the balance between the exploitation and exploration during the optimization process. The experimental studies conducted on 54 instances of a large-scale multi-objective optimization problems test suite and the comparisons against well-known and state-of-the-art algorithms have shown the superiority of the proposed algorithm in terms of Pareto front approximation and computation time by using limited evaluation budget.

Guest Editorial Special Issue on Large-Scale Evolutionary Multiobjective Optimization and Its Practical Applications

Complex optimization problems with hundreds or even thousands of decision variables and dozens of conflicting objectives are not uncommon in the real world. In the past five years, increased research efforts have been dedicated to large-scale multiobjective optimization problems (LSMOPs) by using a variety of search strategies, including variable grouping, variable analysis, problem transformation, dimensionality reduction, and novel recombination operators. Despite the success of these efforts in solving some general LSMOPs, there still remains a big gap between the LSMOPs that have been addressed and those encountered in real life, such as sparse, highly constrained, dynamic, and expensive LMOPs, as well as very large-scale and many-objective optimization problems that are widely seen and of paramount importance for solving scientific and engineering problems. Due to the significant practical importance of large-scale multiobjective optimization, there is a high demand for computationally efficient and effective evolutionary algorithms for solving LSMOPs.