Large-scale Optimization Algorithms Research Articles

A two-stage direction-guided evolutionary algorithm for large-scale multiobjective optimization

Large-scale multiobjective optimization problems (LSMOPs) have exponential growth in the search space as the decision variables increase, and the vast search space poses a challenge to the performance of multiobjective evolutionary algorithms (MOEAs). Many current large-scale MOEAs need to consume a large amount of computational resources to get good performance. This paper proposes a two-stage direction-guided evolutionary algorithm for large-scale multiobjective optimization (LMOEA-S2D) to balance the performance and computational resource overhead. The algorithm exploits the Pareto-optimality property of domination and the diversity-preserving property of decomposition to optimize the performance in the two stages, respectively, and designs a corresponding direction-guided mechanism to improve search efficiency. LMOEA-S2D designs global direction search and local direction search in the domination-based stage for efficient exploitation to accelerate population convergence. To promote greater population diversity, a hybrid direction search was devised to aid diversity exploration in the decomposition-based stage, and this facilitates even distribution of candidate solutions across the Pareto optimal frontier. LMOEA-S2D is compared with five state-of-the-art large-scale MOEAs on some large-scale multiobjective test suites with 100 to 5,000 decision variables. The experimental results show that LMOEA-S2D significantly outperformed all compared algorithms under limited computational resources.

A similarity-detection-based evolutionary algorithm for large-scale multimodal multi-objective optimization

In recent years, there has been a surge in the development of evolutionary algorithms tailored for multimodal multi-objective optimization problems (MMOPs). These algorithms aim to find multiple equivalent Pareto optimal solution sets (PSs). However, little work has been done on MMOPs with large-scale decision variables, especially when the Pareto optimal solutions are sparse. These problems pose significant challenges due to the dimension curse, the unknown sparsity, and the unknown number of equivalent PSs. In this paper, we propose an evolutionary algorithm based on similarity detection called SD-MMEA to solve large-scale MMOPs with sparse Pareto-optimal solutions. Specifically, it employs a multi-population independent evolution to explore multiple PSs and distinguishes different PSs by double detection of the similarity between subpopulations. Simultaneously, develop online scoring mechanisms for decision variables to guide the subpopulations to explore in different directions. In addition, during the latter stage of evolution, the decision variables of individuals are further optimized by a double-layer grouping process. The proposed algorithm is compared with six state-of-the-art algorithms. Experimental results show that SD-MMEA has significant advantages in solving large-scale MMOPs with sparse solutions.

Improving Performance Insensitivity of Large-Scale Multiobjective Optimization via Monte Carlo Tree Search.

The large-scale multiobjective optimization problem (LSMOP) is characterized by simultaneously optimizing multiple conflicting objectives and involving hundreds of decision variables. Many real-world applications in engineering can be modeled as LSMOPs; simultaneously, engineering applications require insensitivity in performance. This requirement typically means that the algorithm should not only produce good results in terms of performance for every run but also the performance of multiple runs should not fluctuate too much. However, existing large-scale multiobjective optimization algorithms often focus on improving algorithm performance, but pay little attention to improving the insensitivity characteristic of algorithms. This directly leads to substantial limitations when solving practical problems. In this work, we propose an evolutionary algorithm called large-scale multiobjective optimization algorithm via Monte Carlo tree search, which is based on the Monte Carlo tree search and aims to improve the performance and insensitivity of solving LSMOPs. The proposed method samples decision variables to construct new nodes on the Monte Carlo tree for optimization and evaluation, and it selects nodes with good evaluations for further searches in order to reduce the performance sensitivity caused by large-scale decision variables. We propose two metrics to measure the sensitivity of the algorithm and compare the proposed algorithm with several state-of-the-art designs on different benchmark functions and metrics. The experimental results confirm the effectiveness and performance insensitivity of the proposed design for solving LSMOPs.

A multi-granularity clustering based evolutionary algorithm for large-scale sparse multi-objective optimization

Sparse multi-objective optimization problems (SMOPs) frequently exist in a variety of disciplines such as machine learning, economy, and signal processing. Evolutionary algorithms have demonstrated their proficiency in optimizing complex problems in recent years, although their performance often deteriorates significantly on large-scale SMOPs. In an effort to accelerate the convergence, this paper suggests a multi-granularity variable clustering method for evolutionary algorithms. This method estimates the sparse distribution of decision variables at each generation and partitions them into a varying number of layers, each with a distinct probability of being zero. These clustering outcomes inspire the development of a crossover operator and a mutation operator, which prove adept at efficiently generating sparse solutions. Experimental evaluations on both benchmark and real-world SMOPs confirm that an evolutionary algorithm incorporating the new crossover operator and mutation operator converges more rapidly than its state-of-the-art counterparts.

Large-scale evolutionary optimization: A review and comparative study

Large-scale global optimization (LSGO) problems have widely appeared in various real-world applications. However, their inherent complexity, coupled with the curse of dimensionality, makes them challenging to solve. Continuous efforts have been devoted to designing computational intelligence-based approaches to solve them. This paper offers a comprehensive review of the latest developments in the field, focusing on the advances in both single-objective and multi-objective large-scale evolutionary optimization algorithms over the past five years. We systematically categorize these algorithms, discuss their distinct features, and highlight benchmark test suites essential for performance evaluation. After that, comparative studies are conducted using numerical solutions to evaluate the performance of state-of-the-art LSGO for both single-objective and multi-objective problems. Finally, we discuss the real-world applications of LSGO, some challenges, and possible future research directions.

A Two-Population Algorithm for Large-Scale Multi-objective Optimization Based on Fitness-Aware Operator and Adaptive Environmental Selection

A Two-Population Algorithm for Large-Scale Multi-objective Optimization Based on Fitness-Aware Operator and Adaptive Environmental Selection

Large-Scale Multi-Objective Optimization Algorithm Based on Weighted Overlapping Grouping of Decision Variables

Large-Scale Multi-Objective Optimization Algorithm Based on Weighted Overlapping Grouping of Decision Variables

A problem transformation-based and decomposition-based evolutionary algorithm for large-scale multiobjective optimization

For large-scale multi-objective optimization problems, the search space becomes exceptionally vast, resulting in increased complexity in the search process. The search space usually contains several local optimal individuals, and the difficulty of finding the global optimal individuals increases greatly. A problem transformation-based and decomposition-based large-scale multi-objective evolutionary algorithm is proposed to solve the problem of reducing the dimensionality of the search space and the optimization of populations. Reducing the number of optimization problems in the decision space through problem transformation has lowered the complexity of multi-objective optimization problems and enhanced computational efficiency. In the decision space, employing two direction vectors adaptively guides the generation of promising individuals, thereby preventing the population from falling into local optima. Due to the dimensionality reduction of the decision space and the optimization of the individuals in the objective space, a set of optimal solutions are effectively obtained and uniformly distributed on the approximate Pareto optimal front. Experimental results show that the algorithm is highly competitive with five large-scale multi-objective evolutionary algorithms for large-scale multi-objective optimization test problems with up to 2000 decision variables.

Parallel-in-time multiple shooting for optimal control problems governed by the Navier–Stokes equations

In the past decades, multiple shooting methods have proven to be a promising direction to speed up the optimization process, especially in the context of ODE-based optimization. Very recently, Fang et al. (Journal of Computational Physics, vol. 452, 110926, 2022) proposed a multiple shooting algorithm for large-scale PDE-constrained optimization. The current paper continues this line of work and explores the potential of multiple shooting methods for optimal control problems governed by the three-dimensional Navier–Stokes equations. Similar to Fang et al., the augmented Lagrangian (AL) method is used to solve the resulting equality-constrained optimization problem, and we employ the classical limited-memory BFGS method for the unconstrained subproblems inside the AL loop. In the current work, we exploit the multiple shooting paradigm in full by processing the shooting windows parallel-in-time, allowing for significant parallel speed-ups compared to single shooting. The proposed method is validated on a velocity tracking case, using up to 100 windows. Our analysis shows that the multiple shooting algorithm allows for considerable algorithmic and parallel speed-ups. While algorithmic speed-up depends on the exact tracking case and initialization of the shooting windows, the multiple shooting algorithm always outperforms single shooting in terms of computational time (if the number of windows is sufficiently high) due to the parallel-in-time implementation. For a given amount of resources, we also show that the proposed parallel-in-time strategy can outperform spatial parallelization alone, especially when the spatial parallelization is saturated.

Coil sketching for computationally efficient MR iterative reconstruction.

Parallel imaging and compressed sensing reconstructions of large MRI datasets often have a prohibitive computational cost that bottlenecks clinical deployment, especially for three-dimensional (3D) non-Cartesian acquisitions. One common approach is to reduce the number of coil channels actively used during reconstruction as in coil compression. While effective for Cartesian imaging, coil compression inherently loses signal energy, producing shading artifacts that compromise image quality for 3D non-Cartesian imaging. We propose coil sketching, a general and versatile method for computationally-efficient iterative MR image reconstruction. We based our method on randomized sketching algorithms, a type of large-scale optimization algorithms well established in the fields of machine learning and big data analysis. We adapt the sketching theory to the MRI reconstruction problem via a structured sketching matrix that, similar to coil compression, considers high-energy virtual coils obtained from principal component analysis. But, unlike coil compression, it also considers random linear combinations of the remaining low-energy coils, effectively leveraging information from all coils. First, we performed ablation experiments to validate the sketching matrix design on both Cartesian and non-Cartesian datasets. The resulting design yielded both improved computatioanal efficiency and preserved signal-to-noise ratio (SNR) as measured by the inverse g-factor. Then, we verified the efficacy of our approach on high-dimensional non-Cartesian 3D cones datasets, where coil sketching yielded up to three-fold faster reconstructions with equivalent image quality. Coil sketching is a general and versatile reconstruction framework for computationally fast and memory-efficient reconstruction.

Stochastic Localization Methods for Convex Discrete Optimization via Simulation

Solving Convex Discrete Optimization via Simulation via Stochastic Localization Algorithms Many decision-making problems in operations research and management science require the optimization of large-scale complex stochastic systems. For a number of applications, the objective function exhibits convexity in the discrete decision variables or the problem can be transformed into a convex one. In “Stochastic Localization Methods for Convex Discrete Optimization via Simulation,” Zhang, Zheng, and Lavaei propose provably efficient simulation-optimization algorithms for general large-scale convex discrete optimization via simulation problems. By utilizing the convex structure and the idea of localization and cutting-plane methods, the developed stochastic localization algorithms demonstrate a polynomial dependence on the dimension and scale of the decision space. In addition, the simulation cost is upper bounded by a value that is independent of the objective function. The stochastic localization methods also exhibit a superior numerical performance compared with existing algorithms.

Difficulty and Contribution-Based Cooperative Coevolution for Large-Scale Optimization

Cooperative coevolution (CC) is a paradigm equipped with the divide-and-conquer strategy for solving large-scale optimization problems. Currently, the computational resource allocation schemes of most CC could be divided into two categories, namely equal allocation to all subproblems and preference allocation to the subproblems with large contribution. However, the difficult subproblems are not carefully considered by the existing computational resource allocation schemes. For these subproblems, the investment of computational resources cannot quickly improve the fitness value, which leads to their small early contribution and being neglected. In this paper, we comprehensively analyze the imbalanced nature of the subproblems from their difficulty and contribution in large-scale optimization problems. First, we propose a method to quantify the optimization difficulty of the problems during the evolution process, which considers both the difficulty of the fitness landscape and the behaviors of the optimization algorithm. Then, we propose a novel both difficulty and contribution based CC framework, called DCCC, which encourages the allocation of the computational resources to more contributing and more difficult subproblems. DCCC is tested on the CEC’2010 and CEC’2013 large-scale optimization benchmarks, and is compared with several typical CC frameworks and state-of-the-art large-scale optimization algorithms. The experimental results demonstrate that DCCC is very competitive.

Cooperative tri-population based evolutionary algorithm for large-scale multi-objective optimization

Cooperative tri-population based evolutionary algorithm for large-scale multi-objective optimization

A two-space-decomposition-based evolutionary algorithm for large-scale multiobjective optimization

Large-scale multiobjective optimization problems (LSMOPs) pose a great challenge to maintaining the diversity of solutions. However, existing large-scale multiobjective optimization algorithms (MOEAs) prefer to directly use environmental selection methods designed for small-scale optimization problems. These methods are not effective in solving complex LSMOPs. To address this issue, this paper proposes a two-space (decision space and objective space) decomposition (TSD)-based diversity maintenance mechanism. Its main idea is to explicitly decompose the decision space and objective space into a number of subspaces, each of which may contain some Pareto-optimal solutions. Searching for Pareto-optimal solutions in these subspaces may help maintain the diversity of solutions. To this end, a diversity design subspace (DDS) is constructed to decompose the decision space. Then, a large-scale MOEA (MOEA/TSD) is designed by using the proposed TSD-based diversity maintenance mechanism. Experimental studies validate the effectiveness of the proposed TSD mechanism. Compared with nine state-of-the-art large-scale MOEAs on 112 benchmark LSMOPs, our proposed algorithm offers considerable advantages in overall optimization performance. The source code of MOEA/TSD is available at https://github.com/yizhizhede/MOEATSD.

A learning and potential area-mining evolutionary algorithm for large-scale multi-objective optimization

For large-scale multi-objective optimization problems, it is not easy for existing multi-objective evolutionary algorithms to search the entire decision variable space with limited computation resources. To alleviate this problem, we propose a learning and potential area-mining evolutionary algorithm to explore and exploit key regions for accelerating optimization. First, we mine promising areas by clustering the population-gathered regions. Then, potential directions are determined by a multi-guiding point scheme in these promising areas. Subsequently, we design a local search and global search scheme to enhance population convergence while ensuring diversity. Finally, a mutation strategy is used to improve diversity. We execute numerical experiments on two widely-used LSMOP benchmarks and compare the proposed algorithm with three state-of-the-art algorithms. The statistical results indicate that the proposed algorithm has significant performance.

Large-Scale Multi-Objective Imaging Satellite Task Planning Algorithm for Vast Area Mapping

With satellite quantity and quality development in recent years, remote sensing products in vast areas are becoming widely used in more and more fields. The acquisition of large regional images requires the scientific and efficient utilization of satellite resources through imaging satellite task planning technology. However, for imaging satellite task planning in a vast area, a large number of decision variables are introduced into the imaging satellite task planning model, making it difficult for existing optimization algorithms to obtain reliable solutions. This is because the search space of the solution increases the exponential growth with the increase in the number of decision variables, which causes the search performance of optimization algorithms to decrease significantly. This paper proposes a large-scale multi-objective optimization algorithm based on efficient competition learning and improved non-dominated sorting (ECL-INS-LMOA) to efficiently obtain satellite imaging schemes for large areas. ECL-INS-LMOA adopted the idea of two-stage evolution to meet the different needs in different evolutionary stages. In the early stage, the proposed efficient competitive learning particle update strategy (ECLUS) and the improved NSGA-II were run alternately. In the later stage, only the improved NSGA-II was run. The proposed ECLUS guarantees the rapid convergence of ECL-INS-LMOA in the early evolution by accelerating particle update, introducing flight time, and proposing a binary competitive swarm optimizer BCSO. The results of the simulation imaging experiments on five large areas with different scales of decision variables show that ECL-INS-LMOA can always obtain the imaging satellite mission planning scheme with the highest regional coverage and the lowest satellite resource consumption within the limited evaluation times. The experiments verify the excellent performance of ECL-INS-LMOA in solving vast area mapping planning problems.

A resource allocation-based multi-objective evolutionary algorithm for large-scale multi-objective optimization

A resource allocation-based multi-objective evolutionary algorithm for large-scale multi-objective optimization

A Population Cooperation based Particle Swarm Optimization algorithm for large-scale multi-objective optimization

There are many multi-objective optimization problems (MOPs) in real life that contain a large number of decision variables, such as auto body parts design, financial investment, engineering design, adversarial textual attack and so on. These problems are known as large-scale multi-objective optimization problems (LSMOPs). Due to the curse of dimensionality, existing multi-objective evolutionary algorithm encounter difficulties in balancing convergence and diversity on LSMOPs. In this paper, a Population Cooperation based Particle Swarm Optimization algorithm (PCPSO) is proposed for tackling LSMOPs. To be specific, PCPSO is a two-stage optimizer with two key components: (1) In the first stage, an inter-population collaboration component named Auxiliary Population Cooperation (APC) is used to improve the convergence speed. (2) In the second stage, an intra-subpopulation collaboration component called SubPopulation Cooperation (SPC) is applied to balance convergence and diversity. Experimental results on benchmark problems with up to 5000 decision variables and 2, 3, 5, 10 objectives demonstrate that the proposed PCPSO achieves better performance than several state-of-the-art large-scale multi-objective evolutionary algorithms (LSMOEAs) on most test problems.

Gene Targeting Differential Evolution: A Simple and Efficient Method for Large-Scale Optimization

Large-scale optimization problems (LSOPs) are challenging because the algorithm is difficult in balancing too many dimensions and in escaping from trapped bottleneck dimensions. To improve solutions, this paper introduces targeted modification to the certain values in the bottleneck dimensions. Analogous to gene targeting (GT) in biotechnology, we experiment on targeting the specific genes in candidate solution to improve its trait in differential evolution (DE). We propose a simple and efficient method, called GT-based DE (GTDE), to solve LSOPs. In the algorithm design, a simple GT-based modification is developed to perform on the best individual, comprising probabilistically targeting the location of bottleneck dimensions, constructing a homologous targeting vector, and inserting the targeting vector into the best individual. In this way, all the bottleneck dimensions of the best individual can be probabilistically targeted and modified to break the bottleneck and to provide global guidance for more optimal evolution. Note that the GT is only performed on the globally best individual and is just carried out as a simple operator that is added to the standard DE. Experimental studies compare the GTDE with some other state-of-the-art large-scale optimization algorithms, including the winners of CEC2010, CEC2012, CEC2013, and CEC2018 competitions on large-scale optimization. The results show that the GTDE is efficient and performs better than or at least comparable to the others in solving LSOPs.

An inexact quasi-Newton algorithm for large-scale ℓ1 optimization with box constraints

In this paper, we develop an inexact quasi-Newton algorithm for ℓ1-regularization optimization problems subject to box constraints. The algorithm uses the identification technique of the proximal gradient algorithm to estimate the active set and free variables. To accelerate the convergence, we utilize the inexact quasi-Newton algorithm to update free variables. Under certain conditions, we show that the sequence generated by the algorithm converges R-linearly to a first-order optimality point of the problem. Moreover, the corresponding sequence of objective function values is also linearly convergent. Experiment results demonstrate the competitiveness of the proposed algorithm.