Large-scale Continuous Optimization Research Articles

A Flexible Ranking-Based Competitive Swarm Optimizer for Large-Scale Continuous Multi-Objective Optimization

A Flexible Ranking-Based Competitive Swarm Optimizer for Large-Scale Continuous Multi-Objective Optimization

A large-scale continuous optimization benchmark suite with versatile coupled heterogeneous modules

The complexity of the engineering design optimizations mainly sources from the large-scale nature of the problems, and the ever-increasing large-scale global optimization (LSGO) problems place high demands on the performance of evolutionary algorithms (EAs). To compare and analyze large-scale optimization algorithms, benchmarks that simulate the features from real-world large-scale optimization are desired. In this paper, we propose a novel large-scale continuous optimization benchmark suite, which includes 15 test functions with the modular structure. The proposed benchmark suite introduces two new features abstracted from the engineering design problems: (1) Heterogeneous design, i.e., the modules are significantly different in terms of difficulty and explicit expressions. (2) Versatile coupling, i.e., different coupling degrees and random coupling topologies of the benchmark are emphasized. Two typical cooperative coevolution frameworks and several state-of-the-art large-scale optimization algorithms are used for comparative studies on the proposed benchmark, and experimental results demonstrate that the proposed benchmark suite is very challenging.

An adaptive two-stage evolutionary algorithm for large-scale continuous multi-objective optimization

An adaptive two-stage evolutionary algorithm for large-scale continuous multi-objective optimization

Integrating Conjugate Gradients Into Evolutionary Algorithms for Large-Scale Continuous Multi-Objective Optimization

Large-scale multi-objective optimization problems (LSMOPs) pose challenges to existing optimizers since a set of well-converged and diverse solutions should be found in huge search spaces. While evolutionary algorithms are good at solving small-scale multi-objective optimization problems, they are criticized for low efficiency in converging to the optimums of LSMOPs. By contrast, mathematical programming methods offer fast convergence speed on large-scale single-objective optimization problems, but they have difficulties in finding diverse solutions for LSMOPs. Currently, how to integrate evolutionary algorithms with mathematical programming methods to solve LSMOPs remains unexplored. In this paper, a hybrid algorithm is tailored for LSMOPs by coupling differential evolution and a conjugate gradient method. On the one hand, conjugate gradients and differential evolution are used to update different decision variables of a set of solutions, where the former drives the solutions to quickly converge towards the Pareto front and the latter promotes the diversity of the solutions to cover the whole Pareto front. On the other hand, objective decomposition strategy of evolutionary multi-objective optimization is used to differentiate the conjugate gradients of solutions, and the line search strategy of mathematical programming is used to ensure the higher quality of each offspring than its parent. In comparison with state-of-the-art evolutionary algorithms, mathematical programming methods, and hybrid algorithms, the proposed algorithm exhibits better convergence and diversity performance on a variety of benchmark and real-world LSMOPs.

Constraint-Objective Cooperative Coevolution for Large-scale Constrained Optimization

Large-scale optimization problems and constrained optimization problems have attracted considerable attention in the swarm and evolutionary intelligence communities and exemplify two common features of real problems, i.e., a large scale and constraint limitations. However, only a little work on solving large-scale continuous constrained optimization problems exists. Moreover, the types of benchmarks proposed for large-scale continuous constrained optimization algorithms are not comprehensive at present. In this article, first, a constraint-objective cooperative coevolution (COCC) framework is proposed for large-scale continuous constrained optimization problems, which is based on the dual nature of the objective and constraint functions: modular and imbalanced components. The COCC framework allocates the computing resources to different components according to the impact of objective values and constraint violations. Second, a benchmark for large-scale continuous constrained optimization is presented, which takes into account the modular nature, as well as both imbalanced and overlapping characteristics of components. Finally, three different evolutionary algorithms are embedded into the COCC framework for experiments, and the experimental results show that COCC performs competitively.

A Three-Level Recursive Differential Grouping Method for Large-Scale Continuous Optimization

Cooperative co-evolution (CC) is widely used to solve large-scale continuous optimization problems, which divides a large-scale problem into several small-scale sub-problems via decomposition methods and then optimizes each sub-problem separately. However, the performance of CC mainly depends on the decomposition methods. A recently proposed bisection-based decomposition method, called recursive differential grouping (RDG), shows good performance when solving large-scale continuous optimization problems. In order to further improve the performance of RDG, this paper develops a novel decomposition method, called three-level recursive differential grouping (TRDG). In TRDG, when the interaction between two sets is detected, the variables in one of the sets are divided into three subsets based on the trichotomy method, and then the interaction between each subset and the other set is detected. Compared with RDG, TRDG can reduce the depth of recursion, thus saving the number of fitness evaluations (FEs). In addition, we devise a novel strategy to update adaptively the threshold for identifying the interactions between variables. The simulation experiment results on CEC’2010 and CEC’2013 benchmark functions show that the performance of TRDG is better than several existing decomposition methods in terms of the accuracy and the number of FEs. Furthermore, TRDG is embedded into two frameworks to tackle CEC’2010 large-scale continuous optimization problems.

Improved Multi-Population Differential Evolution for Large-Scale Global Optimization

Differential evolution (DE) is an efficient population-based search algorithm with good robustness, however, it is challenged to deal with high-dimensional problems. In this paper, we propose an improved multi-population differential evolution with best-and-current mutation strategy (mDE-bcM). The population is divided into three subpopulations based on the fitness values, each of subpopulations uses different mutation strategy. After crossover, mutation and selection, all subpopulations are updated based on the new fitness values of their individuals. An improved mutation strategy is proposed, which uses a new approach to generate base vector that is composed of the best individual and current individual. The performance of mDE-bcM is evaluated on a set of 19 large-scale continuous optimization problems, a comparative study is carried out with other state-of-the-art optimization techniques. The results show that mDE-bcM has a competitive performance compared to the contestant algorithms and better efficiency for large-scale optimization problems.

Large-Scale Estimation of Distribution Algorithms with Adaptive Heavy Tailed Random Projection Ensembles

We present new variants of Estimation of Distribution Algorithms (EDA) for large-scale continuous optimisation that extend and enhance a recently proposed random projection (RP) ensemble based approach. The main novelty here is to depart from the theory of RPs that require (sub-)Gaussian random matrices for norm-preservation, and instead for the purposes of high-dimensional search we propose to employ random matrices with independent and identically distributed entries drawn from a t-distribution. We analytically show that the implicitly resulting high-dimensional covariance of the search distribution is enlarged as a result. Moreover, the extent of this enlargement is controlled by a single parameter, the degree of freedom. For this reason, in the context of optimisation, such heavy tailed random matrices turn out to be preferable over the previously employed (sub-)Gaussians. Based on this observation, we then propose novel covariance adaptation schemes that are able to adapt the degree of freedom parameter during the search, and give rise to a flexible approach to balance exploration versus exploitation. We perform a thorough experimental study on high-dimensional benchmark functions, and provide statistical analyses that demonstrate the state-of-the-art performance of our approach when compared with existing alternatives in problems with 1 000 search variables.

ADMM-based problem decomposition scheme for vehicle routing problem with time windows

Emerging urban logistics applications need to address various challenges, including complex traffic conditions and time-sensitive requirements. In this study, in the context of urban logistics, we consider a vehicle routing problem with time-dependent travel times and time windows (VRPTW), and the goal is to minimize the total generalized costs including the transportation, waiting time, and fixed costs associated with each vehicle. We adopt a high-dimensional space–time network flow model to formulate an underlying vehicle routing problem (VRP) with a rich set of criteria and constraints. A difficult issue, when solving VRPs, is how to iteratively improve both the primal and dual solution quality in general and how to break the symmetry generated by many identical solutions, particularly with homogeneous vehicles. Along this line, many coupling constraints, such as the consensus constraints across different agents or decision makers, need to be carefully addressed to find high-quality optimal or close-to-optimal solutions under medium- or large-scale instances. Currently, the alternating direction method of multipliers (ADMM) is widely used in the field of convex optimization, as an integration of the augmented Lagrangian relaxation and block coordinate descent methods, for machine learning and large-scale continuous systems optimization and control. In this work, we introduce the use of ADMM to solve the multi-VRP, which is a special case of integer linear programming, and demonstrate a manner to reduce the quadratic penalty terms used in ADMM into simple linear functions. In a broader context, a computationally reliable decomposition framework is developed to iteratively improve both the primal and dual solution quality. Essentially, the least-cost path subproblem or other similar subproblems involving binary decisions can be embedded into a sequential solution scheme with an output of both lower bound estimates and upper bound feasible solutions. We examine the performance of the proposed approach using classical Solomon VRP benchmark instances. We also evaluate our approach on a real-world instance based on a problem-solving competition by Jingdong Logistics, a major E-commerce company.

Large scale continuous global optimization based on micro differential evolution with local directional search

Over the years, many optimization algorithms have been developed to solve large-scale optimization problems accurately and efficiently. In this regard, Memetic Algorithms offer robust and efficient framework that hybridizes the Evolutionary Algorithms with a local heuristic search. In this work, we propose micro Differential Evolution with a Directional Local Search (µDSDE) algorithm using a small population size to solve large scale continuous optimization problems. In this technique, the best individual retains its position, the second best individual undergoes mutation and crossover processes of DE, and the rest are reinitialized on the search space. Exploration of the search is carried out with the dispersal of the worst individuals whereas exploitation is performed through DE operators and Directional Local Search (DLS). We conducted extensive empirical studies using two test suites on Large Scale Global Optimization benchmark with up to 5000 dimensions. The results show that µDSDE considerably outperforms existing solutions in terms of the convergence rate and solution quality.

A Recursive Decomposition Method for Large Scale Continuous Optimization

Cooperative co-evolution (CC) is an evolutionary computation framework that can be used to solve high-dimensional optimization problems via a “divide-and-conquer” mechanism. However, the main challenge when using this framework lies in problem decomposition. That is, deciding how to allocate decision variables to a particular subproblem, especially interacting decision variables. Existing decomposition methods are typically computationally expensive. In this paper, we propose a new decomposition method, which we call recursive differential grouping (RDG), by considering the interaction between decision variables based on nonlinearity detection. RDG recursively examines the interaction between a selected decision variable and the remaining variables, placing all interacting decision variables into the same subproblem. We use analytical methods to show that RDG can be used to efficiently decompose a problem, without explicitly examining all pairwise variable interactions. We evaluated the efficacy of the RDG method using large scale benchmark optimization problems. Numerical simulation experiments showed that RDG greatly improved the efficiency of problem decomposition in terms of time complexity. Significantly, when RDG was embedded in a CC framework, the optimization results were better than results from seven other decomposition methods.

Ranking-Based Differential Evolution for Large-Scale Continuous Optimization

Large-scale continuous optimization has gained considerable attention in recent years. Differential evolution (DE) is a simple yet efficient global numerical optimization algorithm, which has been successfully used in diverse fields. Generally, the vectors in the DE mutation operators are chosen randomly from the population. In this paper, we employ the ranking-based mutation operators for the DE algorithm to improve DE's performance. In the ranking-based mutation operators, the vectors are selected according to their rankings in the current population. The ranking-based mutation operators are general, and they are integrated into the original DE algorithm, GODE, and GaDE to verify the enhanced performance. Experiments have been conducted on the large-scale continuous optimization problems. The results indicate that the ranking-based mutation operators are able to enhance the overall performance of DE, GODE, and GaDE in the large-scale continuous optimization problems.

DG2: A Faster and More Accurate Differential Grouping for Large-Scale Black-Box Optimization

Identification of variable interaction is essential for an efficient implementation of a divide-and-conquer algorithm for large-scale black-box optimization. In this paper, we propose an improved variant of the differential grouping (DG) algorithm, which has a better efficiency and grouping accuracy. The proposed algorithm, DG2, finds a reliable threshold value by estimating the magnitude of roundoff errors. With respect to efficiency, DG2 reuses the sample points that are generated for detecting interactions and saves up to half of the computational resources on fully separable functions. We mathematically show that the new sampling technique achieves the lower bound with respect to the number of function evaluations. Unlike its predecessor, DG2 checks all possible pairs of variables for interactions and has the capacity to identify overlapping components of an objective function. On the accuracy aspect, DG2 outperforms the state-of-the-art decomposition methods on the latest large-scale continuous optimization benchmark suites. DG2 also performs reliably in the presence of imbalance among contribution of components in an objective function. Another major advantage of DG2 is the automatic calculation of its threshold parameter ( $\epsilon $ ), which makes it parameter-free. Finally, the experimental results show that when DG2 is used within a cooperative co-evolutionary framework, it can generate competitive results as compared to several state-of-the-art algorithms.

Deterministic metaheuristic based on fractal decomposition for large-scale optimization

In this work a new method based on geometric fractal decomposition to solve large-scale continuous optimization problems is proposed. It consists of dividing the feasible search space into sub-regions with the same geometrical pattern. At each iteration, the most promising ones are selected and further decomposed. This approach tends to provide a dense set of samples and has interesting theoretical convergence properties. Under some assumptions, this approach covers all the search space only in case of small dimensionality problems. The aim of this work is to propose a new algorithm based on this approach with low complexity and which performs well in case of large-scale problems. To do so, a low complex method that profits from fractals properties is proposed. Then, a deterministic optimization procedure is proposed using a single solution-based metaheuristic which is exposed to illustrate the performance of this strategy. Obtained results on common test functions were compared to those of algorithms from the literature and proved the efficiency of the proposed algorithm.

A new algorithm for adapting the configuration of subcomponents in large-scale optimization with cooperative coevolution

The cooperative coevolutionary (CC) approach can be very effective in solving problems of large-scale continuous optimization (LSGO) through their decomposition into lower-dimensional subcomponents. However, it is well known that the CC performance can be significantly influenced by the adopted decomposition. Moreover, since the method may require evolving a number of populations, also the size of the latter can largely affect the optimization process. In this article, focusing on equally sized decompositions, we present the results of an in-depth investigation concerning the effects of both the size of populations and the dimensionality of subcomponents on the performance of a CC optimizer. According to our study, in several cases only a small set of suitable configurations corresponds to a high optimization performance. Furthermore, we propose a new CC algorithm in which part of the available computational budget is spent for adapting both the dimensionality of subcomponents and the number of evolved individuals during the optimization process. Using a rich set of benchmark problems, we show that the proposed approach can outperform a state-of-the art algorithm based on adaptive equally sized decompositions.

Toward Large-Scale Continuous EDA: A Random Matrix Theory Perspective.

Estimations of distribution algorithms (EDAs) are a major branch of evolutionary algorithms (EA) with some unique advantages in principle. They are able to take advantage of correlation structure to drive the search more efficiently, and they are able to provide insights about the structure of the search space. However, model building in high dimensions is extremely challenging, and as a result existing EDAs may become less attractive in large-scale problems because of the associated large computational requirements. Large-scale continuous global optimisation is key to many modern-day real-world problems. Scaling up EAs to large-scale problems has become one of the biggest challenges of the field. This paper pins down some fundamental roots of the problem and makes a start at developing a new and generic framework to yield effective and efficient EDA-type algorithms for large-scale continuous global optimisation problems. Our concept is to introduce an ensemble of random projections to low dimensions of the set of fittest search points as a basis for developing a new and generic divide-and-conquer methodology. Our ideas are rooted in the theory of random projections developed in theoretical computer science, and in developing and analysing our framework we exploit some recent results in nonasymptotic random matrix theory.

Multi-population differential evolution with balanced ensemble of mutation strategies for large-scale global optimization

Differential evolution (DE) is a simple, yet very effective, population-based search technique. However, it is challenging to maintain a balance between exploration and exploitation behaviors of the DE algorithm. In this paper, we boost the population diversity while preserving simplicity by introducing a multi-population DE to solve large-scale global optimization problems. In the proposed algorithm, called mDE-bES, the population is divided into independent subgroups, each with different mutation and update strategies. A novel mutation strategy that uses information from either the best individual or a randomly selected one is used to produce quality solutions to balance exploration and exploitation. Selection of individuals for some of the tested mutation strategies utilizes fitness-based ranks of these individuals. Function evaluations are divided into epochs. At the end of each epoch, individuals between the subgroups are exchanged to facilitate information exchange at a slow pace. The performance of the algorithm is evaluated on a set of 19 large-scale continuous optimization problems. A comparative study is carried out with other state-of-the-art optimization techniques. The results show that mDE-bES has a competitive performance and scalability behavior compared to the contestant algorithms.

Designing benchmark problems for large-scale continuous optimization

Three major sources of complexity in many real-world problems are size, variable interaction, and interdependence of the subcomponents of a problem. With the rapid growth in the size of businesses, the demand for solving large-scale complex problems will continue to grow. In this paper, we propose several major design features that need to be incorporated into large-scale optimization benchmark suites in order to better resemble the features of real-world problems. Non-uniform subcomponent sizes, imbalance between the contribution of various subcomponents of a problem, and the interaction between subcomponents by means of overlapping subcomponents are among these features. The proposed features are designed with the aim of closing the gap between the theory and practice of evolutionary techniques for solving large-scale continuous optimization problems. The general guidelines proposed in this paper can be used to design and construct various benchmark suites to meet different needs. The IEEE CEC’2013 large-scale global optimization benchmark suite [29] is one such implementation. The paper also contains a brief discussion on how the CEC’2013 benchmarks can be extended or modified for various purposes. Finally, a preliminary comparative study is conducted to showcase the performance of several state-of-the-art algorithms on the CEC’2013 large-scale benchmark problems.

A Cooperative Coevolutionary Differential Evolution Algorithm with Adaptive Subcomponents

The performance of cooperative co-evolutionary (CC) algorithms for large-scale continuous optimization is significantly affected by the adopted decomposition of the search space. According to the literature, a typical decomposition in case of separable problems consists of adopting equally sized subcomponents for the whole optimization process (i.e. static decomposition). Such an approach is also often used for non-separable problems, together with a random-grouping strategy. More advanced methods try to determine the optimal size of subcomponents during the optimization process using reinforcement-learning techniques. However, the latter approaches are not always suitable in this case because of the non-stationary and history-dependent nature of the learning environment. This paper investigates a new CC algorithm, based on Differential Evolution, in which several decompositions are applied in parallel during short learning phases. The experimental results on a set of large-scale optimization problems show that the proposed method can lead to a reliable estimate of the suitability of each subcomponent size. Moreover, in some cases it outperforms the best static decomposition.

Parallel ant colony optimisation algorithm for continuous domains on graphics processing unit

A novel parallel approach to run continuous ant colony optimisation CACO algorithm on graphic processing unit GPU is presented in this paper for solving large scale continuous optimisation problem. CACO which is an extension to continuous domains from standard ACO is a kind of population-based meta-heuristics in essence. The mechanism of algorithm is described in detail. Its parallel implementation on compute unified device architecture CUDA is proposed in our work. The experiment results on actual hardware to optimise many-dimensions test functions are given. The results and analyses show the excellent performance of algorithm.