Large-scale Continuous Optimization Problems Research Articles

Constraint-Objective Cooperative Coevolution for Large-scale Constrained Optimization

Large-scale optimization problems and constrained optimization problems have attracted considerable attention in the swarm and evolutionary intelligence communities and exemplify two common features of real problems, i.e., a large scale and constraint limitations. However, only a little work on solving large-scale continuous constrained optimization problems exists. Moreover, the types of benchmarks proposed for large-scale continuous constrained optimization algorithms are not comprehensive at present. In this article, first, a constraint-objective cooperative coevolution (COCC) framework is proposed for large-scale continuous constrained optimization problems, which is based on the dual nature of the objective and constraint functions: modular and imbalanced components. The COCC framework allocates the computing resources to different components according to the impact of objective values and constraint violations. Second, a benchmark for large-scale continuous constrained optimization is presented, which takes into account the modular nature, as well as both imbalanced and overlapping characteristics of components. Finally, three different evolutionary algorithms are embedded into the COCC framework for experiments, and the experimental results show that COCC performs competitively.

A Three-Level Recursive Differential Grouping Method for Large-Scale Continuous Optimization

Cooperative co-evolution (CC) is widely used to solve large-scale continuous optimization problems, which divides a large-scale problem into several small-scale sub-problems via decomposition methods and then optimizes each sub-problem separately. However, the performance of CC mainly depends on the decomposition methods. A recently proposed bisection-based decomposition method, called recursive differential grouping (RDG), shows good performance when solving large-scale continuous optimization problems. In order to further improve the performance of RDG, this paper develops a novel decomposition method, called three-level recursive differential grouping (TRDG). In TRDG, when the interaction between two sets is detected, the variables in one of the sets are divided into three subsets based on the trichotomy method, and then the interaction between each subset and the other set is detected. Compared with RDG, TRDG can reduce the depth of recursion, thus saving the number of fitness evaluations (FEs). In addition, we devise a novel strategy to update adaptively the threshold for identifying the interactions between variables. The simulation experiment results on CEC’2010 and CEC’2013 benchmark functions show that the performance of TRDG is better than several existing decomposition methods in terms of the accuracy and the number of FEs. Furthermore, TRDG is embedded into two frameworks to tackle CEC’2010 large-scale continuous optimization problems.

Improved Multi-Population Differential Evolution for Large-Scale Global Optimization

Differential evolution (DE) is an efficient population-based search algorithm with good robustness, however, it is challenged to deal with high-dimensional problems. In this paper, we propose an improved multi-population differential evolution with best-and-current mutation strategy (mDE-bcM). The population is divided into three subpopulations based on the fitness values, each of subpopulations uses different mutation strategy. After crossover, mutation and selection, all subpopulations are updated based on the new fitness values of their individuals. An improved mutation strategy is proposed, which uses a new approach to generate base vector that is composed of the best individual and current individual. The performance of mDE-bcM is evaluated on a set of 19 large-scale continuous optimization problems, a comparative study is carried out with other state-of-the-art optimization techniques. The results show that mDE-bcM has a competitive performance compared to the contestant algorithms and better efficiency for large-scale optimization problems.

Large scale continuous global optimization based on micro differential evolution with local directional search

Over the years, many optimization algorithms have been developed to solve large-scale optimization problems accurately and efficiently. In this regard, Memetic Algorithms offer robust and efficient framework that hybridizes the Evolutionary Algorithms with a local heuristic search. In this work, we propose micro Differential Evolution with a Directional Local Search (µDSDE) algorithm using a small population size to solve large scale continuous optimization problems. In this technique, the best individual retains its position, the second best individual undergoes mutation and crossover processes of DE, and the rest are reinitialized on the search space. Exploration of the search is carried out with the dispersal of the worst individuals whereas exploitation is performed through DE operators and Directional Local Search (DLS). We conducted extensive empirical studies using two test suites on Large Scale Global Optimization benchmark with up to 5000 dimensions. The results show that µDSDE considerably outperforms existing solutions in terms of the convergence rate and solution quality.

Ranking-Based Differential Evolution for Large-Scale Continuous Optimization

Large-scale continuous optimization has gained considerable attention in recent years. Differential evolution (DE) is a simple yet efficient global numerical optimization algorithm, which has been successfully used in diverse fields. Generally, the vectors in the DE mutation operators are chosen randomly from the population. In this paper, we employ the ranking-based mutation operators for the DE algorithm to improve DE's performance. In the ranking-based mutation operators, the vectors are selected according to their rankings in the current population. The ranking-based mutation operators are general, and they are integrated into the original DE algorithm, GODE, and GaDE to verify the enhanced performance. Experiments have been conducted on the large-scale continuous optimization problems. The results indicate that the ranking-based mutation operators are able to enhance the overall performance of DE, GODE, and GaDE in the large-scale continuous optimization problems.

Deterministic metaheuristic based on fractal decomposition for large-scale optimization

In this work a new method based on geometric fractal decomposition to solve large-scale continuous optimization problems is proposed. It consists of dividing the feasible search space into sub-regions with the same geometrical pattern. At each iteration, the most promising ones are selected and further decomposed. This approach tends to provide a dense set of samples and has interesting theoretical convergence properties. Under some assumptions, this approach covers all the search space only in case of small dimensionality problems. The aim of this work is to propose a new algorithm based on this approach with low complexity and which performs well in case of large-scale problems. To do so, a low complex method that profits from fractals properties is proposed. Then, a deterministic optimization procedure is proposed using a single solution-based metaheuristic which is exposed to illustrate the performance of this strategy. Obtained results on common test functions were compared to those of algorithms from the literature and proved the efficiency of the proposed algorithm.

Multi-population differential evolution with balanced ensemble of mutation strategies for large-scale global optimization

Differential evolution (DE) is a simple, yet very effective, population-based search technique. However, it is challenging to maintain a balance between exploration and exploitation behaviors of the DE algorithm. In this paper, we boost the population diversity while preserving simplicity by introducing a multi-population DE to solve large-scale global optimization problems. In the proposed algorithm, called mDE-bES, the population is divided into independent subgroups, each with different mutation and update strategies. A novel mutation strategy that uses information from either the best individual or a randomly selected one is used to produce quality solutions to balance exploration and exploitation. Selection of individuals for some of the tested mutation strategies utilizes fitness-based ranks of these individuals. Function evaluations are divided into epochs. At the end of each epoch, individuals between the subgroups are exchanged to facilitate information exchange at a slow pace. The performance of the algorithm is evaluated on a set of 19 large-scale continuous optimization problems. A comparative study is carried out with other state-of-the-art optimization techniques. The results show that mDE-bES has a competitive performance and scalability behavior compared to the contestant algorithms.

Designing benchmark problems for large-scale continuous optimization

Three major sources of complexity in many real-world problems are size, variable interaction, and interdependence of the subcomponents of a problem. With the rapid growth in the size of businesses, the demand for solving large-scale complex problems will continue to grow. In this paper, we propose several major design features that need to be incorporated into large-scale optimization benchmark suites in order to better resemble the features of real-world problems. Non-uniform subcomponent sizes, imbalance between the contribution of various subcomponents of a problem, and the interaction between subcomponents by means of overlapping subcomponents are among these features. The proposed features are designed with the aim of closing the gap between the theory and practice of evolutionary techniques for solving large-scale continuous optimization problems. The general guidelines proposed in this paper can be used to design and construct various benchmark suites to meet different needs. The IEEE CEC’2013 large-scale global optimization benchmark suite [29] is one such implementation. The paper also contains a brief discussion on how the CEC’2013 benchmarks can be extended or modified for various purposes. Finally, a preliminary comparative study is conducted to showcase the performance of several state-of-the-art algorithms on the CEC’2013 large-scale benchmark problems.

Contributors

Contributors

An incremental particle swarm for large-scale continuous optimization problems: an example of tuning-in-the-loop (re)design of optimization algorithms

The development cycle of high-performance optimization algorithms requires the algorithm designer to make several design decisions. These decisions range from implementation details to the setting of parameter values for testing intermediate designs. Proper parameter setting can be crucial for the effective assessment of algorithmic components because a bad parameter setting can make a good algorithmic component perform poorly. This situation may lead the designer to discard promising components that just happened to be tested with bad parameter settings. Automatic parameter tuning techniques are being used by practitioners to obtain peak performance from already designed algorithms. However, automatic parameter tuning also plays a crucial role during the development cycle of optimization algorithms. In this paper, we present a case study of a tuning-in-the-loop approach for redesigning a particle swarm-based optimization algorithm for tackling large-scale continuous optimization problems. Rather than just presenting the final algorithm, we describe the whole redesign process. Finally, we study the scalability behavior of the final algorithm in the context of this special issue.

Self-adaptive differential evolution algorithm using population size reduction and three strategies

Many real-world optimization problems are large-scale in nature. In order to solve these problems, an optimization algorithm is required that is able to apply a global search regardless of the problems’ particularities. This paper proposes a self-adaptive differential evolution algorithm, called jDElscop, for solving large-scale optimization problems with continuous variables. The proposed algorithm employs three strategies and a population size reduction mechanism. The performance of the jDElscop algorithm is evaluated on a set of benchmark problems provided for the Special Issue on the Scalability of Evolutionary Algorithms and other Metaheuristics for Large Scale Continuous Optimization Problems. Non-parametric statistical procedures were performed for multiple comparisons between the proposed algorithm and three well-known algorithms from literature. The results show that the jDElscop algorithm can deal with large-scale continuous optimization effectively. It also behaves significantly better than other three algorithms used in the comparison, in most cases.

Restart particle swarm optimization with velocity modulation: a scalability test

Large scale continuous optimization problems are more relevant in current benchmarks since they are more representative of real-world problems (bioinformatics, data mining, etc.). Unfortunately, the performance of most of the available optimization algorithms deteriorates rapidly as the dimensionality of the search space increases. In particular, particle swarm optimization is a very simple and effective method for continuous optimization. Nevertheless, this algorithm usually suffers from unsuccessful performance on large dimension problems. In this work, we incorporate two new mechanisms into the particle swarm optimization with the aim of enhancing its scalability. First, a velocity modulation method is applied in the movement of particles in order to guide them within the region of interest. Second, a restarting mechanism avoids the early convergence and redirects the particles to promising areas in the search space. Experiments are carried out within the scope of this Special Issue to test scalability. The results obtained show that our proposal is scalable in all functions of the benchmark used, as well as numerically very competitive with regards to other compared optimizers.

Self-adaptive differential evolution with multi-trajectory search for large-scale optimization

In this paper, self-adaptive differential evolution (DE) is enhanced by incorporating the JADE mutation strategy and hybridized with modified multi-trajectory search (MMTS) algorithm (SaDE-MMTS) to solve large-scale continuous optimization problems. The JADE mutation strategy, the “DE/current-to-pbest” which is a variation of the classic “DE/current-to-best”, is used for generating mutant vectors. After the mutation phase, the binomial (uniform) crossover, the exponential crossover as well as no crossover option are used to generate each pair of target and trial vectors. By utilizing the self-adaptation in SaDE, both trial vector generation strategies and their associated control parameter values are gradually self-adapted by learning from their previous experiences in generating promising solutions. Consequently, suitable offspring generation strategy along with associated parameter settings will be determined adaptively to match different phases of the search process. MMTS is applied frequently to refine several diversely distributed solutions at different search stages satisfying both the global and the local search requirement. The initialization of step sizes is also defined by a self-adaption during every MMTS step. The success rates of both SaDE and the MMTS are determined and compared; consequently, future function evaluations for both search algorithms are assigned proportionally to their recent past performance. The proposed SaDE-MMTS is employed to solve the 19 numerical optimization problems in special issue of soft computing on scalability of evolutionary algorithms for large-scale continuous optimization problems and competitive results are presented.

A MOS-based dynamic memetic differential evolution algorithm for continuous optimization: a scalability test

Continuous optimization is one of the areas with more activity in the field of heuristic optimization. Many algorithms have been proposed and compared on several benchmarks of functions, with different performance depending on the problems. For this reason, the combination of different search strategies seems desirable to obtain the best performance of each of these approaches. This contribution explores the use of a hybrid memetic algorithm based on the multiple offspring framework. The proposed algorithm combines the explorative/exploitative strength of two heuristic search methods that separately obtain very competitive results. This algorithm has been tested with the benchmark problems and conditions defined for the special issue of the Soft Computing Journal on Scalability of Evolutionary Algorithms and other Metaheuristics for Large Scale Continuous Optimization Problems. The proposed algorithm obtained the best results compared with both its composing algorithms and a set of reference algorithms that were proposed for the special issue.

Editorial scalability of evolutionary algorithms and other metaheuristics for large-scale continuous optimization problems

This editorial note presents the motivations, objectives, and structure of the special issue on scalability of evolutionary algorithms and other metaheuristics for large-scale continuous optimization problems. In addition, it provides the link to an associated Website where complementary material to the special issue is available.