Large-scale Global Optimization Algorithms Research Articles

A space-reduction based three-phase approach for large-scale optimization

With the rapid development of science and technology, large-scale global optimization (LSGO) has received more and more attention. However, large-scale problem is very challenging due to its high dimensionality, nonlinearity, and countless local optimal solutions. In this paper, a space-reduction based three-phase approach (SRTP) is proposed to solve this problem. In the first phase, a space-reduction based line search method is designed to roughly locate good solutions as well as to explore important information such as promising search regions and characteristics of each variable (dimension). In the second phase, a multi-grouping strategy is proposed for fully non-separable large-scale problems. This new decomposition method provides dozens of grouping results for non-separable large-scale problems that not only reduces the dimensionality but also maintains the correlation of variables to the most extent. Then in the third phase, a space-reduction based group search method is designed to optimize the sub-groups. By decomposing the large-scale problem as well as reducing the huge search space to focus on promising areas, this group search method can gain better efficiency. Experiments are conducted on 35 widely used benchmark functions and the proposed SRTP is compared with state-of-the-art LSGO algorithms. Experimental results show that SRTP is effective and efficient.

An approach for initializing the random adaptive grouping algorithm for solving large-scale global optimization problems

Many real-world optimization problems deal with high dimensionality and are known as large-scale global optimization (LSGO) problems. LSGO problems, which have many optima and are not separable, can be very challenging for many heuristic search algorithms. In this study, we have proposed a novel two-stage hybrid heuristic algorithm, which incorporates the coordinate descent algorithm with the golden-section search (CDGSS) and the random adaptive grouping for cooperative coevolution of the Self-adaptive Differential Evolution with Neighborhood Search (DECC-RAG) algorithm. At the first stage, the proposed algorithm roughly scans the search space for a better initial population for the DECC-RAG algorithm. At the second stage, the algorithm uses the DECC-RAG framework for solving the given LSGO problem. We have evaluated the proposed approach (DECC-RAG1.1) with 15 most difficult LSGO problems from the IEEE CEC’2013 benchmark set. The experimental results show that DECC-RAG1.1 outperforms the standard DECC-RAG and some the state-of-the-art LSGO algorithms.

Hybrid Genetic Grey Wolf Algorithm for Large-Scale Global Optimization

Most real-world optimization problems tackle a large number of decision variables, known as Large-Scale Global Optimization (LSGO) problems. In general, the metaheuristic algorithms for solving such problems often suffer from the “curse of dimensionality.” In order to improve the disadvantage of Grey Wolf Optimizer when solving the LSGO problems, three genetic operators are embedded into the standard GWO and a Hybrid Genetic Grey Wolf Algorithm (HGGWA) is proposed. Firstly, the whole population using Opposition-Based Learning strategy is initialized. Secondly, the selection operation is performed by combining elite reservation strategy. Then, the whole population is divided into several subpopulations for cross-operation based on dimensionality reduction and population partition in order to increase the diversity of the population. Finally, the elite individuals in the population are mutated to prevent the algorithm from falling into local optimum. The performance of HGGWA is verified by ten benchmark functions, and the optimization results are compared with WOA, SSA, and ALO. On CEC’2008 LSGO problems, the performance of HGGWA is compared against several state-of-the-art algorithms, CCPSO2, DEwSAcc, MLCC, and EPUS-PSO. Simulation results show that the HGGWA has been greatly improved in convergence accuracy, which proves the effectiveness of HGGWA in solving LSGO problems.

Distributed Contribution-Based Quantum-Behaved Particle Swarm Optimization With Controlled Diversity for Large-Scale Global Optimization Problems

Many cooperative coevolution optimization algorithms have been proposed recently for solving large-scale global optimization problems. These algorithms first decompose a large-scale global optimization problem into several subproblems, each with a specific number of decision variables, and then optimize the subproblems separately. However, if computing resources are not reasonably allocated to subproblems, computational resources may be wasted. In this paper, we propose a distributed contribution-based quantum-behaved particle swarm optimization with controlled diversity (DC-QPSO) for large-scale global optimization problems. According to the level of optimized contribution of each subproblem, the computing resources are reallocated automatically in each stage, guaranteeing that subgroups with more contribution get more computational resources. Moreover, a parallel diversity control strategy is proposed to enhance the capability of finding better solutions to problems. CEC'2010 and CEC'2013 benchmark function suits are selected to test the performance of the proposed algorithms. The experimental results demonstrate the better performance of the proposed DC-QPSO on the benchmark functions, compared to other state-of-the-art large-scale global optimization algorithms.

A review of the recent use of Differential Evolution for Large-Scale Global Optimization: An analysis of selected algorithms on the CEC 2013 LSGO benchmark suite

Abstract This paper gives a review of recent extensions of the Differential Evolution (DE) algorithm for use in Large-Scale Global Optimization (LSGO) and presents an empirical analysis of DE-based and some other state-of-the-art algorithms for LSGO on the CEC 2013 LSGO benchmark suite. Since witnessing the first successful applications of DE for a wide variety of optimization problems in the early nineties, researchers have developed several new algorithms in this field. In this paper, we are especially interested in algorithms for solving LSGO. As LSGO is one of the most active research lines, not only in DE, but in many evolutionary and meta-heuristic algorithms, we discuss general approaches for dealing with LSGO first. The main focus of the paper is DE. We review its basic algorithm and discuss several extensions used for coping with large-scale problems. This paper has two main objectives: (1) To propose, from a theoretical point of view, the grouping of DE mechanisms for dealing with LSGO into nine groups, and (2) To evaluate sixteen recently proposed algorithms for LSGO empirically. Many benchmark suites were designed with the aim of providing a suitable evaluation platform for testing and comparing large-scale optimization algorithms. In this paper, the CEC 2013 LSGO benchmark suite was chosen for comparison, because it resembles the following features of real-world problems: Non-uniform subcomponent sizes; imbalance in the contribution of subcomponents; and functions with interdependent overlapping subcomponents. The performances of state-of-the-art algorithms are compared, and the algorithms are ranked using three different metrics, which evaluate the performance from different perspectives. The conducted research shows that DE is among the best algorithms for LSGO on the CEC 2013 LSGO benchmark suite, especially when used with other mechanisms for dealing with large numbers of variables. Finally, the analysis has shown that there is still some room for further improvements in DE towards the solution of LSGO problems.

A latent space-based estimation of distribution algorithm for large-scale global optimization

Large-scale global optimization problems (LSGOs) have received considerable attention in the field of meta-heuristic algorithms. Estimation of distribution algorithms (EDAs) are a major branch of meta-heuristic algorithms. However, how to effectively build the probabilistic model for EDA in high dimensions is confronted with obstacle, making them less attractive due to the large computational requirements. To overcome the shortcomings of EDAs, this paper proposed a latent space-based EDA (LS-EDA), which transforms the multivariate probabilistic model of Gaussian-based EDA into its principal component latent subspace with lower dimensionality. LS-EDA can efficiently reduce the complexity of EDA while maintaining its probability model without losing key information to scale up its performance for LSGOs. When the original dimensions are projected to the latent subspace, those dimensions with larger projected value make more contribution to the optimization process. LS-EDA can also help recognize and understand the problem structure, especially for black-box optimization problems. Due to dimensionality reduction, its computational budget and population size can be effectively reduced while its performance is highly competitive in comparison with the state-of-the-art meta-heuristic algorithms for LSGOs. In order to understand the strengths and weaknesses of LS-EDA, we have carried out extensive computational studies. Our results revealed LS-EDA outperforms the others on the benchmark functions with overlap and nonseparate variables.

Cooperative Coevolution with Formula-Based Variable Grouping for Large-Scale Global Optimization.

For a large-scale global optimization (LSGO) problem, divide-and-conquer is usually considered an effective strategy to decompose the problem into smaller subproblems, each of which can then be solved individually. Among these decomposition methods, variable grouping is shown to be promising in recent years. Existing variable grouping methods usually assume the problem to be black-box (i.e., assuming that an analytical model of the objective function is unknown), and they attempt to learn appropriate variable grouping that would allow for a better decomposition of the problem. In such cases, these variable grouping methods do not make a direct use of the formula of the objective function. However, it can be argued that many real-world problems are white-box problems, that is, the formulas of objective functions are often known a priori. These formulas of the objective functions provide rich information which can then be used to design an effective variable group method. In this article, a formula-based grouping strategy (FBG) for white-box problems is first proposed. It groups variables directly via the formula of an objective function which usually consists of a finite number of operations (i.e., four arithmetic operations "", "", "", "" and composite operations of basic elementary functions). In FBG, the operations are classified into two classes: one resulting in nonseparable variables, and the other resulting in separable variables. In FBG, variables can be automatically grouped into a suitable number of non-interacting subcomponents, with variables in each subcomponent being interdependent. FBG can easily be applied to any white-box problem and can be integrated into a cooperative coevolution framework. Based on FBG, a novel cooperative coevolution algorithm with formula-based variable grouping (so-called CCF) is proposed in this article for decomposing a large-scale white-box problem into several smaller subproblems and optimizing them respectively. To further enhance the efficiency of CCF, a new local search scheme is designed to improve the solution quality. To verify the efficiency of CCF, experiments are conducted on the standard LSGO benchmark suites of CEC'2008, CEC'2010, CEC'2013, and a real-world problem. Our results suggest that the performance of CCF is very competitive when compared with those of the state-of-the-art LSGO algorithms.

Decentralizing and coevolving differential evolution for large-scale global optimization problems

This paper presents a novel decentralizing and coevolving differential evolution (DCDE) algorithm to address the issue of scaling up differential evolution (DE) algorithms to solve large-scale global optimization (LSGO) problems. As most evolutionary algorithms (EAs) display their weaknesses on LSGO problems due to the exponentially increasing complexity, the cooperative coevolution (CC) framework is often used to overcome such weaknesses. However, the cooperative but greedy coevolution of CC sometimes gives inferior performance, especially on non-separable and multimodal problems. In the proposed DCDE algorithm, to balance the search behavior between exploitation and exploration, the original population is decomposed into several subpopulations in ring connection, and the multi-context vectors according to this connection are introduced into the coevolution. Moreover, a novel “DE/current-to-SP-best-ring/1” mutation operation is also adopted in the DCDE. On a comprehensive set of 1000- dimensional benchmarks, the performance of DCDE compared favorably against several state-of-the-art LSGO algorithms. The experimental analysis results suggest that DCDE is a highly competitive optimization algorithm on LSGO problems, especially on some non-separable and multimodal problems.

Joint operations algorithm for large-scale global optimization

Large-scale global optimization (LSGO) is a very important but thorny task in optimization domain, which widely exists in management and engineering problems. In order to strengthen the effectiveness of meta-heuristic algorithms when handling LSGO problems, we propose a novel meta-heuristic algorithm, which is inspired by the joint operations strategy of multiple military units and called joint operations algorithm (JOA). The overall framework of the proposed algorithm involves three main operations: offensive, defensive and regroup operations. In JOA, offensive operations and defensive operations are used to balance the exploration ability and exploitation ability, and regroup operations is applied to alleviate the problem of premature convergence. To evaluate the performance of the proposed algorithm, we compare JOA with six excellent meta-heuristic algorithms on twenty LSGO benchmark functions of IEEE CEC 2010 special session and four real-life problems. The experimental results show that JOA performs steadily, and it has the best overall performance among the seven compared algorithms.

Two-stage based ensemble optimization framework for large-scale global optimization

Large-scale global optimization (LSGO) is a very important and challenging task in optimization domain, which is embedded in many scientific and engineering applications. In order to strengthen both effectiveness and efficiency of LSGO algorithm, this paper designs a two-stage based ensemble optimization evolutionary algorithm (EOEA) framework, which serially implements two sub-optimizers. These two sub-optimizers mainly focus on exploration and exploitation separately. The EOEA framework can be easily generated, flexibly altered and modified, according to different implementation conditions. In order to analyze the effects of EOEA’s components, we compare its performance on diverse kinds of problems with its two sub-optimizers and three variants. To show its superiorities over the previous LSGO algorithms, we compare its performance with six classical LSGO algorithms on the LSGO test functions of IEEE Congress of Evolutionary Computation (CEC 2008). The performance of EOEA is further evaluated by experimental comparison with four state-of-the-art LSGO algorithms on the test functions of CEC 2010 LSGO competition. To benchmark the practical applicability of EOEA, we adopt EOEA to the parameter calibration problem of water pipeline system. Based on the experimental results on diverse scales of systems, EOEA performs steadily and robustly.