Large-scale Global Optimization Problems Research Articles

An Efficient Differential Grouping Algorithm for Large-Scale Global Optimization

Cooperative co-evolution (CC) is a practical and efficient evolutionary framework for solving large-scale global optimization problems (LSGOPs). The performance of CC depends on how variables are being grouped and can be improved through guided variable decomposition for various optimization problems. However, achieving a proper variable decomposition is computationally expensive. This paper proposes an effective yet efficient differential grouping (EDG) method to reduce the associated computational cost. Our method exploits the historical interrelationship information of previous variable groups to examine interactions between the remnant variable groups. This allows us to spend less computing resources without compromising the accuracy of the final grouping result. Our proposal utilizes the covariance matrix adaptation evolution strategy (CMA-ES) algorithm, in conjunction with EDG, to solve LSGOPs. Further, to reduce time complexity and improve the stability of CMA-ES, we substitute the complex matrix decomposition step with simpler matrix operations to compute the square root of the covariance matrix. Results from our experiments and analysis indicate that EDG is a competitive method to solve LSGOPs and improve the performance of CC. The proposed schemes significantly enhance the searchability of CMA-ES compared to the other large-scale variants of CMA-ES and state-of-the-art large-scale optimizers. Moreover, our EDG could be integrated with evolutionary optimizers of different flavors like Differential Evolution (DE).

Large-scale evolutionary optimization: A review and comparative study

Large-scale global optimization (LSGO) problems have widely appeared in various real-world applications. However, their inherent complexity, coupled with the curse of dimensionality, makes them challenging to solve. Continuous efforts have been devoted to designing computational intelligence-based approaches to solve them. This paper offers a comprehensive review of the latest developments in the field, focusing on the advances in both single-objective and multi-objective large-scale evolutionary optimization algorithms over the past five years. We systematically categorize these algorithms, discuss their distinct features, and highlight benchmark test suites essential for performance evaluation. After that, comparative studies are conducted using numerical solutions to evaluate the performance of state-of-the-art LSGO for both single-objective and multi-objective problems. Finally, we discuss the real-world applications of LSGO, some challenges, and possible future research directions.

Using cooperative coevolution in large-scale black-box constraint satisfaction problems

Solving constrained large-scale global optimization problems poses a challenging task. In these problems with constraints, when the number of variables is measured in the thousands, when the constraints are presented in the form of a black box, and neither the size nor the configuration of the feasible region is known, it is very difficult to find at least one feasible solution. In general, such a problem of finding a feasible region is known as a constraint satisfaction problem. In this paper, we have extended a well-known benchmark set based on constrained optimization problems up to 1000 variables. We have evaluated the CC-SHADE performance, to tackle constraints in large-scale search space. CC-SHADE merges the power of cooperative coevolution and self-adaptive differential evolution. Our extensive experimental evaluations on a range of benchmark problems demonstrate the strong dependence of the performance of CC-SHADE on the number of individuals and the subcomponent number. The numerical results emphasize the importance of using a cooperative coevolution framework for evolutionary-based approaches compared to conventional methods. All numerical experiments are proven by the Wilcoxon test.

A survey of meta-heuristic algorithms in optimization of space scale expansion

Optimization problem of space scale expansion widely exists in practical applications, such as transportation, logistics, scheduling, social networks, etc. According to different expansion directions, the problem of space scale expansion can be divided into three categories: expansion of decision space, expansion of objective space and simultaneous expansion of decision space and objective space. These three types of problems correspond to large-scale global optimization problem, multi/many-objective optimization problem, and large-scale multi/many-objective optimization problem, respectively. Driven by the above problems, meta-heuristic algorithms (MHAs) with scalable characteristics have received extensive attention in this field. This paper summarizes the research progress of MHAs for space scale expansion optimization from three perspectives. Starting from the key difficulties of the optimization problem, the challenges brought by the expansion of the space scale to the existing MHAs are emphatically analyzed. From the perspective of methodology, the optimization methods of MHAs are divided into three modules: simplify problem structure, improve algorithm performance and expand application fields. Based on the review of performance evaluation benchmark problems, the simulation degree of various test suites for different practical application problems is summarized. In addition, some remaining challenges and future research directions on optimization of space scale expansion are discussed and analyzed.

A Decomposition Method for Both Additively and Nonadditively Separable Problems

Problem decomposition is crucial for coping with large-scale global optimization problems, which relies heavily on highly precise variable grouping methods. The state-of-the-art decomposition methods identify separability based on the finite differences principle, which is valid only for additively separable functions but not applicable to non-additively separable functions. Therefore, we need to investigate separability in more depth in order to propose a more general principle and design more universal decomposition methods. In this paper, we conduct a comprehensive theoretical investigation on separability, the core of which is proposing an innovative separability identification principle: the minimum points shift principle. By utilizing the new principle, we develop a general separability grouping (GSG) method that can handle both additively and non-additively separable functions with high accuracy. In addition, we design a new set of benchmark functions based on non-additive separability, which compensates for the lack of non-additively separable functions in the previous test suites. Extensive experiments demonstrate that the proposed GSG achieves high grouping accuracy on both new and CEC series benchmark problems, especially on non-additively separable problems Finally, we verify that the proposed GSG can effectively improve the optimization performance of non-additively separable problems through optimization experiments.

Low-Dimensional Space Modeling-Based Differential Evolution for Large-Scale Global Optimization Problems

Large-Scale Global Optimization (LSGO) has been an active research field. Part of this interest is supported by its application to cutting-edge research such as Deep Learning, Big Data, and complex real-world problems such as image encryption, real-time traffic management, and more. However, the high dimensionality makes solving LSGO a significant challenge. Some recent research deal with the high dimensionality by mapping the optimization process to a reduced alternative space. Nonetheless, these works suffer from the changes in the search space topology and the loss of information caused by the dimensionality reduction. This paper proposes a hybrid metaheuristic, so-called LSMDE (Low-dimensional Space Modeling-based Differential Evolution), that uses the Singular Value Decomposition to build a low-dimensional search space from the features of candidate solutions generated by a new SHADE-based algorithm (GM-SHADE). GM-SHADE combines a Gaussian Mixture Model (GMM) and two specialized local algorithms: MTS-LS1 and L-BFGS-B, to promote a better exploration of the reduced search space. GMM mitigates the loss of information in mapping high-dimensional individuals to low-dimensional individuals. Furthermore, the proposal does not require prior knowledge of the search space topology, which makes it more flexible and adaptable to different LSGO problems. The results indicate that LSMDE is the most efficient method to deal with partially separable functions compared to other state-of-the-art algorithms and has the best overall performance in two of the three proposed experiments. Experimental results also show that the new approach achieves competitive results for non-separable and overlapping functions on the most recent test suite for LSGO problems.

Enhanced opposition-based grey wolf optimizer for global optimization and engineering design problems

A recently developed swarm-based meta-heuristic algorithm namely Grey Wolf Optimization algorithm (GWO), which is based on the hunting and leadership behaviours of the grey wolves in nature, has shown superior performance when compared with existing meta-heuristic algorithms. However, like other approaches, the GWO has the limitation of poor exploitation ability and being stuck in local optima when solving challenging optimization problems. To overcome these limitations, a novel technique, namely “Enhanced Opposition-Based Learning” (EOBL), has been proposed and is implemented with the GWO algorithm. The EOBL technique is largely inspired by Opposition-Based Learning (OBL) and Random Opposition-Based Learning (ROBL) techniques to efficiently balance exploration and exploitation. As a result, the Enhanced Opposition-Based Grey Wolf Optimizer (EOBGWO), an innovative approach, is proposed to increase the effectiveness of the conventional GWO algorithm. To test the efficiency of the proposed EOBGWO method, it has been tested on the standard IEEECEC2005, IEEECEC2017, and IEEECEC2019 test functions, along with several real-life engineering design problems. Furthermore, to evaluate the effectiveness and stability of the proposed technique, it has been evaluated on the challenging IEEECEC2008 special session on large scale global optimization problems. The experimental outcomes and statistical measures such as the t-test and Wilcoxon rank-sum test demonstrate that the proposed EOBGWO method outperforms the other state-of-the-art algorithms in both global optimization and engineering design problems.

A cultural evolution with a modified selection function and adaptive [formula omitted]-cognition procedure for numerical optimization

In recent years, several population-based evolutionary and swarm algorithms have been developed and used in the literature. This work introduces an improved Cultural Algorithm with a modified selection function and a dynamic α-cognition procedure to handle a variety of challenging numerical optimization problems. The modified selection function is used to support a balanced evolutionary search. A process that starts with a clearer exploration early in the search process and gradually begins to focus on exploitation towards the end of the search process. This work uses the elites of each knowledge source that are at a certain distance from each other. The dynamic α-cognition procedure assists in providing effective learning of individuals through preserving the diversity of the population during the evolution process. In this procedure, each individual is able to learn from the top α% individuals controlled by its knowledge source in the belief space, where the proportion of the affecting subpopulation (α) is adaptively modified during the evolution. The performance of the proposed work has been evaluated on the CEC’2010 and CEC’2013 benchmark suites developed for the special sessions on large-scale global optimization problems. An appropriate comparative study with the best results in the literature is presented. The results confirm how the merits of the improved Cultural Algorithm can achieve superior performance over other cutting-edge algorithms for these data sets.

Hybrid Evolutionary Algorithm for Solving the Large-Scale Global Optimization Problems

Hybrid Evolutionary Algorithm for Solving the Large-Scale Global Optimization Problems

A large-scale continuous optimization benchmark suite with versatile coupled heterogeneous modules

The complexity of the engineering design optimizations mainly sources from the large-scale nature of the problems, and the ever-increasing large-scale global optimization (LSGO) problems place high demands on the performance of evolutionary algorithms (EAs). To compare and analyze large-scale optimization algorithms, benchmarks that simulate the features from real-world large-scale optimization are desired. In this paper, we propose a novel large-scale continuous optimization benchmark suite, which includes 15 test functions with the modular structure. The proposed benchmark suite introduces two new features abstracted from the engineering design problems: (1) Heterogeneous design, i.e., the modules are significantly different in terms of difficulty and explicit expressions. (2) Versatile coupling, i.e., different coupling degrees and random coupling topologies of the benchmark are emphasized. Two typical cooperative coevolution frameworks and several state-of-the-art large-scale optimization algorithms are used for comparative studies on the proposed benchmark, and experimental results demonstrate that the proposed benchmark suite is very challenging.

An innovative time-varying particle swarm-based Salp algorithm for intrusion detection system and large-scale global optimization problems

An innovative time-varying particle swarm-based Salp algorithm for intrusion detection system and large-scale global optimization problems

Model parameter identification for lithium-ion batteries using adaptive multi-context cooperatively co-evolutionary parallel differential evolution algorithm

Lithium-ion battery (LIB) has a polarization phenomenon during charging and discharging processes, and the internal chemistry of LIB is characterized by severe time-varying nonlinearity. Therefore, the equivalent circuit model (ECM) is usually used for LIB research. To address the problems of low identification accuracy and local optimization in the offline identification of battery parameters, this paper proposes a novel adaptive multi-context cooperatively co-evolutionary parallel differential evolution (AMCC-PDE) algorithm to identify parameters of LIB. Firstly, the data segment to be identified is divided into a plurality of segments according to the state of charge (SOC), and each segment is called a unit data segment (UDS). In addition, the UDS has a parameter group (PG) for the first-order RC model. Secondly, according to the differential equation of the first-order RC model, the PG (R0+,R0−, D1, R1, U1, init) of each UDS and the open-circuit voltage (UOCV) at each sampling point are considered as variables to be optimized. Then, such an optimization problem is transformed into a large-scale global optimization (LSGO) problem. In addition, to trade off the relationship between population diversity and convergence speed, a novel parallel mutation strategy is proposed. Finally, an AMCC-PDE algorithm is proposed to solve the above LSGO parameters identification model. In both of the DST and FUDS datasets, the identification error at each point obtained by AMCC-PDE is lower than 2 mV, which shows the effectiveness of AMCC-PDE and indicates that the algorithm can avoid destroying the variable-coupling relationship.

A new metaheuristic algorithm: Shrimp and Goby association search algorithm and its application for damage identification in large-scale and complex structures

This paper presents the application of a new Shrimp and Goby Association Search algorithm (SGA) to solve large-scale global optimization problems. The performance of SGA is assessed using 13 benchmark high-dimensional functions, 10 classical benchmark functions, and several real-world engineering applications. For the first time, an efficient optimization approach for structural health monitoring (SHM) in truss-like structures is presented. The proposed approach is applied for damage identification of complex structures. A real structure, namely Canton Tower in Guangzhou, China, is served as an example for damage detection. Interestingly, this tower was the tallest structure in the world in 2009 with a height of 610 m. The great merit of this example is that it provides a real complex structure with a high-dimensional problem to assess the performance of SGA in the real world. The results show that SGA can deal with this problem effectively, at the same time, it operates better to escape from local optima with faster convergence rate than population-based algorithms.

Merged Differential Grouping for Large-Scale Global Optimization

The divide-and-conquer strategy has been widely used in cooperative co-evolutionary algorithms to deal with large-scale global optimization problems, where a target problem is decomposed into a set of lower-dimensional and tractable subproblems to reduce the problem complexity. However, such a strategy usually demands a large number of function evaluations to obtain an accurate variable grouping. To address this issue, a merged differential grouping (MDG) method is proposed in this article based on the subset–subset interaction and binary search. In the proposed method, each variable is first identified as either a separable variable or a nonseparable variable. Afterward, all separable variables are put into the same subset, and the nonseparable variables are divided into multiple subsets using a binary-tree-based iterative merging method. With the proposed algorithm, the computational complexity of interaction detection is reduced to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$O(\max \{n,n_{ns}\times \log _{2} k\})$ </tex-math></inline-formula> , where <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$n$ </tex-math></inline-formula> , <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$n_{ns}(\leq n)$ </tex-math></inline-formula> , and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k( &lt; n)$ </tex-math></inline-formula> indicate the numbers of decision variables, nonseparable variables, and subsets of nonseparable variables, respectively. The experimental results on benchmark problems show that MDG is very competitive with the other state-of-the-art methods in terms of efficiency and accuracy of problem decomposition.

A surrogate-assisted variable grouping algorithm for general large-scale global optimization problems

Problem decomposition plays an important role when applying cooperative coevolution (CC) to large-scale global optimization problems. However, most learning-based decomposition algorithms only apply to additively separable problems, while the others insensitive to problem type perform low decomposition accuracy and efficiency. Given this limitation, this study designs a general-separability-oriented detection criterion, and further proposes a novel decomposition algorithm called surrogate-assisted variable grouping (SVG). The new criterion detects the separability between a variable and some other variables by checking whether its optimum changes with the latter. Consistent with the definition of general separability, this criterion endows SVG with strong applicability and high accuracy. To reduce expensive fitness evaluations, SVG locates the optimum of a variable with the help of a surrogate model rather than the original high-dimensional model. Moreover, it converts the variable-grouping process into a search process in a binary tree by taking variable subsets as tree nodes. This facilitates the reutilization of historical separability information, thereby reducing separability detection times. Experimental results on a general benchmark suite indicate that compared with six state-of-the-art decomposition algorithms, SVG achieves higher accuracy and efficiency on both additively and nonadditively separable problems. Furthermore, it can significantly enhance the optimization performance of CC.

A Novel Self-Adaptive Cooperative Coevolution Algorithm for Solving Continuous Large-Scale Global Optimization Problems

Unconstrained continuous large-scale global optimization (LSGO) is still a challenging task for a wide range of modern metaheuristic approaches. A cooperative coevolution approach is a good tool for increasing the performance of an evolutionary algorithm in solving high-dimensional optimization problems. However, the performance of cooperative coevolution approaches for LSGO depends significantly on the problem decomposition, namely, on the number of subcomponents and on how variables are grouped in these subcomponents. Also, the choice of the population size is still an open question for population-based algorithms. This paper discusses a method for selecting the number of subcomponents and the population size during the optimization process (“on fly”) from a predefined pool of parameters. The selection of the parameters is based on their performance in the previous optimization steps. The main goal of the study is the improvement of coevolutionary decomposition-based algorithms for solving LSGO problems. In this paper, we propose a novel self-adapt evolutionary algorithm for solving continuous LSGO problems. We have tested this algorithm on 15 optimization problems from the IEEE LSGO CEC’2013 benchmark suite. The proposed approach, on average, outperforms cooperative coevolution algorithms with a static number of subcomponents and a static number of individuals.

Parameter identification for lithium batteries: Model variable-coupling analysis and a novel cooperatively coevolving identification algorithm

In the operational control of renewable energy system, the efficient parameter identification for lithium battery is of great importance. In this study, the parameter identification of lithium battery is modelled as a large-scale global optimization problem with thousands of dimensionalities. In addition, the developed identification model is proved to be a partial-separable problem by comprehensively analysing its variable-coupling relationships, and the detailed proof is also provided. In order to overcome the high-dimensional characteristic of the developed model, a novel algorithm namely incomplete multi-context cooperatively coevolving PSO (IMCCPSO) is developed, in which some efficient algorithmic mechanisms are proposed: On one hand, the non-separable variables are grouped together with each of the separable-variable components, and the context vectors for separable and non-separable variables are discriminatively reconstituted for balancing the local and global exploration; On the other hand, the coevolving efficiency index is proposed for selecting the group-size values and coevolving rules dynamically and adaptively. Experimental results show that the developed methodology can effectively identify the parameters of the evaluated lithium battery bank under typical load profiles, and the developed IMCCPSO also outperforms the compared state-of-the-art algorithms on identification accuracy and robustness.

Solving large-scale global optimization problems and engineering design problems using a novel biogeography-based optimization with Lévy and Brownian movements

Solving large-scale global optimization problems and engineering design problems using a novel biogeography-based optimization with Lévy and Brownian movements

A comprehensive investigation on novel center-based sampling for large-scale global optimization

During the last decade, metaheuristic algorithms have been well-established approaches which are utilized for solving complex real-world optimization problems. The most metaheuristic algorithms uses stochastic strategies in their initialization phase as well as during their new candidate generation steps when there is no a-priori knowledge about the solution, which is a common valid assumption for any black-box optimization algorithm. In recent years, researchers have introduced a new concept called center-based sampling which can be utilized in any search component of the optimization process, but so far it mainly has been used just for population initialization. This novel concept clarifies that in a search space, the center point has a higher chance to be closer to an unknown solution compared to a random point, especially when the dimension of the search space increases. Thus, this concept helps the optimizer to find a better solution in a shorter time. In this paper, a comprehensive study has been conducted on the effect of center-based sampling to solve an optimization problem using three different levels of detailed investigation. These levels are as the follows: 1) considering no specific algorithm and no specific landscape (i.e., Monte-Carlo simulation); 2) considering a specific landscape but no specific algorithm (i.e., random search vs. center-based random search), and finally, 3) considering a specific algorithm and specific landscape which includes the proposing three different schemes for using center-based sampling scheme for solving Large-scale Global Optimization (LSGO) problems effectively. Furthermore, in this study, we seek to investigate the properties and capabilities of center-based sampling during optimization, which can be extended to utilize it in machine learning too, because optimization is a key role player in search and learning models. The proposed methods in this paper are evaluated on CEC 2013 LSGO benchmark functions and a real-world optimization problem, i.e., evolving ANN on two medical data sets. The experimental results confirm that center-based sampling has a crucial impact in improving the convergence rate of optimization/search algorithms when solving high-dimensional optimization problems.

An Advanced Amalgam of Nature-Inspired Algorithms for Global Optimization Problems

Large-scale global optimization problems are ambitious and quite difficult to handle with deterministic methods. The use of stochastic optimization techniques is a good choice for dealing with these problems. Nature-inspired algorithms (NIAs) are stochastic in nature, computer-based, and quite easy to implement due to their population-based nature. The Grey wolf optimizer (GWO) and teaching-learning-based optimization are the most recently developed and well-known NIAs. GWO is based on the preying strategies of grey wolves while TLBO is based on the effect of the influence of a teacher on the output of learners in a class. NIAs are quite often stuck in the local basins of attraction due to the improper balancing of exploration versus exploitation. In this paper, an advanced amalgam of nature-inspired algorithms (ANIA) is developed by employing GWO and TLBO as constituent algorithms. Initially, an equal number of solutions are assigned to both NIAs to perform their search process of population evolution; then, in later iterations, the number of solutions are allocated to each constituent algorithm based on their individual performance and achievements gained by each algorithm in the previous iteration. The performance of an algorithm is determined at the end of iteration by calculating the ratio of total updated solutions to the total assigned solutions in the amalgam. The proposed strategy effectively balanced the exploration versus exploitation dilemma via compelling the parent algorithms to show continuous improvement during the whole course of the optimization process. The performance of the proposed algorithm, ANIA is evaluated on recently designed benchmark functions of large-scale global optimization problems. The approximated results found by the proposed algorithm are promising as compared to state-of-the-art evolutionary algorithms including the GWO and TLBO in terms of diversity and proximity. The proposed ANIA has tackled most of the benchmark functions efficiently in the parlance of evolutionary computing communities.