Differential Evolution Variants Research Articles

Differential evolution with collective ensemble learning

Ensemble learning of multiple search strategies has demonstrated effectiveness in improving the performance of differential evolution (DE) for global optimization. In existing ensemble learning methods, offspring for each target solution is generated by one of the strategies from the strategy pool according to the optimization requirement of the considered problem. However, since the genetic material of an offspring generally comes from one single strategy, the advantages of different strategies could not be simultaneously utilized for optimizing decision variables with different properties. To deal with this problem, this paper proposes the collective ensemble learning (CEL) paradigm, which merges the advantages of multiple strategies for generating an offspring. CEL mainly includes the component decomposition (CD) and component integration (CI) mechanisms. The CD mechanism divides the target solutions and the candidates generated by different generation strategies into exploitative and explorative components respectively, in consideration of the evolution status at the dimension level. Then, the CI mechanism is implemented to integrate the appropriate components to form the offspring according to the sub-similarity measurement performed at the component level. The effectiveness and advantages of CEL have been validated and discussed by performance comparisons with the baseline, each single mechanism, existing ensemble learning methods, as well as several state-of-the-art DE variants.

Global Opposition Learning and Diversity ENhancement based Differential Evolution with exponential crossover for numerical optimization

Since the inception of Differential Evolution (DE), the vast majority of studies on it indicate that exponential crossover does not solve the real-parameter optimization in continuous spaces very well. However, we find that once the appropriate crossover rate CR and its corresponding parameter control are found, the DE variants with exponential crossover can achieve comparable performance with the ones employing binomial crossover. In this paper, a new DE algorithm, called Global Opposition Learning and Diversity ENhancement based DE with exponential crossover (GOLDEN-DE), is proposed to fill the gap in this field. The GOLDEN-DE algorithm has the following highlights: First, a novel global opposition learning mechanism is proposed in our GOLDEN-DE algorithm; Second, a Novel Parameter Control, namely NPC technique, with fitness-independent characteristic is incorporated in the GOLDEN-DE algorithm. Third, a novel population Diversity ENhancement (DEN) mechanism is proposed, and individuals in the stagnation status can be re-calculated in order to enhance population diversity. Fourth, a time stamp mechanism for handling too old inferior solutions in the external archive is also put forward. We evaluate the GOLDEN-DE algorithm for real-parameter single-objective global optimization on 88 benchmarks from the three universal test sets, including CEC2013, CEC2014, and CEC2017 test suites, and experiment results show that our GOLDEN-DE algorithm is competitive with several state-of-the-art DE variants.

Differential evolution algorithm with a complementary mutation strategy and data Fusion-Based parameter adaptation

As an excellent optimization algorithm widely used to solve various practical problems, differential evolution (DE) algorithm has few parameters, yet its performance is significantly affected by these parameters. To address this issue, this paper presents a novel variant of DE called DACDE, which utilizes data fusion-based parameter adaptation and a complementary mutation strategy. Most parameter adaptation methods based on successful history only analyze the mean of parameters and ignore their degree of dispersion. In DACDE, the successful parameter distribution is recorded and described by both the mean and variance. Data fusion is then used to combine records and generate an estimated distribution, which is applied to a Gaussian distribution to generate new parameters. Inspired by opposition-based learning, we introduce a complementary mutation strategy. This strategy employs a symmetric selection mechanism to adapt to the varying search abilities required by the algorithm at different stages. The new variant is verified on 32 single-objective functions from CEC 2011 and 2014 benchmark suites, and the results show that DACDE is competitive compared to other 29 evolutionary algorithms.

Soft computing methods in the solution of an inverse heat transfer problem with phase change: A comparative study

Inverse heat transfer problems are ill-posed problems and their solution is challenging. Conventional (hard computing) solution methods were developed for this purpose in the past, but they are not well applicable in cases including phase change, which contain strong non-linearity and bring additional computational difficulties. Soft computing methods, which currently experience very rapid development, are a promising tool for the solution of such problems. This paper addresses an inverse heat transfer problem with phase change, in which the boundary heat flux is estimated. Four methods based on distinct mathematical principles are applied to this problem and thoroughly compared. These methods include a conventional Levenberg–Marquardt method (LMM), a predictive fuzzy logic (PFL)-based method, a population-based meta-heuristic method called LSHADE (a state-of-the-art differential evolution variant), and a recently developed surrogate-assisted method coupled with differential evolution, referred to as LSADE method. Furthermore, a reformulation of the problem was developed, utilising a dimension reduction scheme and a decomposition scheme that led to sub-problems with different time frames. This reformulation brought extensive computational improvements. Results of the comparison of the methods then showed that the LMM and the PFL behave well in case without phase change but their performance deteriorates substantially in case with phase change. The LSHADE and the LSADE showed superior performance in the solution of the inverse problem with the phase change. Moreover, their performance was rather stable and insensitive to the location of the temperature sensor, which was the source of data for the estimation.

The Importance of Being Constrained: Dealing with Infeasible Solutions in Differential Evolution and Beyond.

We argue that results produced by a heuristic optimisation algorithm cannot be considered reproducible unless the algorithm fully specifies what should be done with solutions generated outside the domain, even in the case of simple bound constraints. Currently, in the field of heuristic optimisation, such specification is rarely mentioned or investigated due to the assumed triviality or insignificance of this question. Here, we demonstrate that, at least in algorithms based on Differential Evolution, this choice induces notably different behaviours in terms of performance, disruptiveness, and population diversity. This is shown theoretically (where possible) for standard Differential Evolution in the absence of selection pressure and experimentally for the standard and state-of-the-art Differential Evolution variants, on a special test function and the BBOB benchmarking suite, respectively. Moreover, we demonstrate that the importance of this choice quickly grows with problem dimensionality. Differential Evolution is not at all special in this regard-there is no reason to presume that other heuristic optimisers are not equally affected by the aforementioned algorithmic choice. Thus, we urge the heuristic optimisation community to formalise and adopt the idea of a new algorithmic component in heuristic optimisers, which we refer to as the strategy of dealing with infeasible solutions. This component needs to be consistently: (a) specified in algorithmic descriptions to guarantee reproducibility of results, (b) studied to better understand its impact on an algorithm's performance in a wider sense (i.e., convergence time, robustness, etc.), and (c) included in the (automatic) design of algorithms. All of these should be done even for problems with bound constraints.

Improved Differential Evolution Algorithm Guided by Best and Worst Positions Exploration Dynamics.

The exploration of premium and new locations is regarded as a fundamental function of every evolutionary algorithm. This is achieved using the crossover and mutation stages of the differential evolution (DE) method. A best-and-worst position-guided novel exploration approach for the DE algorithm is provided in this study. The proposed version, known as "Improved DE with Best and Worst positions (IDEBW)", offers a more advantageous alternative for exploring new locations, either proceeding directly towards the best location or evacuating the worst location. The performance of the proposed IDEBW is investigated and compared with other DE variants and meta-heuristics algorithms based on 42 benchmark functions, including 13 classical and 29 non-traditional IEEE CEC-2017 test functions and 3 real-life applications of the IEEE CEC-2011 test suite. The results prove that the proposed approach successfully completes its task and makes the DE algorithm more efficient.

Emulation-based adaptive differential evolution: fast and auto-tunable approach for moderately expensive optimization problems

AbstractIn the field of expensive optimization, numerous papers have proposed surrogate-assisted evolutionary algorithms (SAEAs) for a few thousand or even hundreds of function evaluations. However, in reality, low-cost simulations suffice for a lot of real-world problems, in which the number of function evaluations is moderately restricted, e.g., to several thousands. In such moderately restricted scenario, SAEAs become unnecessarily time-consuming and tend to struggle with premature convergence. In addition, tuning the SAEA parameters becomes impractical under the restricted budgets of function evaluations—in some cases, inadequate configuration may degrade performance instead. In this context, this paper presents a fast and auto-tunable evolutionary algorithm for solving moderately restricted expensive optimization problems. The presented algorithm is a variant of adaptive differential evolution (DE) algorithms, and is called emulation-based adaptive DE or EBADE. The primary aim of EBADE is to emulate the principle of sample-efficient optimization, such as that in SAEAs, by adaptively tuning the DE parameter configurations. Specifically, similar to Expected Improvement-based sampling, EBADE identifies parameter configurations that may produce expected-to-improve solutions, without using function evaluations. Further, EBADE incepts a multi-population mechanism and assigns a parameter configuration to each subpopulation to estimate the effectiveness of parameter configurations with multiple samples carefully. This subpopulation-based adaptation can help improve the selection accuracy of promising parameter configurations, even when using an expected-to-improve indicator with high uncertainty, by validating with respect to multiple samples. The experimental results demonstrate that EBADE outperforms modern adaptive DEs and is highly competitive compared to SAEAs with a much shorter runtime.

Adaptation of the Scaling Factor Based on the Success Rate in Differential Evolution

Differential evolution is a popular heuristic black-box numerical optimization algorithm which is often used due to its simplicity and efficiency. Parameter adaptation is one of the main directions of study regarding the differential evolution algorithm. The main reason for this is that differential evolution is highly sensitive to the scaling factor and crossover rate parameters. In this study, a novel adaptation technique is proposed which uses the success rate to replace the popular success history-based adaptation for scaling factor tuning. In particular, the scaling factor is sampled with a Cauchy distribution, whose location parameter is set as an nth order root of the current success rate, i.e., the ratio of improved solutions to the current population size. The proposed technique is universal and can be applied to any differential evolution variant. Here it is tested with several state-of-the-art variants of differential evolution, and on two benchmark sets, CEC 2017 and CEC 2022. The performed experiments, which include modifications of algorithms developed by other authors, show that in many cases using the success rate to determine the scaling factor can be beneficial, especially with relatively small computational resource.

Collaborative resource allocation-based differential evolution for solving numerical optimization problems

Differential evolution (DE) is an efficient and powerful population-based search algorithm for solving numerical optimization problems in continuous spaces. It has been proven that multi-strategy DE algorithms are more effective than single-strategy DE algorithms in addressing benchmark and real-world problems. However, most multi-strategy DE variants focus on maintaining population diversity and balancing exploitation and exploration, ignoring the dynamic allocation of computational resources. Moreover, the success of these algorithms often depends on additional designed techniques, leading to increased computational complexity. In this paper, the Collaborative Resource Allocation-based Differential Evolution (CRADE) is introduced. It involves a collaborative resource allocation mechanism that utilizes the historical performance ranking of three mutation strategies to automatically allocate computational resources to various subpopulations during the search process. The parameter adaptation technique is used to adjust the associated control parameters of different mutation strategies. As a result, the most efficient mutation strategy consumes the majority of computational resources at different search stages to mitigate inefficient search under constrained resources. The performance of CRADE is evaluated on the well-known CEC2013 benchmark function set. The paper also investigates its application in the parameter identification of photovoltaic solar cells and modules. The overall results show that CRADE exhibits superior and competitive performance compared to other state-of-the-art algorithms. Consequently, CRADE has emerged as a novel and effective approach for addressing numerical optimization problems, distinguished by its excellence, practicality, and unwavering reliability.

Joint Optimization of Deployment and Flight Planning of Multi-UAVs for Long-Distance Data Collection From Large-Scale IoT Devices

Internet of Things (IoT) devices have been widely deployed to build smart cities. How to efficiently collect data from large-scale IoT devices is a valuable and challenging research topic. Benefiting from agility, flexibility, and deployability, an unmanned aerial vehicle (UAV) has great potential to be an aerial base station. However, given the limited battery capacity, the flight time of a UAV is limited. This paper focuses on using multi-UAVs to execute long-distance data collection from large-scale IoT devices. We design a multi-UAVs-assisted large-scale IoT data collection system. The core facilities of this system are the data center and charging stations, which are equipped with a limited number of charging piles to provide charging services for UAVs. To ensure the efficient operation of the system, the problem of deployment and flight planning of UAVs is formulated as a joint optimization problem. To solve the problem, a population-based optimization algorithm with a three-layer structure, namely EDDE-DPDE, is proposed. It includes two core components: elite-driven differential evolution (EDDE) and differential evolution with a dynamic population (DPDE), which are two variants of differential evolution. Thanks to ideas of reusing elite individuals and historical information, the proposed EDDE-DPDE shows an improvement of at least 11.11% compared with four powerful algorithms in terms of average travel time.

Multi-operator opposition-based learning with the neighborhood structure for numerical optimization problems and its applications

Opposition-based learning (OBL) is an effective strategy that adjusts the population to accelerate the convergence of the algorithm. However, OBL involves two phases (generation and calculation) that are seldom mentioned simultaneously. Besides, individual information in OBL is not fully used to guide evolution. First, a novel neighborhood generation strategy is proposed to fully use individual information. There are numerous variants of the neighborhood structure. Three prominent neighborhood structures are designed to demonstrate their benefits in various dimensions. Second, a multi-operator calculation method with two distributions is introduced to reach the equilibrium between diversification and intensification. Consequently, this paper proposes three multi-operator neighborhood-based OBL variants (MNOBLs). Compared with the traditional OBL variants, MNOBLs generate the opposite population by using a neighborhood of the individuals. A well-balanced calculation method can lead to good convergence behavior. Subsequently, the proposed MNOBLs are embedded into the basic differential evolution (DE) algorithm to obtain the proposed algorithms. The extensive and rigorous numerical experiments are designed at three levels: strategies, algorithms, and functions. Specifically, the effectiveness and efficiency of the proposed algorithms are verified by comparing them with 13 state-of-the-art DE variants and four metaheuristics on the CEC 2017, and CEC 2022 test suites, and two real-world optimization problems. The comprehensive results show that the proposed OBL variants provide promising results for most problems.

Triple competitive differential evolution for global numerical optimization

As optimization problems become more and more complex in real-world scenarios, the effectiveness of many existing differential evolution (DE) methods is critically challenged. To circumvent this predicament, this paper proposes a triple competitive DE (TCDE) to tackle increasingly complicated optimization problems. Specifically, a triple competition mechanism is devised to first randomly arrange individuals into exclusive triples and then let individuals in each triple compete with each other. Subsequently, heterogeneous mutation schemes are designed to update the three individuals in each triple based on the competition results. With the cooperation between these two techniques, on the one hand, better individuals in the triples tend to exploit the areas where they lie to find high-accuracy solutions, while worse individuals in the triples prefer to explore the intricate problem space to locate promising regions. On the other hand, different individuals are mutated by following the guidance of different leading exemplars, which ensures that individuals traverse the sophisticated solution space in diverse directions along with slight intensification to optimal regions. Furthermore, a modified adaptive parameter adjustment scheme is designed for individuals in each triple by considering their differences so as to meet the expectation that better individuals focus on exploitation while worse individuals concentrate on exploration. Finally, experiments are performed to first compare TCDE with totally 10 advanced DE variants and then further compare it with totally 10 CEC winners and their latest variants on the CEC'2014 and the CEC'2017 problem suites with totally 4 different dimensionality settings. Experimental results prove that TCDE achieves highly competitive or even much better performance than the totally 20 compared methods. Particularly, TCDE is demonstrated to be especially capable of solving complicated optimization problems. Furthermore, the efficacy of TCDE in tackling real-world optimization problems is substantiated through experiments conducted on the CEC'2011 real-world optimization problem set.

Differential Evolution with perturbation mechanism and covariance matrix based stagnation indicator for numerical optimization

Differential Evolution (DE), as a promising population-based stochastic optimization algorithm, has drawn attention from researchers of various fields owing to its simple operation, strong robustness and few control parameters. However, classic DE suffers from drawbacks such as premature convergence and stagnation resulted from reduced population diversity when tackling complicated optimization problems with high dimensions. To mitigate these deficiencies, Differential Evolution with Perturbation mechanism and Covariance Matrix based stagnation indicator (PCM-DE) is proposed in this paper. There are three main modifications in the algorithm. First, a two-phase parameter adaptation strategy based on fitness value is proposed to guide the search towards promising direction according to different stages of evolution. Second, a perturbation mechanism for archived population is proposed, where a new weight coefficient is defined by exploiting information of fitness value and position in archived individuals. Third, a stagnation indicator based on covariance matrix is proposed to assess the population diversity and the information of variance is employed to perturb stagnant individuals. To validate the effectiveness of PCM-DE, it is compared with several state-of-the-art DE variants and non-DE based algorithms under a large test suite containing 100 benchmark functions. Besides, three truss optimization problems are also employed to verify the feasibility and scalability of the algorithm. The experiment results confirm its highly competitive performance in terms of solution accuracy and convergence speed.

An adaptive differential evolution with opposition-learning based diversity enhancement

Differential Evolution (DE), as a powerful population-based stochastic optimization algorithm, has attracted the attention of researchers from various fields due to its advantages such as simple operation, strong robustness, and few control parameters. However, many existing DE variants often suffer from drawbacks such as premature convergence and stagnation when solving complicated optimization problems. In view of the aforementioned issues, this paper proposes an adaptive DE with opposition learning-based diversity enhancement (OLBADE). The main contributions can be summarized as follows: Firstly, a new adaptive parameter control is proposed with a non-linear weighting strategy incorporating into the framework of parameter adaptation. Secondly, a donor vector perturbation strategy is introduced to complement existing strategy for increasing population diversity. Thirdly, a novel stagnation indicator is proposed, and then opposition learning strategy is employed to renew stagnated individuals in the population when stagnation occurs. OLBADE is compared with five excellent DE variants under a large test-bed containing CEC2013, CEC2014, CEC2017 and CEC2022 test suites to verify its effectiveness. In addition, OLBADE is applied in parameter identification problem of photovoltaic model to verify its feasibility. Experimental results demonstrate that OLBADE achieves higher solution accuracy, faster convergence speed and better stability.

Best-worst individuals driven multiple-layered differential evolution

Conventional differential evolution (DE) algorithms have been widely used for optimisation problems but suffer from low performance and premature convergence. Hence, researchers have proposed advanced variants to enhance performance using information and strategies. However, the performance of the variants remains limited because they only utilise limited information of individuals. A more suitable search orientation for the algorithm is required to effectively leverage individual information and enhance the processing of mid-population data. This study presents a novel best-worst individual-driven multiple-layered differential evolution (BWDE) algorithm. A best-worst individual-driven mechanism is designed that leverages various pieces of individual information to overcome local optima or stagnation, facilitating escape from the current search space and maintaining group fitness levels. In addition, the five-layer structure of the BWDE algorithm allows for the adequate use of multiple layers of information to determine the evolutionary direction of a population. Consequently, a balance is achieved between population development and exploration at distinct evolutionary stages. Extensive experiments are conducted using the Congress on Evolutionary Computation (CEC) 2017 and 2011 standard benchmark functions to evaluate the effectiveness of the proposed algorithm. The results are compared with those of classical algorithms, a winning algorithm at a CEC competition, and state-of-the-art DE variants. The experimental results demonstrate that the proposed BWDE algorithm outperforms its competitors and achieves more competitive results.

SaMDE: A Self Adaptive Choice of DNDE and SPIDE Algorithms with MRLDE.

Differential evolution (DE) is a proficient optimizer and has been broadly implemented in real life applications of various fields. Several mutation based adaptive approaches have been suggested to improve the algorithm efficiency in recent years. In this paper, a novel self-adaptive method called SaMDE has been designed and implemented on the mutation-based modified DE variants such as modified randomized localization-based DE (MRLDE), donor mutation based DE (DNDE), and sequential parabolic interpolation based DE (SPIDE), which were proposed by the authors in previous research. Using the proposed adaptive technique, an appropriate mutation strategy from DNDE and SPIDE can be selected automatically for the MRLDE algorithm. The experimental results on 50 benchmark problems taken of various test suits and a real-world application of minimization of the potential molecular energy problem validate the superiority of SaMDE over other DE variations.

Differential evolution with migration mechanism and information reutilization for global optimization

Differential evolution (DE) is an efficacious global optimization algorithm, and many variants have been advanced since its inception. During the iterative search process, any individual in any DE-type algorithm can likely locate a local optimum, and once that happens it may need many attempts for this individual to find another better solution, thus leading to ineffective consumption of computing resources and decline in the opportunity to search other promising regions. Therefor, this article proposes a DE with migration mechanism and information reutilization (MIDE). Specifically, a migration mechanism is first presented to make individuals located at local optima abandon current locations and move to other regions to continue their search, tending to solve the problem above. Meanwhile, a new mutation strategy named DE/pbest/1 with external archive is introduced to reutilize abandoned local optima to provide helpful information of evolution. Additionally, the settings of control parameters associated with this mutation strategy are designed in such a manner that they can contribute to well balancing exploration and exploitation. To evaluate the performance of MIDE, extensive experiments are conducted on CEC 2017 and CEC 2014 test suites, and the comparison results between MIDE and 19 competitors (including 13 state-of-the-art DE variants and six winner algorithms of CEC 2014 and CEC 2017 competitions) demonstrate MIDE's competitive performance.

A differential evolution with autonomous strategy selection and its application in remote sensing image denoising

Differential evolution (DE) is an efficient and effective global optimization algorithm, and many DE variants with adaptive strategy selection have been proposed. However, among the existing DE variants with adaptive strategy selection, only a few last generations of population information are utilized to conduct statistical analysis for strategy selection, and the population cumulative historical experience information does not be fully used to assist population to effectively search. In this paper, a DE with autonomous strategy selection (ASS-DE) is proposed to make full use of the population cumulative historical experience information. In ASS-DE, an individual can autonomously choose the better mutation strategy guided by the cumulative historical experience. As the same time, a parameter updating mechanism with archive is introduced to assign appropriate control parameters to the strategy. Additionally, an evolutionary learning mechanism based on individual similarity is proposed to guide the learning of historical experience information. To verify the performance of ASS-DE, CEC2015 and CEC2017 benchmark functions and the real-world optimization problem of wavelet parameter optimization in remote sensing image denoising is compared with some state-of-the-art intelligence algorithms. The experimental results show that ASS-DE obtains the promising performance compared with other DE variants. The source code of ASS-DE can be found in GitHub at https://github.com/Ramessis/ASS-DE_denoising.git.

A non-dominated discrete differential evolution for fuel loading pattern optimization of a nuclear research reactor

Abstract This paper presents a detailed description of a new variant of differential evolution for nuclear reactor refueling optimization problem. This variant combines the elitism strategy with a discrete differential evolution. The elitism strategy allows non-dominated solutions found during the search and stored in the archive to participate in the differential evolution operation. The population size is the same as the archive size, and the number of non-dominated solutions participating in the search at a particular generation is controlled by a specific probability. The proposed method is successfully applied to a nuclear research reactor for its first refueling time to search for optimal loading patterns that both maximize the effective multiplication k eff and minimize the power peaking factor PPF of the reactor. The optimal loading patterns can significantly improve the operational time and safety of the reactor compared to the loading pattern used in practice.

Differential Evolution With Domain Transform

Although a significant advancement of differential evolution (DE) for global optimization has been witnessed in the past two decades, the problems of premature convergence and stagnation are still open questions that hinder the performance. Both are likely to occur on complicated multi-modal functions, but the phenomena differ. Premature convergence refers to a rapid loss of population diversity when attracted to a local minimum while stagnation happens even though the population is diverse. To deal with these problems, this paper proposes a domain transform (DT) methodology. Different from existing fitness analysis which mainly utilizes the original fitness landscape information, DT yields a transformed fitness landscape with transform operation to the frequency domain and inverse transform operation back to the solution domain, between which the first few highest frequencies are removed. With the deletion operation, the transformed fitness landscape becomes smoother and facilitates the escape from local minima and stagnation on complicated multi-modal functions. Simulation results show that DT significantly improves the population successful update rate and population convergence. The constructed DTDE algorithm consequently exhibits remarkable improvements on the baseline algorithm and outperforms several state-of-the-art DE variants. DT has also been extended for noisy optimization and it performs better than the baseline, the classic resampling method, state-ofthe-art DE variants as well as several popular noisy evolutionary optimization algorithms.