Real-parameter Single Objective Optimization Research Articles

An Experimental Comparison of Self-Adaptive Differential Evolution Algorithms to Induce Oblique Decision Trees

This study addresses the challenge of generating accurate and compact oblique decision trees using self-adaptive differential evolution algorithms. Although traditional decision tree induction methods create explainable models, they often fail to achieve optimal classification accuracy. To overcome these limitations, other strategies, such as those based on evolutionary computation, have been proposed in the literature. In particular, we evaluate the use of self-adaptive differential evolution variants to evolve a population of oblique decision trees encoded as real-valued vectors. Our proposal includes (1) an alternative initialization strategy that reduces redundant nodes and (2) a fitness function that penalizes excessive leaf nodes, promoting smaller and more accurate decision trees. We perform a comparative performance analysis of these differential evolution variants, showing that while they exhibit similar statistical behavior, the Single-Objective real-parameter optimization (jSO) method produces the most accurate oblique decision trees and is second best in compactness. The findings highlight the potential of self-adaptive differential evolution algorithms to improve the effectiveness of oblique decision trees in machine learning applications.

A framework of integrated differential evolution variants based on adaptive relay mode for global optimization

Differential evolution (DE) is highly competitive in single-objective real parameter optimization. However, there are still some problems with differential evolutionary variants in optimizing complex multimode functions, such as poor solution accuracy and slow convergence. To address these problems, an optimization framework of integrated DE variants based on adaptive relay mode (fDE-ARM) is proposed. In this framework, different DE variants are integrated through two adaptive relay mechanisms to give full play to the advantages of the algorithms, thereby improving the performance of the whole algorithm. For the first adaptive relay mode, when the currently executed algorithm is judged to have converged, the population is updated by Gaussian random walk, and then the optimization is continued through the relay algorithm. For the second adaptive relay mode, the relay condition is determined by the average fitness improvement rate. If the condition is met, the relay algorithm takes over the optimization. At the same time, if the diversity is lower than the threshold, to improve the exploration, the roulette wheel method is used to select some individuals to perform the Gaussian random walk. In addition, a feedback mechanism is introduced in the relay process to avoid false switching. To verify the performance of the proposed algorithm, extensive simulations are performed on CEC2005, CEC2014, CEC2017, and CEC2021 benchmark functions. In addition, three engineering problems are used to test the performance. Compared with some newly proposed optimization algorithms, fDE-ARM is statistically superior to the comparison algorithms in terms of solution accuracy, convergence speed, and stability.

DCWPSO: particle swarm optimization with dynamic inertia weight updating and enhanced learning strategies.

Particle swarm optimization (PSO) stands as a prominent and robust meta-heuristic algorithm within swarm intelligence (SI). It originated in 1995 by simulating the foraging behavior of bird flocks. In recent years, numerous PSO variants have been proposed to address various optimization applications. However, the overall performance of these variants has not been deemed satisfactory. This article introduces a novel PSO variant, presenting three key contributions: First, a novel dynamic oscillation inertia weight is introduced to strike a balance between exploration and exploitation; Second, the utilization of cosine similarity and dynamic neighborhood strategy enhances both the quality of solution and the diversity of particle populations; Third, a unique worst-best example learning strategy is proposed to enhance the quality of the least favorable solution and consequently improving the overall population. The algorithm's validation is conducted using a test suite comprised of benchmarks from the CEC2014 and CEC2022 test suites on real-parameter single-objective optimization. The experimental results demonstrate the competitiveness of our algorithm against recently proposed state-of-the-art PSO variants and well-known algorithms.

Differential evolution with alternation between steady monopoly and transient competition of mutation strategies

Real parameter single objective optimization has been studied for decades. In recent, long-term search is emphasized based on the consideration that, in the field, solving difficulty often scales exponentially with the increase of function dimensionality. For long-term search, Differential Evolution (DE) still performs outstanding among types of population-based metaheuristics. In this paper, based on IMODE – a DE algorithm with three mutation strategies, we propose AMCDE – Differential Evolution with Alternation between steady Monopoly and transient Competition of mutation strategies. Our algorithm has two states. In the steady state – monopoly, a selected mutation strategy controls the whole population. Once improvement in fitness becomes difficult, the transient state – competition – arises. In the competition state, similar with IMODE, each of the mutation strategies controls a proportion of positions in the population and competes with the others. For enhancement, we propose that positions controlled by the winner among the three mutation strategies continue to be controlled by the mutation strategy in the next generation. Besides, in the competition state, the original selection strategy of IMODE is revised by us for diversification, while adaptation of the crossover rate is updated. Our experiment is based on three CEC benchmark test suites and the CEC 2011 suite of real world optimization problems. AMCDE is compared with seven peers. Based on experimental results, our algorithm demonstrates superior or at the very least comparable performance for long-term search compared to the peers. Moreover, we do experimental observation on AMCDE.

Differential Evolution with wavelet basis function based parameter control and dimensional interchange for diversity enhancement

Differential Evolution (DE) is a potent population-based global optimization algorithm which has already proven efficient for optimization demands in engineering applications. However, even the state-of-the-art DE variants still suffer premature convergence and lack of diversity. In order to overcome the above mentioned weaknesses, this paper proposes a brand-new DE variant, namely zDE algorithm, for single-objective numerical optimization. The main contributions can be summarized as follows: First, an improved wavelet basis function is incorporated into the generation of the scale factor F and a Minkowski Distance based adaptation scheme is proposed for the adaptation of it. Second, a new trial vector generation strategy is first proposed as a supplementary of the existing strategies, and t-distribution is incorporated into this strategy. Third, a novel diversity enhancement technique is firstly proposed by changing the dimensional parameters of the individuals in the population. The zDE algorithm is validated under 88 benchmarks from the CEC2013, CEC2014, and CEC2017 test suites for real-parameter single-objective optimization, and the results show the superiority of our algorithm in comparison with the recent state-of-the-art DE variants.

Dimension improvements based adaptation of control parameters in Differential Evolution: A fitness-value-independent approach

In this paper, we proposed a novel fitness-value-independent Differential Evolution to tackle complex real-parameter single-objective optimization. There were three innovations in the algorithm: First, dimension improvements based adaptation schemes for control parameters F and CR were advanced in this paper. Different from those fitness value improvements based parameter control in the recent winner DE variants, our algorithm has fitness-value-independent characteristic, therefore, it can be applied in much wider optimization scenarios especially for those that the exact fitness values of the objectives are unavailable. Second, a combined parabolic–linear reduction scheme for population size is employed in the algorithm based on the observation that the slower reduction of population size at the earlier stage of the evolution usually helps to get better perception of the objectives while a linear reduction of population size in the later stage helps to get better exploitation. Third, an indicator is proposed to monitor the diversity of the population and a corresponding population enhancement technique is launched when the diversity is detected bad. The first two innovations have the Parameter adaptive characteristic of the DE algorithm while the third innovation refers to population enhancement technique, therefore, we name our algorithm “the PaDE-pet algorithm”. Then, the algorithm is validated under a larger test suite containing 88 benchmarks from CEC2013, CEC2014 and CEC2017 test suites for real-parameter single objective optimization, which may avoid over-fitting problem in comparison with employing a test suite containing a small number of benchmarks. The experiments support the superiority of the novel PaDE-pet algorithm in comparison with several recently proposed powerful DE variants.

Multiple Elite Individual Guided Piecewise Search-Based Differential Evolution

The differential evolution (DE) algorithm relies mainly on mutation strategy and control parameters' selection. To take full advantage of top elite individuals in terms of fitness and success rates, a new mutation operator is proposed. The control parameters such as scale factor and crossover rate are tuned based on their success rates recorded over past evolutionary stages. The proposed DE variant, MIDE, performs the evolution in a piecewise manner, i.e., after every predefined evolutionary stages, MIDE adjusts its settings to enrich its diversity skills. The performance of the MIDE is validated on two different sets of benchmarks: CEC 2014 and CEC 2017 (special sessions & competitions on real-parameter single objective optimization) using different performance measures. In the end, MIDE is also applied to solve constrained engineering problems. The efficiency and effectiveness of the MIDE are further confirmed by a set of experiments.

A novel particle swarm optimization algorithm with Lévy flight and orthogonal learning

• In LOPSO, the there major highlights are described below. • First, a novel PSO based on Lévy flight and orthogonal learning with enhanced exploitation ability and faster search efficiency is proposed. The jumping ability of Lévy flights is utilized to enhance the exploration, while the orthogonal learning process aims to strengthen the exploitation;. • Second, an orthogonal learning process for particle swarms is developed. In the orthogonal learning process, the inclusion of the Lévy flights term in the levels of the Taguchi method is determined based on the flight probability value. Besides, parallel computation is easier to perform during orthogonal learning, which greatly improves efficiency. • Third, each particle in the proposed LOPSO learns from the only best experience generated by a new search strategy instead of the history best experience of each particle and the global best experience that used in the classical PSO variants, thus avoiding the conflict between above two guides. Taguchi method This paper presents a novel particle swarm optimization algorithm (PSO) variant to tackle single-objective numerical optimization, named “Lévy flight orthogonal learning particle swarm optimization” (LOPSO). There are three contributions mentioned in the paper: First, a novel PSO based on Lévy flight and orthogonal learning with enhanced exploitation ability and faster search efficiency is proposed. The jumping ability of Lévy flight is utilized to enhance the exploration, while the orthogonal learning process aims to strengthen the exploitation; Second, an orthogonal learning process for particle swarms is developed. In the orthogonal learning process, the Lévy flight term's inclusion in the Taguchi method levels is determined based on the flight probability value. Besides, parallel computation is easier to perform during orthogonal learning, which greatly improves efficiency. Third, each particle in the proposed LOPSO learns from the only best experience generated by a new search strategy instead of the historical best experience of each particle and the global best experience that is used in the classical PSO variants, thus avoiding the conflict between the above two guides. A large test suite containing benchmarks from the CEC2013 and the CEC2015 test suites on real-parameter single-objective optimization is employed in the algorithm validation, and extensive experiments demonstrate that the comprehensive performance of LOPSO ranks first in terms of accuracy and Wilcoxon signed-rank test in most cases with the same population size and iteration steps and similar time consumption, when compared with the canonical PSO and 13 state-of-the-art PSO variants.

A hyper-heuristic guided by a probabilistic graphical model for single-objective real-parameter optimization

A hyper-heuristic guided by a probabilistic graphical model for single-objective real-parameter optimization

A Voting-Mechanism-Based Ensemble Framework for Constraint Handling Techniques

Effective constraint handling techniques (CHTs) are of great significance for evolutionary algorithms (EAs) dealing with constrained optimization problems (COPs). To date, many CHTs, such as penalty function, superiority of feasible solutions, and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\epsilon $ </tex-math></inline-formula> -constraint (EC), have been designed. However, different CHTs are usually suited to different problems, even the most appropriate technique changes along with the stages of the optimization process. Motivated by this phenomenon, we propose a voting-mechanism-based ensemble framework, named voting mechanism for constraint handling (VMCH), to integrate multiple CHTs for solving various COPs. In this framework, each CHT acts as a voter, all voters vote for each pair of solutions, and the solution in each pair with the highest weighted votes is considered better. In addition, an adaptive strategy is developed to adjust the voter weights according to their historical voting performance. To investigate the performance of VMCH in improving existing algorithms, the proposed VMCH is embedded into the three best algorithms in the competition on constrained single objective real-parameter optimization at CEC 2018, namely, MAgES, iLSHADE <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\epsilon $ </tex-math></inline-formula> , and IUDE, to form three new algorithm versions, i.e., MAgES-VMCH, iLSHADE <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\epsilon $ </tex-math></inline-formula> -VMCH, and IUDE-VMCH. They are compared with seven state-of-the-art peer algorithms. Extensive experiments are conducted on 57 real-world COPs. The ranking results show that the new algorithm version MAgES-VMCH takes first place among the ten comparison algorithms. Moreover, all the new VMCH-enhanced versions of the three best algorithms are superior to their original versions. Therefore, the proposed VMCH framework can achieve competitive performance in solving COPs.

Individuals redistribution based on differential evolution for covariance matrix adaptation evolution strategy

Among population-based metaheuristics, both Differential Evolution (DE) and Covariance Matrix Adaptation Evolution Strategy (CMA-ES) perform outstanding for real parameter single objective optimization. Compared with DE, CMA-ES stagnates much earlier in many occasions. In this paper, we propose CMA-ES with individuals redistribution based on DE, IR-CMA-ES, to address stagnation in CMA-ES. We execute experiments based on two benchmark test suites to compare our algorithm with nine peers. Experimental results show that our IR-CMA-ES is competitive in the field of real parameter single objective optimization.

Flower pollination algorithm with pollinator attraction

The Flower Pollination Algorithm (FPA) is a highly efficient optimization algorithm that is inspired by the evolution process of flowering plants. In the present study, a modified version of FPA is proposed accounting for an additional feature of flower pollination in nature that is the so-called pollinator attraction. Pollinator attraction represents the natural tendency of flower species to evolve in order to attract pollinators by using their colour, shape and scent as well as nutritious rewards. To reflect this evolution mechanism, the proposed FPA variant with Pollinator Attraction (FPAPA) provides fitter flowers of the population with higher probabilities of achieving pollen transfer via biotic pollination than other flowers. FPAPA is tested against a set of 28 benchmark mathematical functions, defined in IEEE-CEC’13 for real-parameter single-objective optimization problems, as well as structural optimization problems. Numerical experiments show that the modified FPA represents a statistically significant improvement upon the original FPA and that it can outperform other state-of-the-art optimization algorithms offering better and more robust optimal solutions. Additional research is suggested to combine FPAPA with other modified and hybridized versions of FPA to further increase its performance in challenging optimization problems.

PSO-sono: A novel PSO variant for single-objective numerical optimization

Particle Swarm Optimization(PSO) is a well-known and powerful meta-heuristic algorithm in Swarm Intelligence (SI), and it was invented by simulating the foraging behavior of bird flock in 1995. Recently, many different PSO variants were proposed to tackle different optimization applications, however, the overall performance of these variants were not satisfactory. In this paper, a new PSO variant is advanced to tackle single-objective numerical optimization, and there are three contributions mentioned in the paper: First, a sorted particle swarm with hybrid paradigms is proposed to improve the optimization performance; Second, novel adaptation schemes both for the ratio of each paradigm and the constriction coefficients are proposed during the iteration; Third, a fully-informed search scheme based on the global optimum in each generation is proposed which helps the algorithm to jump out the local optimum and improve the overall performance. A large test suite containing benchmarks from CEC2013, CEC2014 and CEC2017 test suites on real-parameter single-objective optimization is employed in the algorithm validation, and the experiment results show the competitiveness of our algorithm with the famous or recently proposed state-of-the-art PSO variants.

Flower pollination algorithm parameters tuning.

The flower pollination algorithm (FPA) is a highly efficient metaheuristic optimization algorithm that is inspired by the pollination process of flowering species. FPA is characterised by simplicity in its formulation and high computational performance. Previous studies on FPA assume fixed parameter values based on empirical observations or experimental comparisons of limited scale and scope. In this study, a comprehensive effort is made to identify appropriate values of the FPA parameters that maximize its computational performance. To serve this goal, a simple non-iterative, single-stage sampling tuning method is employed, oriented towards practical applications of FPA. The tuning method is applied to the set of 28 functions specified in IEEE-CEC’13 for real-parameter single-objective optimization problems. It is found that the optimal FPA parameters depend significantly on the objective functions, the problem dimensions and affordable computational cost. Furthermore, it is found that the FPA parameters that minimize mean prediction errors do not always offer the most robust predictions. At the end of this study, recommendations are made for setting the optimal FPA parameters as a function of problem dimensions and affordable computational cost.

Improved ensemble of differential evolution variants.

In the field of Differential Evolution (DE), a number of measures have been used to enhance algorithm. However, most of the measures need revision for fitting ensemble of different combinations of DE operators—ensemble DE algorithm. Meanwhile, although ensemble DE algorithm may show better performance than each of its constituent algorithms, there still exists the possibility of further improvement on performance with the help of revised measures. In this paper, we manage to implement measures into Ensemble of Differential Evolution Variants (EDEV). Firstly, we extend the collecting range of optional external archive of JADE—one of the constituent algorithm in EDEV. Then, we revise and implement the Event-Triggered Impulsive (ETI) control. Finally, Linear Population Size Reduction (LPSR) is used by us. Then, we obtain Improved Ensemble of Differential Evolution Variants (IEDEV). In our experiments, good performers in the CEC competitions on real parameter single objective optimization among population-based metaheuristics, state-of-the-art DE algorithms, or up-to-date DE algorithms are involved. Experiments show that our IEDEV is very competitive.

On Selection of a Benchmark by Determining the Algorithms’ Qualities

The authors got the motivation for writing the article based on an issue, with which developers of the newly developed nature-inspired algorithms are usually confronted today: How to select the test benchmark such that it highlights the quality of the developed algorithm most fairly? In line with this, the CEC Competitions on Real-Parameter Single-Objective Optimization benchmarks that were issued several times in the last decade, serve as a testbed for evaluating the collection of nature-inspired algorithms selected in our study. Indeed, this article addresses two research questions: (1) How the selected benchmark affects the ranking of the particular algorithm, and (2) If it is possible to find the best algorithm capable of outperforming all the others on all the selected benchmarks. Ten outstanding algorithms (also winners of particular competitions) from different periods in the last decade were collected and applied to benchmarks issued during the same time period. A comparative analysis showed that there is a strong correlation between the rankings of the algorithms and the benchmarks used, although some deviations arose in ranking the best algorithms. The possible reasons for these deviations were exposed and commented on.

L-SHADE-E: Ensemble of two differential evolution algorithms originating from L-SHADE

For real parameter single objective optimization, Differential Evolution (DE) performs better than other types of population-based metaheuristic. Nevertheless, in the field of DE, it is impossible for a given combination of operators to perform well in all fitness landscapes. Practice has proved that assembling more than one combinations of operators may lead to improvement in solution. However, some DE algorithms with good performance, e.g., one of the best performers in competitions of real parameter single objective optimization among population-based metaheuristics held by the series of IEEE Congress on Evolutionary Computation - L-SHADE-EpSin, are still not involved in ensemble. Furthermore, ensemble based on similar DE algorithms is rarely reported. Based on such a background, we propose L-SHADE-E, ensemble of L-SHADE-EpSin and L-SHADE-RSP in this paper. The two constituent algorithms are both variants of L-SHADE and belong to similar DE algorithms. In our algorithm, L-SHADE-EpSin or L-SHADE-RSP is selected randomly to run at the beginning. If progress on fitness is low for generations, the other constituent algorithm takes over population immediately. Our experiments are based on the CEC 2014 and CEC 2017 benchmark test suites. Altogether, our algorithm is compared with nine DE algorithms and three population-based metaheuristics other than DE. Experimental results show that our algorithm is very competitive.

Understanding the problem space in single-objective numerical optimization using exploratory landscape analysis

In benchmarking theory, creating a comprehensive and uniformly distributed set of problems is a crucial first step to designing a good benchmark. However, this step is also one of the hardest, as it can be difficult to determine how to evaluate the quality of the chosen problem set.In this article, we evaluate if the field of exploratory landscape analysis can be used to develop a generalized method of visualizing a set of arbitrary optimization functions. We present a method for visually determining the distribution of problems within a benchmark set using exploratory landscape analysis combined with clustering and t-sne visualization, and evaluate and explain the visualization this methodology produces.The proposed method is evaluated on a set of benchmark problems taken from two well known state-of-the-art real-parameter single objective optimization benchmarks: the CEC Special Sessions and Competitions on Real-Parameter Single Objective optimization, and the GECCO Black-Box Optimization Benchmark workshops.The main goal of this paper is to present an analysis of how exploratory landscape analysis can be used to visualize a benchmark problem set. We show that this method can provide a clear visualization of a benchmark problem set and shows the similarities of the problems in it by placing similar problems visually close together. We also show that the problem sets of the above benchmarks have a somewhat distinct set of problems that do not overlap.In addition, by applying feature selection approaches we show that a number of landscape features provided by state-of-the-art exploratory landscape analysis libraries are redundant and that a large amount of them are not invariant to simple transforms like scaling and shifting, at least when analyzing these two datasets.

Enhancing Differential Evolution With Novel Parameter Control

In this paper, we proposed a novel DE variant named DE-NPC for real parameter single objective optimization. In DE-NPC algorithm, a novel adaptation scheme for the scale factor F is first proposed, which is based on the location information of the population rather than the fitness difference. The adaptation scheme of crossover rate CR in our DE-NPC is based on its success probability. Furthermore, a novel population size reduction scheme is also employed in DE-NPC, which can get a better perception of the landscape of objectives and consequently obtain an overall better performance. The algorithm validation is conducted under our test suite containing 88 benchmarks from CEC2013, CEC2014 and CEC2017 in comparison with several state-of-the-art DE variants. The experiment results show that our novel DE-NPC algorithm is competitive with these state-of-the-art DE variants.

PaDE-NPC: Parameter Adaptive Differential Evolution With Novel Parameter Control for Single-Objective Optimization

Single objective real-parameter optimization problems exist in many areas of the real world, and Differential Evolution (DE) is a powerful population based stochastic optimization approach for tackling such problems. There are many different mutation strategies mentioned in the literature, and each of them has its own advantage. In this paper, we propose combined mutation strategies which can make a full use of the advantage of each mutation strategy regarding a population diversity indicator during the evolution. Furthermore, Novel Parameter Control (NPC) for the three control parameters including the scale factor $F$ , crossover rate $CR$ and population size $PS$ are also proposed in the paper. Different from employing the fitness value as a weight in recent proposed state-of-the-art DE variants, our PaDE-NPC algorithm can tackle a large optimization problems especially for those the fitness differences are unavailable; Moreover, a platform based population size reduction scheme is also involved in the NPC, which can get a better perception of the landscape at the early stage of the evolution while obtaining a balance between exploration and exploitation in the later part of the evolution. The novel PaDE-NPC algorithm is verified under 58 benchmark functions from CEC2013 and CEC2017 test suits for real-parameter optimization competitions and experiment results show that our proposed PaDE-NPC algorithm outperforms these recently proposed powerful DE variants.