Abstract
During the last two decades, a large number of metaheuristics have been proposed, leading to various studies that call for a deeper insight into the behaviour, efficiency and effectiveness of such methods. Among numerous concerns that are briefly reviewed in this paper, the presence of a structural bias (i.e. the tendency, not justified by the fitness landscape, to visit some regions of the search space more frequently than other regions) has recently been detected in simple versions of the genetic algorithm and particle swarm optimization. As of today, it remains unclear how frequently such a behaviour occurs in population-based swarm intelligence and evolutionary computation methods, and to what extent structural bias affects their performance. The present study focuses on the search for structural bias in various variants of particle swarm optimization and differential evolution algorithms, as well as in the traditional direct search methods proposed by Nelder–Mead and Rosenbrock half a century ago. We found that these historical direct search methods are structurally unbiased. However, most tested new metaheuristics are structurally biased, and at least some presence of structural bias can be observed in almost all their variants. The presence of structural bias seems to be stronger in particle swarm optimization algorithms than in differential evolution algorithms. The relationships between the strength of the structural bias and the dimensionality of the search space, the number of allowed function calls and the population size are complex and hard to generalize. For 14 algorithms tested on the CEC2011 real-world problems and the CEC2014 artificial benchmarks, no clear relationship between the strength of the structural bias and the performance of the algorithm was found. However, at least for artificial benchmarks, such old and structurally unbiased methods like Nelder–Mead algorithm performed relatively well. This is a warning that the presence of structural bias in novel metaheuristics may hamper their search abilities.
Highlights
A number of direct search methods (Kolda et al 2003) to solve continuous optimization problems were already proposed over half a of century ago, the last two decades have witnessed a very rapid influx of papers introducing novel metaheuristics (e.g. Boussad et al 2013; Xing and Gao 2014; Salcedo-Sanz 2016)
As one may conclude from this brief review, the question regarding the presence of structural bias in such popular families of optimization algorithms as particle swarm optimization and differential evolution just touches the tip of the iceberg that, if warmed up, may flood the metaheuristics with at least partly-deserved scepticism
Three out of four methods that are almost structurally unbiased (CLPSO, CDE and Nelder–Mead algorithm (NMA)) show quite good performance: CLPSO is among the best particle swarm optimization (PSO) variants; CDE is ranked in the middle of the differential evolution (DE)-based ones; NMA remains competitive against 50 years younger optimizers on artificial benchmark problems from CEC2014 and at least for some real-world problems from CEC2011
Summary
A number of direct search methods (Kolda et al 2003) to solve continuous optimization problems were already proposed over half a of century ago, the last two decades have witnessed a very rapid influx of papers introducing novel metaheuristics (e.g. Boussad et al 2013; Xing and Gao 2014; Salcedo-Sanz 2016). The function defined in Eq (1) does not have any stable fitness landscape, as each time a particular location x is visited, the value of f (x) is generated anew from the uniform distribution This is a slightly different approach from that proposed in Cleghorn and Engelbrecht (2014, 2015), where f is static during a particular run. When searching for the minimum of f , if the algorithm does not sample some parts of the search space more often than the others (due to algorithms’ own structural bias), the distribution of the positions of the best solutions found during multiple runs should be uniform in the search space This can be confirmed by a visual inspection of the respective plots or, more objectively, by using some statistical tests. The objective is to check whether the presence of structural bias can affect the practical application of metaheuristics (see discussion in Michalewicz 2012)
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
