Single-objective Numerical Optimization Research Articles

Multi-strategy learning-based particle swarm optimization algorithm for COVID-19 threshold segmentation

With advancements in science and technology, the depth of human research on COVID-19 is increasing, making the investigation of medical images a focal point. Image segmentation, a crucial step preceding image processing, holds significance in the realm of medical image analysis. Traditional threshold image segmentation proves to be less efficient, posing challenges in selecting an appropriate threshold value. In response to these issues, this paper introduces Inner-based multi-strategy particle swarm optimization (IPSOsono) for conducting numerical experiments and enhancing threshold image segmentation in COVID-19 medical images. A novel dynamic oscillatory weight, derived from the PSO variant for single-objective numerical optimization (PSOsono) is incorporated. Simultaneously, the historical optimal positions of individuals in the particle swarm undergo random updates, diminishing the likelihood of algorithm stagnation and local optima. Moreover, an inner selection learning mechanism is proposed in the update of optimal positions, dynamically refining the global optimal solution. In the CEC 2013 benchmark test, PSOsono demonstrates a certain advantage in optimization capability compared to algorithms proposed in recent years, proving the effectiveness and feasibility of PSOsono. In the Minimum Cross Entropy threshold segmentation experiments for COVID-19, PSOsono exhibits a more prominent segmentation capability compared to other algorithms, showing good generalization across 6 CT images and further validating the practicality of the algorithm.

A cross-benchmark examination of feature-based algorithm selector generalization in single-objective numerical optimization

The task of selecting the best optimization algorithm for a particular problem is known as algorithm selection (AS). This involves training a model using landscape characteristics to predict algorithm performance, but a key challenge remains: making AS models generalize effectively to new, untrained benchmark suites. This study assesses AS models’ generalizability in single-objective numerical optimization across diverse benchmark suites. Using Exploratory Landscape Analysis (ELA) and transformer-based (TransOpt) features, the research investigates their individual and combined effectiveness in AS across four benchmarks: BBOB, AFFINE, RANDOM, and ZIGZAG. AS models perform differently based on benchmark suite similarities in algorithm performance distributions and single-best solvers. When suites align, these models underperform against a baseline predicting mean algorithm performance; yet, they outperform the baseline when suites differ in performance distributions and solvers. The AS models trained using the ELA landscape features are better than the models trained using the TransOpt features on the BBOB and AFFINE benchmark suites, while the opposite is true for the RANDOM benchmark suite. Ultimately, the study reveals challenges in accurately capturing algorithm performance through problem landscape features (ELA or TransOpt), impacting AS model applicability.

Automatic design of mutation parameter adaptation for differential evolution

In this paper the Efficient Global Optimization algorithm is applied to design the adaptation strategy for mutation parameter in Differential Evolution. The adaptation strategy is represented as a Taylor series, to allow exploring a search space of different curves. The tuning of the adaptation is performed on the L-NTADE algorithm using the benchmark of Congress on Evolutionary Computation competition on single-objective numerical optimization 2017. The experimental results show that the discovered dependence between the success rate and the parameter in current-to-pbest mutation strategy allows improving the algorithm performance in various cases.

A sequential quadratic programming based strategy for particle swarm optimization on single-objective numerical optimization

Over the last decade, particle swarm optimization has become increasingly sophisticated because well-balanced exploration and exploitation mechanisms have been proposed. The sequential quadratic programming method, which is widely used for real-parameter optimization problems, demonstrates its outstanding local search capability. In this study, two mechanisms are proposed and integrated into particle swarm optimization for single-objective numerical optimization. A novel ratio adaptation scheme is utilized for calculating the proportion of subpopulations and intermittently invoking the sequential quadratic programming for local search start from the best particle to seek a better solution. The novel particle swarm optimization variant was validated on CEC2013, CEC2014, and CEC2017 benchmark functions. The experimental results demonstrate impressive performance compared with the state-of-the-art particle swarm optimization-based algorithms. Furthermore, the results also illustrate the effectiveness of the two mechanisms when cooperating to achieve significant improvement.

A Multi-Strategy Differential Evolution Algorithm with Adaptive Similarity Selection Rule

The differential evolution (DE) algorithm is a simple and efficient population-based evolutionary algorithm. In DE, the mutation strategy and the control parameter play important roles in performance enhancement. However, single strategy and fixed parameter are not universally applicable to problems and evolution stages with diverse characteristics; besides, the weakness of the advanced DE optimization framework, called selective-candidate framework with similarity selection rule (SCSS), is found by focusing on its single strategy and fixed parameter greedy degree (GD) setting. To address these problems, we mainly combine the multiple candidates generation with multi-strategy (MCG-MS) and the adaptive similarity selection (ASS) rule. On the one hand, in MCG-MS, two symmetrical mutation strategies, “DE/current-to-pbest-w/1” and designed “DE/current-to-cbest-w/1”, are utilized to build the multi-strategy to produce two candidate individuals, which prevents the over-approximation of the candidate in SCSS. On the other hand, the ASS rule provides the individual selection mechanism for multi-strategy to determine the offspring from two candidates, where parameter GD is designed to increase linearly with evolution to maintain diversity at the early evolution stage and accelerate convergence at the later evolution stage. Based on the advanced algorithm jSO, replacing its offspring generation strategy with the combination of MCG-MS and ASS rule, this paper proposes multi-strategy differential evolution algorithm with adaptive similarity selection rule (MSDE-ASS). It combines the advantages of two symmetric strategies and has an efficient individual selection mechanism without parameter adjustment. MSDE-ASS is verified under the Congress on Evolutionary Computation (CEC) 2017 competition test suite on real-parameter single-objective numerical optimization, and the results indicate that, of the 174 cases in total, it wins in 81 cases and loses in 30 cases, and it has the smallest performance ranking value, of 3.05. Therefore, MSDE-ASS stands out compared to the other state-of-the-art DEs.

HPDE: A dynamic Hierarchical Population based Differential Evolution with novel diversity metric

Differential Evolution (DE) is one of the most effective methods for single-objective numerical optimization, however, the former state-of-the-art DE variants ignore the huge differences among the individuals in the population. To take full advantage of different individuals in the population, in this paper, a dynamic Hierarchical Population based Differential Evolution (HPDE) with novel diversity metric is proposed, and the HPDE algorithm has the following innovations: Firstly, the individuals in the hierarchical population are dynamically divided into two layers, the layer of elites, and the layer of ordinary individuals. Moreover, the ordinary layer individuals are further divided into secondary elites and inferior individuals. The individuals in the two layers employed different mutation strategies and the corresponding parameter control schemes. Secondly, a novel hierarchical population based mutation strategy is firstly proposed in our HPDE algorithm and the individuals in the elite layer employ the novel mutation strategy aiming at scouting the landscape of the objective while the ordinary individuals employ an improved “DE/target-pbest/1/bin” strategy aiming at getting a better balance between exploration and exploitation of the solution space. Thirdly, a Novel Parameter Control technique, namely NPC technique, is employed in the adaptation of control parameters during the evolution. Fourthly, a new metric of population diversity is proposed and premature convergence can be avoided by maintaining better population diversity. To evaluate the performance of our HPDE algorithm, intensive experiments are conducted on 88 benchmark functions from the CEC2013, CEC2014 and CEC2017 test suites, and the results show the competitiveness of our HPDE with six recent state-of-the-art DE variants, e.g. it obtained 28 similar or better performance improvements out of the total 30 benchmarks on 30D optimization in comparison with the winner algorithm, the LSHADE algorithm, of the CEC2014 competition and it also obtained 19 similar or better performance improvements out of the total 30 benchmarks on 30D optimization in comparison with the winner DE variant, the jSO algorithm, of the CEC2017 competition. Furthermore, the novel HPDE algorithm is utilized to estimate the parameters of the photovoltaic model, and the experiment results support its superiority as well.

Differential Evolution with wavelet basis function based parameter control and dimensional interchange for diversity enhancement

Differential Evolution (DE) is a potent population-based global optimization algorithm which has already proven efficient for optimization demands in engineering applications. However, even the state-of-the-art DE variants still suffer premature convergence and lack of diversity. In order to overcome the above mentioned weaknesses, this paper proposes a brand-new DE variant, namely zDE algorithm, for single-objective numerical optimization. The main contributions can be summarized as follows: First, an improved wavelet basis function is incorporated into the generation of the scale factor F and a Minkowski Distance based adaptation scheme is proposed for the adaptation of it. Second, a new trial vector generation strategy is first proposed as a supplementary of the existing strategies, and t-distribution is incorporated into this strategy. Third, a novel diversity enhancement technique is firstly proposed by changing the dimensional parameters of the individuals in the population. The zDE algorithm is validated under 88 benchmarks from the CEC2013, CEC2014, and CEC2017 test suites for real-parameter single-objective optimization, and the results show the superiority of our algorithm in comparison with the recent state-of-the-art DE variants.

FD-DE: Differential Evolution with fitness deviation based adaptation in parameter control

Differential Evolution (DE) is arguably one of the most powerful stochastic optimization algorithms for different optimization applications, however, even the state-of-the-art DE variants still have many weaknesses. In this study, a new powerful DE variant for single-objective numerical optimization is proposed, and there are several contributions within it: First, an enhanced wavelet basis function is proposed to generate scale factor F of each individual in the first stage of the evolution; Second, a hybrid trial vector generation strategy with perturbation and t-distribution is advanced to generate different trial vectors regarding different stages of the evolution; Third, a fitness deviation based parameter control is proposed for the adaptation of control parameters; Fourth, a novel diversity indicator is proposed and a restart scheme can be launched if necessary when the quality of the individuals is detected bad. The novel algorithm is validated using a large test suite containing 130 benchmarks from the universal test suites on single-objective numerical optimization, and the results approve the big improvement in comparison with several well-known state-of-the-art DE variants. Moreover, our algorithm is also validated under real-world optimization applications, and the results also support its superiority.

An improved ensemble particle swarm optimizer using niching behavior and covariance matrix adapted retreat phase

Over the past two decades, to overcome the limitations of certain algorithms, ensemble strategies or self-adaptive mechanisms for evolutionary computation algorithms have been proposed. Regardless of how these strategies or mechanisms were designed, their objective was to control the balance between the global and local search capabilities during the evolutionary process. Inspired by this, a novel ensemble strategy with three groups to improve the performance of the ensemble particle swarm optimizer (EPSO) is proposed. The first group uses a covariance matrix adapted retreat phase (CMAR), the second group induces the niching behavior for inertia weight particle swarm optimization (PSO), and the third group maintains the characteristics of a large subpopulation of EPSO. Furthermore, a sample pool and replacement mechanism are proposed to perturb the three groups. This strategy also recommends a group of empirical allocation rates for subpopulations based on various proportion combination tests. The performance of the proposed strategy is evaluated using CEC2005 benchmark functions with 10, 30, and 50-dimensional tests and compared with those of the state-of-the-art PSO variants: EPSO, a modified PSO using adaptive strategy (MPSO), PSO variant for single-objective numerical optimization (PSO-sono), terminal crossover and steering-based PSO (TCSPSO), self-adapting hybrid strategy PSO (SaHSPS), and pyramid PSO (PPSO). Experimental results demonstrate that the improved EPSO, using CMAR, niching behavior, and sample pool, performs best.

A novel particle swarm optimization algorithm with Lévy flight and orthogonal learning

• In LOPSO, the there major highlights are described below. • First, a novel PSO based on Lévy flight and orthogonal learning with enhanced exploitation ability and faster search efficiency is proposed. The jumping ability of Lévy flights is utilized to enhance the exploration, while the orthogonal learning process aims to strengthen the exploitation;. • Second, an orthogonal learning process for particle swarms is developed. In the orthogonal learning process, the inclusion of the Lévy flights term in the levels of the Taguchi method is determined based on the flight probability value. Besides, parallel computation is easier to perform during orthogonal learning, which greatly improves efficiency. • Third, each particle in the proposed LOPSO learns from the only best experience generated by a new search strategy instead of the history best experience of each particle and the global best experience that used in the classical PSO variants, thus avoiding the conflict between above two guides. Taguchi method This paper presents a novel particle swarm optimization algorithm (PSO) variant to tackle single-objective numerical optimization, named “Lévy flight orthogonal learning particle swarm optimization” (LOPSO). There are three contributions mentioned in the paper: First, a novel PSO based on Lévy flight and orthogonal learning with enhanced exploitation ability and faster search efficiency is proposed. The jumping ability of Lévy flight is utilized to enhance the exploration, while the orthogonal learning process aims to strengthen the exploitation; Second, an orthogonal learning process for particle swarms is developed. In the orthogonal learning process, the Lévy flight term's inclusion in the Taguchi method levels is determined based on the flight probability value. Besides, parallel computation is easier to perform during orthogonal learning, which greatly improves efficiency. Third, each particle in the proposed LOPSO learns from the only best experience generated by a new search strategy instead of the historical best experience of each particle and the global best experience that is used in the classical PSO variants, thus avoiding the conflict between the above two guides. A large test suite containing benchmarks from the CEC2013 and the CEC2015 test suites on real-parameter single-objective optimization is employed in the algorithm validation, and extensive experiments demonstrate that the comprehensive performance of LOPSO ranks first in terms of accuracy and Wilcoxon signed-rank test in most cases with the same population size and iteration steps and similar time consumption, when compared with the canonical PSO and 13 state-of-the-art PSO variants.

A Sequential Quadratic Programming Based Strategy for Particle Swarm Optimization on Single-Objective Numerical Optimization

A Sequential Quadratic Programming Based Strategy for Particle Swarm Optimization on Single-Objective Numerical Optimization

PSO-sono: A novel PSO variant for single-objective numerical optimization

Particle Swarm Optimization(PSO) is a well-known and powerful meta-heuristic algorithm in Swarm Intelligence (SI), and it was invented by simulating the foraging behavior of bird flock in 1995. Recently, many different PSO variants were proposed to tackle different optimization applications, however, the overall performance of these variants were not satisfactory. In this paper, a new PSO variant is advanced to tackle single-objective numerical optimization, and there are three contributions mentioned in the paper: First, a sorted particle swarm with hybrid paradigms is proposed to improve the optimization performance; Second, novel adaptation schemes both for the ratio of each paradigm and the constriction coefficients are proposed during the iteration; Third, a fully-informed search scheme based on the global optimum in each generation is proposed which helps the algorithm to jump out the local optimum and improve the overall performance. A large test suite containing benchmarks from CEC2013, CEC2014 and CEC2017 test suites on real-parameter single-objective optimization is employed in the algorithm validation, and the experiment results show the competitiveness of our algorithm with the famous or recently proposed state-of-the-art PSO variants.

Particle Swarm Optimization Algorithm with Multiple Phases for Solving Continuous Optimization Problems

An algorithm with different parameter settings often performs differently on the same problem. The parameter settings are difficult to determine before the optimization process. The variants of particle swarm optimization (PSO) algorithms are studied as exemplars of swarm intelligence algorithms. Based on the concept of building block thesis, a PSO algorithm with multiple phases was proposed to analyze the relation between search strategies and the solved problems. Two variants of the PSO algorithm, which were termed as the PSO with fixed phase (PSOFP) algorithm and PSO with dynamic phase (PSODP) algorithm, were compared with six variants of the standard PSO algorithm in the experimental study. The benchmark functions for single-objective numerical optimization, which includes 12 functions in 50 and 100 dimensions, are used in the experimental study, respectively. The experimental results have verified the generalization ability of the proposed PSO variants.

Enhancing QUasi-Affine TRansformation Evolution (QUATRE) with adaptation scheme on numerical optimization

Optimization problems exists extensively in real life, especially in science and engineering. Over the past decades, various optimization techniques have been developed to solve complex optimization problems in different areas especially that are unable to be solved by traditional methods. QUasi-Affine TRansformation Evolutionary (QUATRE) algorithm is a new novel evolution structure for global optimization, which is a swarm based algorithm and use quasi-affine transformation approach for evolution. Nevertheless, there are still some weaknesses in these QUATRE variants. This paper presents a novel E-QUATRE algorithm in which an automatically generated evolution matrix with self-adaptive mechanism and an adaptive control parameter F are proposed for the enhancement of the QUATRE algorithm. Moreover, this paper also discusses the relationship between QUATRE algorithm, Particle Swarm Optimization (PSO) and Differential Evolution (DE) algorithm, all of which are also famous swarm based Stochastic Algorithms (SAs). Algorithm validation is conducted under CEC2013 test suite on single-objective numerical optimization, and E-QUATRE algorithm is compared with several famous Particle Swarm Optimization (PSO) variants, Differential Evolution (DE) variants and QUATRE variants. The experiment results indicate that the proposed E-QUATRE algorithm has a better performance than these swarm based algorithms with fixed population.

Understanding the problem space in single-objective numerical optimization using exploratory landscape analysis

In benchmarking theory, creating a comprehensive and uniformly distributed set of problems is a crucial first step to designing a good benchmark. However, this step is also one of the hardest, as it can be difficult to determine how to evaluate the quality of the chosen problem set.In this article, we evaluate if the field of exploratory landscape analysis can be used to develop a generalized method of visualizing a set of arbitrary optimization functions. We present a method for visually determining the distribution of problems within a benchmark set using exploratory landscape analysis combined with clustering and t-sne visualization, and evaluate and explain the visualization this methodology produces.The proposed method is evaluated on a set of benchmark problems taken from two well known state-of-the-art real-parameter single objective optimization benchmarks: the CEC Special Sessions and Competitions on Real-Parameter Single Objective optimization, and the GECCO Black-Box Optimization Benchmark workshops.The main goal of this paper is to present an analysis of how exploratory landscape analysis can be used to visualize a benchmark problem set. We show that this method can provide a clear visualization of a benchmark problem set and shows the similarities of the problems in it by placing similar problems visually close together. We also show that the problem sets of the above benchmarks have a somewhat distinct set of problems that do not overlap.In addition, by applying feature selection approaches we show that a number of landscape features provided by state-of-the-art exploratory landscape analysis libraries are redundant and that a large amount of them are not invariant to simple transforms like scaling and shifting, at least when analyzing these two datasets.

A covariance matrix adaptation evolution strategy variant and its engineering application

This paper proposes a novel covariance matrix adaptation evolution strategy (CMA-ES) variant, named AEALSCE, for single-objective numerical optimization problems in the continuous domain. To avoid premature convergence and strengthen the exploration capacity of the basic CMA-ES, AEALSCE is obtained by integrating the CMA-ES with two strategies that can adjust the evolutionary directions and enrich the population diversity. The first strategy is named the anisotropic eigenvalue adaptation (AEA) technique, which adapts the search scope towards the optimal evolutionary directions. It scales the eigenvalues of the covariance matrix anisotropically based on local fitness landscape detection. The other strategy is named the local search (LS) strategy, which is executed under the eigen coordinate system and can be subdivided into two parts. In the first part, the new candidates of superior solutions are sampled around the best solution to perform local exploration. In the other part, the new candidates of inferior solutions are generated using a modified mean point along the fitness descent direction. The proposed AEALSCE algorithm is compared with other top competitors, including the CEC 2014 champion, L-SHADE, and the promising NBIPOP-aCMA-ES, by benchmarking the CEC 2014 testbed. Moreover, AEALSCE is applied in solving three constrained engineering design problems and parameter estimation of photovoltaic (PV) models. According to the statistical results of the experiments, our proposed AEALSCE is competitive with other algorithms in convergence efficiency and accuracy. AEALSCE benefits from a good balance of exploration and exploitation, and it exhibits a potential to address real-world optimization problems.

PaDE: An enhanced Differential Evolution algorithm with novel control parameter adaptation schemes for numerical optimization

Differential Evolution (DE) variants have been proven to be excellent algorithms in tackling real-parameter single objective numerical optimization because they have secured the front ranks of these competitions for many years. Nevertheless, there are still some weaknesses, e.g. (1) improper control parameter adaptation schemes; and (2) defect in a given mutation strategy., existing in some state-of-the-art DE variants, which may result in slow convergence and worse optimization performance. Therefore, in this paper, a novel Parameter adaptive DE (PaDE) is proposed to tackle the above mentioned weaknesses and the PaDE algorithm has three advantages: (1) A grouping strategy with novel adaptation scheme for Cr is proposed to tackle the improper adaptation schemes of Cr in some state-of-the-art DE variants; (2) A novel parabolic population size reduction scheme is proposed to tackle the weakness in linear population size reduction scheme; (3) An enhanced time stamp based mutation strategy is proposed to tackle the weakness in a former mutation strategy. The novel PaDE algorithm is verified under 58 benchmarks from two Congress on Evolutionary Computation (CEC) Competition test suites on real-parameter single objective numerical optimization, and experiment results show that the proposed PaDE algorithm is competitive with the other state-of-the-art DE variants.

Some metaheuristics should be simplified

Users have a large and constantly increasing number of optimization metaheuristics at their disposal. In pursuit of ever better approaches, popular and successful methods are often improved a number of times in a row, or various methods are hybridized. One should, however, pay attention to whether such deliberately improved or hybridized methods do not become unnecessarily complicated, or if some of their elements do not introduce a structural bias. In the first case the algorithm should be simplified by eliminating the unneeded elements. This will make it more clear to the users, and, as shown in this study, may even improve the results. In the second case, operators that are responsible for over-frequent sampling of some part of the search space have to be removed.In this study we address the problem of over-complexity of metaheuristics, focusing on two joint winners of the CEC2016 competition on single-objective numerical optimization, l-SHADE-EpSin and UMOEA-II algorithms. It is shown that each of them includes a procedure which introduces a structural bias by attracting population towards the origin O. As discussed in the text, in the case of seven out of 30 benchmark problems considered in the CEC2016 competition the objective function values in the origin O are better than those found by the vast majority of algorithms, hence structural bias affects the results obtained by l-SHADE-EpSin and UMOEA-II. In this work we simplify both algorithms by removing operators that are the main cause of structural bias. Further, we show that in the l-SHADE-EpSin algorithm the Ensemble Sinusoidal adaptation mechanism of control parameter F, that is used during the first half of the search, is not needed. Slightly better results on both artificial benchmarks used in the CEC2016 competition and on a wide set of real-world problems may be obtained if, during that period of the search, the value of F is simply set to 0.5. Such a simplified algorithm is competitive against a large number of metaheuristics and may be much easier for the users to understand.

Adaptive backtracking search optimization algorithm with pattern search for numerical optimization

The backtracking search optimization algorithm (BSA) is one of the most recently proposed population-based evolutionary algorithms for global optimization. Due to its memory ability and simple structure, BSA has powerful capability to find global optimal solutions. However, the algorithm is still insufficient in balancing the exploration and the exploitation. Therefore, an improved adaptive backtracking search optimization algorithm combined with modified Hooke-Jeeves pattern search is proposed for numerical global optimization. It has two main parts: the BSA is used for the exploration phase and the modified pattern search method completes the exploitation phase. In particular, a simple but effective strategy of adapting one of BSA's important control parameters is introduced. The proposed algorithm is compared with standard BSA, three state-of-the-art evolutionary algorithms and three superior algorithms in IEEE Congress on Evolutionary Computation 2014 (IEEE CEC2014) over six widely-used benchmarks and 22 real-parameter single objective numerical optimization benchmarks in IEEE CEC2014. The results of experiment and statistical analysis demonstrate the effectiveness and efficiency of the proposed algorithm.