Number Of Local Optima Research Articles

A Genetic Programming Based Heuristic to Simplify Rugged Landscapes Exploration

Some optimization problems are difficult to solve due to a considerable number of local optima, which may result in premature convergence of the optimization process. To address this problem, we propose a novel heuristic method for constructing a smooth surrogate model of the original function. The surrogate function is easier to optimize but maintains a fundamental property of the original rugged fitness landscape: the location of the global optimum. To create such a surrogate model, we consider a linear genetic programming approach coupled with a self-tuning fitness function. More specifically, to evaluate the fitness of the produced surrogate functions, we employ Fuzzy Self-Tuning Particle Swarm Optimization, a setting-free version of particle swarm optimization. To assess the performance of the proposed method, we considered a set of benchmark functions characterized by high noise and ruggedness. Moreover, the method is evaluated over different problems’ dimensionalities. The proposed approach reveals its suitability for performing the proposed task. In particular, experimental results confirm its capability to find the global argminimum for all the considered benchmark problems and all the domain dimensions taken into account, thus providing an innovative and promising strategy for dealing with challenging optimization problems. Doi: 10.28991/ESJ-2023-07-04-01 Full Text: PDF

Analyzing randomness effects on the reliability of exploratory landscape analysis

The inherent difficulty of solving a continuous, static, bound-constrained and single-objective black-box optimization problem depends on the characteristics of the problem’s fitness landscape and the algorithm being used. Exploratory landscape analysis (ELA) uses numerical features generated via a sampling process of the search space to describe such characteristics. Despite their success in a number of applications, these features have limitations related with the computational costs associated with generating accurate results. Consequently, only approximations are available in practice which may be unreliable, leading to systemic errors. The overarching aim of this paper is to evaluate the reliability of five well-known ELA feature sets across multiple dimensions and sample sizes. For this purpose, we propose a comprehensive experimental methodology combining exploratory and statistical validation stages, which uses resampling techniques to minimize the sampling cost, and statistical significance tests to identify strengths and weaknesses of individual features. The data resulting from the methodology is collected and made available in the LEarning and OPtimization Archive of Research Data v1.0. The results show that instances of the same function can have feature values that are significantly different; hence, non-generalizable across instances, due to the effects produced by the boundary constraints. In addition, some landscape features under evaluation are highly volatile, and strongly susceptible to changes in sample size. Finally, the results show evidence of a curse of modality, meaning that the sample size should increase with the number of local optima.

Finite element model updating of an 18-story structure using branch-and-bound algorithm with epsilon-constraint

This paper studies the finite element (FE) model updating of an 18-story experimental structure. FE model updating requires solving optimization problems that are generally non-convex and have unknown number of local optima. For such problems, neither randomized local optimization algorithms nor stochastic search algorithms can guarantee global optimality. To obtain the global optimum and improve the accuracy of FE model updating, this paper proposes the branch-and-bound (B&B) algorithm for solving non-convex optimization problems in FE model updating. The paper focuses on the modal property difference formulation that minimizes the difference between experimental and simulated eigenvalues and eigenvectors. We propose a reformulation of the modal property difference approach using epsilon-constraint, to enable the application of the B&B algorithm in FE model updating. The proposed approach is first investigated in simulation and compared with the interior-point method and the genetic algorithm. The model updating results using the B&B algorithm are next validated by the shaking table test data of an 18-story steel frame structure.

PSO-ACSC: a large-scale evolutionary algorithm for image matting

Image matting is an essential image processing technology due to its wide range of applications. Sampling-based image matting is one of the main branches of image matting research that estimates alpha mattes by selecting the best pixel pairs. It is essentially a large-scale multi-peak optimization problem of pixel pairs. Previous study shows that particle swarm optimization (PSO) can effectively optimize the pixel pairs. However, it still suffers from premature convergence problem which often occurs in pixel pair optimization that involves a large number of local optima. To address this problem, this work presents a parameter-free strategy for PSO called adaptive convergence speed controller (ACSC). ACSC monitors and conditionally controls the particles by competitive pixel pair recombination operator (CPPRO) and pixel pair reset operator (PPRO) during the iteration. ACSC performs CPPRO to improve the competitiveness of a particle when the performance of most of the pixel pairs is worse than that of the best-so-far solution. PPRO is performed to avoid premature convergence when the alpha mattes regarding two selected particles are highly similar. Experimental results show that ACSC significantly enhances the performance of PSO for image matting and provides competitive alpha mattes comparing with state-of-the-art evolutionary algorithms.

Homotopic Convex Transformation: A New Landscape Smoothing Method for the Traveling Salesman Problem.

This article proposes a novel landscape smoothing method for the symmetric traveling salesman problem (TSP). We first define the homotopic convex (HC) transformation of a TSP as a convex combination of a well-constructed simple TSP and the original TSP. The simple TSP, called the convex-hull TSP, is constructed by transforming a known local or global optimum. We observe that controlled by the coefficient of the convex combination, with local or global optimum: 1) the landscape of the HC transformed TSP is smoothed in terms that its number of local optima is reduced compared to the original TSP and 2) the fitness distance correlation of the HC transformed TSP is increased. Furthermore, we observe that the smoothing effect of the HC transformation depends highly on the quality of the used optimum. A high-quality optimum leads to a better smoothing effect than a low-quality optimum. We then propose an iterative algorithmic framework in which the proposed HC transformation is combined within a heuristic TSP solver. It works as an escaping scheme from local optima aiming to improve the global searchability of the combined heuristic. Case studies using the 3-Opt and the Lin-Kernighan local search as the heuristic solver show that the resultant algorithms significantly outperform their counterparts and two other smoothing-based TSP heuristic solvers on most of the test instances with up to 20 000 cities.

Surfing on Fitness Landscapes: A Boost on Optimization by Fourier Surrogate Modeling.

Surfing in rough waters is not always as fun as wave riding the “big one”. Similarly, in optimization problems, fitness landscapes with a huge number of local optima make the search for the global optimum a hard and generally annoying game. Computational Intelligence optimization metaheuristics use a set of individuals that “surf” across the fitness landscape, sharing and exploiting pieces of information about local fitness values in a joint effort to find out the global optimum. In this context, we designed surF, a novel surrogate modeling technique that leverages the discrete Fourier transform to generate a smoother, and possibly easier to explore, fitness landscape. The rationale behind this idea is that filtering out the high frequencies of the fitness function and keeping only its partial information (i.e., the low frequencies) can actually be beneficial in the optimization process. We prove our theory by combining surF with a settings free variant of Particle Swarm Optimization (PSO) based on Fuzzy Logic, called Fuzzy Self-Tuning PSO. Specifically, we introduce a new algorithm, named F3ST-PSO, which performs a preliminary exploration on the surrogate model followed by a second optimization using the actual fitness function. We show that F3ST-PSO can lead to improved performances, notably using the same budget of fitness evaluations.

Broken Rotor Bar Fault Detection of Induction Motors Using a Joint Algorithm of Trust Region and Modified Bare-bones Particle Swarm Optimization

A precise detection of the fault feature parameter of motor current is a new research hotspot in the broken rotor bar (BRB) fault diagnosis of induction motors. Discrete Fourier transform (DFT) is the most popular technique in this field, owing to low computation and easy realization. However, its accuracy is often limited by the data window length, spectral leakage, fence effect, etc. Therefore, a new detection method based on a global optimization algorithm is proposed. First, a BRB fault current model and a residual error function are designed to transform the fault parameter detection problem into a nonlinear least-square problem. Because this optimization problem has a great number of local optima and needs to be resolved rapidly and accurately, a joint algorithm (called TR-MBPSO) based on a modified bare-bones particle swarm optimization (BPSO) and trust region (TR) is subsequently proposed. In the TR-MBPSO, a reinitialization strategy of inactive particle is introduced to the BPSO to enhance the swarm diversity and global search ability. Meanwhile, the TR is combined with the modified BPSO to improve convergence speed and accuracy. It also includes a global convergence analysis, whose result proves that the TR-MBPSO can converge to the global optimum with the probability of 1. Both simulations and experiments are conducted, and the results indicate that the proposed detection method not only has high accuracy of parameter estimation with short-time data window, e.g., the magnitude and frequency precision of the fault-related components reaches 10−4, but also overcomes the impacts of spectral leakage and non-integer-period sampling. The proposed research provides a new BRB detection method, which has enough precision to extract the parameters of the fault feature components.

An Asymptotically Optimal Set Approach for Simulation Optimization

We propose an asymptotically optimal set (AOS) approach for solving stochastic optimization problems with discrete or continuous feasible regions. Our AOS approach is a framework for designing provably convergent algorithms that are adaptive in seeking new points and in resampling or discarding already sampled points. The framework is an improvement over the adaptive search with resampling (ASR) method for stochastic optimization in that it spends less effort on inferior points and uses a more robust estimate of the optimal solution. We present conditions guaranteeing that the AOS approach is globally convergent and will eventually discard suboptimal sampled points with probability one, compare the algorithms, and analyze when (additional) resampling (beyond the minimum) is desirable. Our theoretical results show that AOS has stronger performance guarantees than ASR. Our numerical results suggest that AOS makes substantial improvements over ASR, especially for difficult problems with large numbers of local optima. The online supplement is available at https://doi.org/10.1287/ijoc.2018.0811 .

Estimating the number of basins of attraction of multi-objective combinatorial problems

The efficiency of local search is proportional to the number and the distribution of basins of attraction. Often combinatorial optimisation problems have a large number of local optima, uncountable with available computational resources. Approximating the number of basins of attraction and the minimal number of samples for visiting all basins at least once are complex problems for which we assume specific distributions of basins of attraction. We define two types of basins of attraction of multi-objective combinatorial optimisation problems with complementary properties. Acknowledging that each local search run generates a Pareto front of solutions, either each Pareto local solution corresponds to a basin of attraction, or a Pareto basin matches an entire Pareto local front. Simulations compare parametric and non-parametric estimators on bi-objective quadratic assignment problem instances.

Wireless localization for mmWave networks in urban environments

Millimeter wave (mmWave) technology is expected to be a major component of 5G wireless networks. Ultra-wide bandwidths of mmWave signals and the possibility of utilizing large number of antennas at the transmitter and the receiver allow accurate identification of multipath components in temporal and angular domains, making mmWave systems advantageous for localization applications. In this paper, we analyze the performance of a two-step mmWave localization approach that can utilize time-of-arrival, angle-of-arrival, and angle-of-departure from multiple nodes in an urban environment with both line-of-sight (LOS) and non-LOS (NLOS) links. Networks with/without radio-environmental mapping (REM) are considered, where a network with REM is able to localize nearby scatterers. Estimation of a UE location is challenging due to large numbers of local optima in the likelihood function. To address this problem, a gradient-assisted particle filter (GAPF) estimator is proposed to accurately estimate a user equipment (UE) location as well as the locations of nearby scatterers. Monte-Carlo simulations show that the GAPF estimator performance matches the Cramer-Rao bound (CRB). The estimator is also used to create a REM. It is seen that significant localization gains can be achieved by increasing beam directionality or by utilizing REM.

Stopping rule of multi-start local search for structural optimization

Stopping rule for multi-start local search is investigated for application to structural optimization problems with moderately large number of local optima. The size of attractor of an unknown local optimal solution is estimated using the information of already obtained solutions. A stopping rule is defined using the likelihood of obtaining the specific set of local optimal solutions. This way, characteristics of the specific optimization problem is successfully incorporated. The proposed rule is first verified using the mathematical problems in comparison with the existing rule utilizing the estimated ratio of the total size of attractors to the size of feasible domain. The rule is next applied to an optimization problem of a plane frame under constraints on inter-story drift angle and stress against static loads. Characteristics of the distribution of attractors of optimal solutions are also investigated.

Theoretical analysis of dynamic selection of switching auxiliary objectives on XdivK problem

Subject of Research. The paper deals with analysis of the EA+RL method inefficiency reasons on XdivK optimization problem with switching auxiliary objectives. It is proposed to modify the EA+RL method. The EA+RL method increases efficiency of an evolutionary algorithm by introducing auxiliary objectives. The XdivK problem is characterized by a large number of local optima. Switching objectives help to escape from local optima on some stages of optimization while being obstructive on the other stages. Method. To perform theoretical analysis of the EA+RL method and its proposed modification, the corresponding optimization process was modeled by Markov chains. The number of fitness function evaluations needed to reach the optimum was estimated based on the analysis of transition probabilities. Main Results. The EA+RL method and its proposed modification were theoretically analyzed on the XdivK problem with switching auxiliary objectives. It was proved that the proposed modification ignores obstructive objectives contrary to the EA+RL method. The lower and upper bounds on the running time of the proposed modification were obtained. Practical Relevance. The proposed modification increases the efficiency of EA+RL method, successfully used to solve NP-hard optimization problems, such as the test case generation problem.

Adaptive Multimodal Continuous Ant Colony Optimization

Seeking multiple optima simultaneously, which multimodal optimization aims at, has attracted increasing attention but remains challenging. Taking advantage of ant colony optimization (ACO) algorithms in preserving high diversity, this paper intends to extend ACO algorithms to deal with multimodal optimization. First, combined with current niching methods, an adaptive multimodal continuous ACO algorithm is introduced. In this algorithm, an adaptive parameter adjustment is developed, which takes the difference among niches into consideration. Second, to accelerate convergence, a differential evolution mutation operator is alternatively utilized to build base vectors for ants to construct new solutions. Then, to enhance the exploitation, a local search scheme based on Gaussian distribution is self-adaptively performed around the seeds of niches. Together, the proposed algorithm affords a good balance between exploration and exploitation. Extensive experiments on 20 widely used benchmark multimodal functions are conducted to investigate the influence of each algorithmic component and results are compared with several state-of-the-art multimodal algorithms and winners of competitions on multimodal optimization. These comparisons demonstrate the competitive efficiency and effectiveness of the proposed algorithm, especially in dealing with complex problems with high numbers of local optima.

Requirements for papers focusing on new or improved global optimization algorithms

The No-Free Lunch theorem (Wolpert and Macready 1997) provides an important limitation of global optimization algorithms. Improving an algorithm for one class of problem is likely to make it perform more poorly for other problems. This may explain the much larger number of popular global optimizat ion algorithms compared to local optimization algorithms. This means that when we propose a new or improved global optimization algorithm, it should be targeted towards a particular application or set of applications rather than tested against a fixed set of problems. For a paper describing such an algorithm for Structural and Multidisciplinary Optimization, it should target an application including solid or fluid mechanics models that is not solved well by existing algorithms. This has become difficult to demonstrate because of the large number of available global optimization algorithms. Problems where the objective function and constraints are differentiable are not likely to satisfy the above requirement. This is because gradient based local optimizers are very efficient in finding solutions for such problems even with large number of design variables. Unless the number of local optima is very high, multi-start execution of local search techniques is likely to yield a solution more efficiently than a global search algorithm. Problems with discrete or combinatorial design variables are more likely to satisfy the requirement. Finally, many global search algorithms are stochastic in nature. This means that to demonstrate their effectiveness, one must run them multiple times so as to establish the statistical distribution of the solution as function of the design parameters.

Analysing the fitness landscape of search-based software testing problems

Search-based software testing automatically derives test inputs for a software system with the goal of improving various criteria, such as branch coverage. In many cases, evolutionary algorithms are implemented to find near-optimal test suites for software systems. The result of the search is usually received without any indication of how successful the search has been. Fitness landscape characterisation can help understand the search process and its probability of success. In this study, we recorded the information content, negative slope coefficient and the number of improvements during the progress of a genetic algorithm within the EvoSuite framework. Correlating the metrics with the branch and method coverages and the fitness function values reveals that the problem formulation used in EvoSuite could be improved by revising the objective function. It also demonstrates that given the current formulation, the use of crossover has no benefits for the search as the most problematic landscape features are not the number of local optima but the presence of many plateaus.

An Analysis of the Fitness Landscape of Travelling Salesman Problem

The fitness landscape of the travelling salesman problem is investigated for 11 different types of the problem. The types differ in how the distances between cities are generated. Many different properties of the landscape are studied. The properties chosen are all potentially relevant to choosing an appropriate search algorithm. The analysis includes a scaling study of the time to reach a local optimum, the number of local optima, the expected probability of reaching a local optimum as a function of its fitness, the expected fitness found by local search and the best fitness, the probability of reaching a global optimum, the distance between the local optima and the global optimum, the expected fitness as a function of the distance from an optimum, their basins of attraction and a principal component analysis of the local optima. The principal component analysis shows the correlation of the local optima in the component space. We show how the properties of the principal components of the local optima change from one problem type to another.

Quadratic assignment problem: a landscape analysis

The anatomy of the fitness landscape for the quadratic assignment problem is studied in this paper. We study the properties of both random problems, and real-world problems. Using auto-correlation as a measure for the landscape ruggedness, we study the landscape of the problems and show how this property is related to the problem matrices with which the problems are represented. Our main goal in this paper is to study new properties of the fitness landscape, which have not been studied before, and we believe are more capable of reflecting the problem difficulties. Using local search algorithm which exhaustively explore the plateaus and the local optima, we explore the landscape, store all the local optima we find, and study their properties. The properties we study include the time it takes for a local search algorithm to find local optima, the number of local optima, the probability of reaching the global optimum, the expected cost of the local optima around the global optimum and the basin of attraction of the global and local optima. We study the properties for problems of different sizes, and through extrapolations, we show how the properties change with the system size and why the problem becomes harder as the system size grows. In our study we show how the real-world problems are similar to, or different from the random problems. We also try to show what properties of the problem matrices make the landscape of the real problems be different from or similar to one another.

Anatomy of the fitness landscape for dense graph-colouring problem

Graph-colouring is one of the best-known combinatorial optimisation problems. This paper provides a systematic analysis of many properties of the fitness landscape for random instances as a function of both the problem size and the number of colours used. The properties studied include both statistical properties of the bulk of the states, such as the distribution of fitnesses and the auto-correlation, but also properties related to the local optima of the problem. These properties include the mean time to reach the local optima, the number of local optima and the probability of reaching local optima of a given cost, the average distance between global optima and between local optima of a given cost and the closest local optimum, the expected cost as a function of the distance from a configuration and the fitness–distance correlation. Finally, an analysis of how a successful algorithm exploits the fitness distance correlation is carried out.

The Stochastic Simplex Bisection Algorithm

We propose the stochastic simplex bisection algorithm. It randomly selects one from a set of simplexes, bisects it, and replaces it with its two offspring. The selection probability is proportional to a score indicating how promising the simplex is to bisect. We generalize intervals to simplexes, rather than to hyperboxes, as bisection then only requires evaluating the function in one new point, which is somewhat randomized. Using a set of simplexes that partition the search space yields completeness and avoids redundancy. We provide an appropriate scale- and offset-invariant score definition and add an outer loop for handling hyperboxes. Experiments show that the algorithm is capable of exploring vast numbers of local optima, over huge ranges, yet finding the global one. The ease with which it handles quadratic functions makes it ideal for non-linear regression: it is here successfully applied to logistic regression. The algorithm does well, also when the number of function evaluations is severely restricted.

An Analysis of NK Landscapes: Interaction Structure, Statistical Properties, and Expected Number of Local Optima

Simulated landscapes have been used for decades to evaluate search strategies whose goal is to find the landscape location with maximum fitness. Understanding properties of landscapes is important for understanding search difficulty. This paper presents a novel and transparent characterization of $N\!K$ landscapes and derives an analytic expression representing the expected number of local optima. We prove that $N\!K$ landscapes can be represented by parametric linear interaction models where model coefficients have meaningful interpretations. We derive the statistical properties of the model coefficients, providing insight into how the $N\!K$ algorithm parses importance to main effects and interactions. An important insight derived from the linear model representation is that the rank of the linear model defined by the $N\!K$ algorithm is correlated with the number of local optima, a strong determinant of landscape complexity, and search difficulty. We show that the maximal rank for an $N\!K$ landscape is achieved through epistatic interactions that form partially balanced incomplete block designs. Finally, an analytic expression representing the expected number of local optima on the landscape is derived, providing a way to quickly compute the expected number of local optima for very large landscapes.