Continuous Function Optimization Research Articles

Massively parallel differential evolution—pattern search optimization with graphics hardware acceleration: an investigation on bound constrained optimization problems

This paper presents a novel parallel Differential Evolution (DE) algorithm with local search for solving function optimization problems, utilizing graphics hardware acceleration. As a population-based meta-heuristic, DE was originally designed for continuous function optimization. Graphics Processing Units (GPU) computing is an emerging desktop parallel computing technology that is becoming popular with its wide availability in many personal computers. In this paper, the classical DE was adapted in the data-parallel CPU-GPU heterogeneous computing platform featuring Single Instruction-Multiple Thread (SIMT) execution. The global optimal search of the DE was enhanced by the classical local Pattern Search (PS) method. The hybrid DE---PS method was implemented in the GPU environment and compared to a similar implementation in the common computing environment with a Central Processing Unit (CPU). Computational results indicate that the GPU-accelerated SIMT-DE-PS method is orders of magnitude faster than the corresponding CPU implementation. The main contribution of this paper is the parallelization analysis and performance analysis of the hybrid DE---PS with GPU acceleration. The research results demonstrate a promising direction for high speed optimization with desktop parallel computing on a personal computer.

Discrete optimum design of truss structures using artificial bee colony algorithm

Over the past few years, swarm intelligence based optimization techniques such as ant colony optimization and particle swarm optimization have received considerable attention from engineering researchers and practitioners. These algorithms have been used in the solution of various engineering problems. Recently, a relatively new swarm based optimization algorithm called the Artificial Bee Colony (ABC) algorithm has begun to attract interest from researchers to solve optimization problems. The aim of this study is to present an optimization algorithm based on the ABC algorithm for the discrete optimum design of truss structures. The ABC algorithm is a meta-heuristic optimization technique that mimics the process of food foraging of honeybees. Originally the ABC algorithm was developed for continuous function optimization problems. This paper describes the modifications made to the ABC algorithm in order to solve discrete optimization problems and to improve the algorithm's performance. In order to demonstrate the effectiveness of the modified algorithm, four structural problems with up to 582 truss members and 29 design variables were solved and the results were compared with those obtained using other well-known meta-heuristic search techniques. The results demonstrate that the ABC algorithm is very effective and robust for the discrete optimization designs of truss structural problems.

An improved shuffled frog-leaping algorithm with extremal optimisation for continuous optimisation

Several types of evolutionary computing methods are documented in the literature and are well known for solving unconstrained optimisation problems. This paper proposes a hybrid scheme that combines the merits of a global search algorithm, the shuffled frog-leaping algorithm (SFLA) and local exploration, extremal optimisation (EO) and that exhibits strong robustness and fast convergence for high-dimensional continuous function optimisation. A modified shuffled frog-leaping algorithm (MSFLA) is investigated that improves the leaping rule by properly extending the leaping step size and adding a leaping inertia component to account for social behaviour. To further improve the local search ability of MSFLA and speed up convergence, we occasionally introduce EO, which has an excellent local exploration capability, in the local exploration process of the MSFLA. It is characterised by alternating the coarse-grained Cauchy mutation and the fine-grained Gaussian mutation. Compared with standard particle swarm optimisation (PSO), SFLA and MSFLA for six widely used benchmark examples, the hybrid MSFLA-EO is shown to be a good and robust choice for solving high-dimensional continuous function optimisation problems. It possesses excellent performance in terms of the mean function values, the success rate and the fitness function evaluations (FFE), which is a rough measure of the complexity of the algorithm.

Learning hybridization strategies in evolutionary algorithms

Evolutionary Algorithms are powerful optimization techniques which have been applied to many different problems, from complex mathematical functions to real-world applications. Some studies report performance improvements through the combination of different evolutionary approaches within the same hybrid algorithm. However, the mechanisms used to control this combination of evolutionary approaches are not as satisfactory as would be desirable. In most cases, there is no feedback from the algorithm nor any regulatory component that modifies the participation of each evolutionary approach in the overall search process. In some cases, the algorithm makes use of some information for an on-line adaptation of the participation of each algorithm. In this paper, the use of Reinforcement Learning (RL) is proposed as a mechanism to control how the different evolutionary approaches contribute to the overall search process. In particular, three learning policies based on one of the state-of-the-art RL algorithms, Q-Learning, have been considered and used to control the participation of each algorithm by learning the best-response mixed strategy. To test this approach, a benchmark made up of six large-scale (500 dimensions) continuous optimization functions has been considered. The experimentation carried out has proved that RL control mechanisms successfully learn optimal patterns for the combination of Evolutionary Algorithms in most of the proposed functions, being able to improve the performance of both individual and non RL hybrid algorithms.

Study of confinements in the particle swarm optimisation for application to the nuclear reactor reload problem

Particle Swarm Optimisation (PSO) is a metaheuristic technique based on the social aspects of intelligence. Some PSO models have been developed for combinatorial optimisation, although none of them presented satisfactory results to optimise the combinatorial problem of the Nuclear Reactor Reload Problem (NRRP). The Particle Swarm Optimisation with Random Keys (PSORK) model is a variant of the PSO applied to combinatorial problems such as the NRRP. In this paper, we present the results of the in-core fuel optimisation of the Angra 1 Nuclear Power Plant (NPP) located at the southeast of Brazil and a survey on the confinement of particles in the PSORK for the NRRP. A confinement analysis is interesting in a continuous function optimisation since it may influence the search, resulting in biases that favour particular regions of the search space. Nevertheless, there are no similar studies for confinements applied to a combinatorial optimisation. We have submitted the PSORK to a confinement analysis when applied to the Travelling Salesman Problem (TSP) Rykel48 (ry48p), a benchmark for a combinatorial optimisation, in order to study the consequences of the confined PSORK in this type of combinatorial problem. Finally, the Confinement for the Combinatorial Optimisation (CCO) of the NRRP is proposed and the results are presented.

StPSO: Strengthened particle swarm optimization

In this paper, we present a novel approach to strengthen Particle Swarm Optimization (PSO). PSO is a population-based metaheuristic that takes advantage of individual memory and social cooperation in a swarm. It has been applied to a variety of optimization problems because of its simplicity and fast convergence. However, straightforward application of PSO suffers from premature convergence and lack of intensification around the local best locations. To rectify these problems, we modify update procedure for the best particle in the swarm and propose a simple and random moving strategy. We perform a Reduced Variable Neighborhood Search (RVNS) based local search around the particle, as well. The resulting strengthened PSO (StPSO) algorithm not only has superior exploration and exploitation mechanisms but also provides a dynamical balance between them. Experimental analysis of StPSO is performed on continuous function optimization problems and a discrete problem, Orienteering Problem. Its performance is quite robust and consistent for all problem types; discrete or continuous, unimodal or multimodal. StPSO either reproduces the best known solution or provides a competitive solution for each problem instance. So, it is a valuable tool producing promising solutions for all problem types.

A Lyapunov-Based Extension to Particle Swarm Dynamics for Continuous Function Optimization

The paper proposes three alternative extensions to the classical global-best particle swarm optimization dynamics, and compares their relative performance with the standard particle swarm algorithm. The first extension, which readily follows from the well-known Lyapunov's stability theorem, provides a mathematical basis of the particle dynamics with a guaranteed convergence at an optimum. The inclusion of local and global attractors to this dynamics leads to faster convergence speed and better accuracy than the classical one. The second extension augments the velocity adaptation equation by a negative randomly weighted positional term of individual particle, while the third extension considers the negative positional term in place of the inertial term. Computer simulations further reveal that the last two extensions outperform both the classical and the first extension in terms of convergence speed and accuracy.

Wavelet chaotic neural networks and their application to continuous function optimization

Neural networks have been shown to be pow-erful tools for solving optimization problems. In this paper, we first retrospect Chen’s chaotic neural network and then propose several novel chaotic neural networks. Second, we plot the figures of the state bifurcation and the time evolution of most positive Lyapunov exponent. Third, we apply all of them to search global minima of continuous functions, and respec-tively plot their time evolution figures of most positive Lyapunov exponent and energy func-tion. At last, we make an analysis of the per-formance of these chaotic neural networks.

Optimization of multimodal continuous functions using a new crossover for the real-coded genetic algorithms

In this study, a new crossover approach to the real-coded genetic algorithm is proposed. The approach is simply based on efficiently tuned real-coded crossover operation using the probability distribution function of Gauss distribution to generate rather dissimilar strings which may be candidates of possible solutions. Also linear and quadratic mapping algorithms comparatively used both to constrain individuals in the given search spaces and to produce different individuals in order to increase average fitness relatively for the same population. Moreover, to refine genetically found optimum points the local search technique based on Newton’s method was performed. The designed software was first implemented on 11 well-known test functions and their results were compared with previous findings as shown in tables. In few test functions, the elitism operator was put into effect to maintain fitness stability helping increase the search performance of the proposed algorithm. The results indicate that the solutions to the test functions were almost the same with theoretical ones and the number of function evaluations for each test function was less than that obtained from using previous approaches.

Improving genetic algorithms’ performance by local search for continuous function optimization

The genetic algorithms (GAs) can be used as a global optimization tool for continuous and discrete functions problems. However, a simple GA may suffer from slow convergence, and instability of results. GAs’ problem solution power can be increased by local searching. In this study a new local random search algorithm based on GAs is suggested in order to reach a quick and closer result to the optimum solution.

A lower bound on complexity of optimization on the Wiener space

This paper is a study of the complexity of optimization of continuous univariate functions using a fixed number of sequentially selected function evaluations. The complexity is studied in the average case under a conditioned Wiener measure. We show that to obtain an error of at most ϵ , on the order of log log ( 1 / ϵ ) log ( 1 / ϵ ) function evaluations are required.

An efficient algorithm for large scale global optimization of continuous functions

A fast descent algorithm, resorting to a “stretching” function technique and built on one hybrid method (GRSA) which combines simulated annealing (SA) algorithm and gradient based methods for large scale global optimizations, is proposed. Unlike the previously proposed method in which the original objective functions remain unchanged during the whole course of optimization, the new method firstly constructs an auxiliary function on one local minimizer obtained by gradient based methods and then SA is executed on this constructed auxiliary function instead of on the original objective function in order that we can improve the jumping ability of SA algorithm to escape from the currently discovered local minimum to a better one from which the gradient based methods restart a new local search. The above procedure is repeated until a global minimum is detected. In addition, corresponding to the adopted “stretching” technique, a new next trial point generating scheme is designed. It is verified by simulation especially on large scale problems that the convergence speed is greatly accelerated, which is its main difference from many other reported methods that mostly cope with functions with less than 50 variables and does not apply to large scale optimization problems. Furthermore, the new algorithm functions as a global optimization procedure with a high success probability and high solution precision.

Particle swarm and ant colony algorithms hybridized for improved continuous optimization

This paper proposes PSACO (particle swarm ant colony optimization) algorithm for highly non-convex optimization problems. Both particle swarm optimization (PSO) and ant colony optimization (ACO) are co-operative, population-based global search swarm intelligence metaheuristics. PSO is inspired by social behavior of bird flocking or fish schooling, while ACO imitates foraging behavior of real life ants. In this study, we explore a simple pheromone-guided mechanism to improve the performance of PSO method for optimization of multimodal continuous functions. The proposed PSACO algorithm is tested on several benchmark functions from the usual literature. Numerical results comparisons with different metaheuristics demonstrate the effectiveness and efficiency of the proposed PSACO method.

Stochastic optimal growth with bounded or unbounded utility and with bounded or unbounded shocks

This paper studies a one-sector stochastic optimal growth model with i.i.d. productivity shocks in which utility is allowed to be bounded or unbounded, the shocks are allowed to be bounded or unbounded, and the production function is not required to satisfy the Inada conditions at zero and infinity. Our main results are three-fold. First, we confirm the Euler equation as well as the existence of a continuous optimal policy function under a minimal set of assumptions. Second, we establish the existence of an invariant distribution under quite general assumptions. Third, we show that the density of optimal output converges to a unique invariant density independently of initial output under the assumption that the shock distribution has a density whose support is an interval, bounded or unbounded. In addition, we provide existence and stability results for general one-dimensional Markov processes.

Non-convex global optimization by the beta algorithm: A MAPLE code

A variant of the beta algorithm based on cubic algorithm (Math. Comput. Modelling 38 (2003) 77–97) is presented for global optimization of continuous functions over general compact robust sets. This is a set-monotonic algorithm over non-convex, disconnected set not satisfying any qualification constraint other than being compact and robust. On this basis, a MAPLE code is developed for full global optimization of functions of n variables. The code does not create ill-conditioned situations. Graphics are included, and the solution set can be visualized in projections on coordinate planes. The code is ready for engineering applications. Results of numerical experiments are presented, with graphs, to illustrate the use of the code.

1103 分散確率モデル遺伝的アルゴリズムにおける解探索能力についての検討(OS22 最適化計算法)

Distributed Probabilistic Model-building Genetic Algorithms (DPMBGAs) are a new type of Genetic Algorithm. In the DPMBGA, when the offspring are generated, Principal Component Analysis (PCA) considers the correlations among the design variables. Moreover, this model applies the island model to maintain population diversity. The effectiveness of DPMBGA has been demonstrated through optimization of continuous functions. This paper describes the effectiveness of the parameters in the DPMBGA. Experiments indicated that these parameters are important factor for maintenance of the diversity of the population.

A learning automata based algorithm for optimization of continuous complex functions

This paper presents a new method for optimizing continuous complex functions based on a learning automaton. This method can be considered as active learning permitting to select on-line the most significant data samples in order to quickly converge to a quasi global optimum of the functions to be optimized with a fewer number of tests or calculations. Like other stochastic optimization algorithms, it aims at finding a compromise between exploitation and exploration, i.e. converging to the nearest local optima and exploring the function behavior in order to discover global optimal regions. During the optimization procedure, this method enhances local search in interesting regions or intervals and reduces the whole searching space by removing useless regions or intervals.

Local Function Approximation in Evolutionary Algorithms for the Optimization of Costly Functions

We develop an approach for the optimization of continuous costly functions that uses a space-filling experimental design and local function approximation to reduce the number of function evaluations in an evolutionary algorithm. Our approach is to estimate the objective function value of an offspring by fitting a function approximation model over the k nearest previously evaluated points, where k=(d+1)(d+2)/2 and d is the dimension of the problem. The estimated function values are used to screen offspring to identify the most promising ones for function evaluation. To fit function approximation models, a symmetric Latin hypercube design (SLHD) is used to determine initial points for function evaluation. We compared the performance of an evolution strategy (ES) with local quadratic approximation, an ES with local cubic radial basis function (RBF) interpolation, an ES whose initial parent population comes from an SLHD, and a conventional ES. These algorithms were applied to a twelve-dimensional (12-D) groundwater bioremediation problem involving a complex nonlinear finite-element simulation model. The performances of these algorithms were also compared on the Dixon-Szego test functions and on the ten-dimensional (10-D) Rastrigin and Ackley test functions. All comparisons involve analysis of variance (ANOVA) and the computation of simultaneous confidence intervals. The results indicate that ES algorithms with local approximation were significantly better than conventional ES algorithms and ES algorithms initialized by SLHDs on all Dixon-Szego test functions except for Goldstein-Price. However, for the more difficult 10-D and 12-D functions, only the cubic RBF approach was successful in improving the performance of an ES. Moreover, the results also suggest that the cubic RBF approach is superior to the quadratic approximation approach on all test functions and the difference in performance is statistically significant for all test functions with dimension d/spl ges/4.

Cubic algorithm for global optimization with box and equality constraints and application to optimal allocation of resources

C++ code of the cubic algorithm [1] is proposed for nonconvex global optimization of continuous functions with box and equality constraints, and for the solution of nonlinear systems of equations, including underdetermined and overdetermined systems. The algorithm is upgraded for application to problems with linear and nonlinear equality constraints usually encountered in optimal resource allocation, investment planning, construction projects, and many engineering applications. It is further extended for visualization analysis of high dimensional sets of global optimizers by means of projections on linear subspaces and on arbitrary planes. C++ windows are presented for input, output, algorithmic control, and special research possibilities including graphics and screen display. The code does not create ill-conditioned situations.

Continuous interacting ant colony algorithm based on dense heterarchy

Ant colony algorithms are a class of metaheuristics which are inspired from the behavior of real ants. The original idea consisted in simulating the stigmergic communication, therefore these algorithms are considered as a form of adaptive memory programming. A new formalization is proposed for the design of ant colony algorithms, introducing the biological notions of heterarchy and communication channels. We are interested in the way ant colonies handle the information. According to these issues, a heterarchical algorithm called “Continuous Interacting Ant Colony” (CIAC) is designed for the optimization of multiminima continuous functions. CIAC uses two communication channels showing the properties of trail and direct communications. CIAC presents interesting emergent properties as it was shown through some analytical test functions.