A direct search variant of the simulated annealing algorithm for optimization involving continuous variables

Abstract

A memory-based simulated annealing algorithm is proposed which fundamentally differs from the previously developed simulated annealing algorithms for continuous variables by the fact that a set of points rather than a single working point is used. The implementation of the new method does not need differentiability properties of the function being optimized. The method is well tested on a range of problems classified as easy, moderately difficult and difficult. The new algorithm is compared with other simulated annealing methods on both test problems and practical problems. Results showing an improved performance in finding the global minimum are given. Scope and purpose The inherent difficulty of global optimization problems lies in finding the very best optimum (maximum or minimum) from a multitude of local optima. Many practical global optimization problems of continuous variables are non-differentiable and noisy and even the function evaluation may involve simulation of some process. For such optimization problems direct search approaches are the methods of choice. Simulated annealing is a stochastic global optimization algorithm, initially designed for combinatorial (discrete) optimization problems. The algorithm that we propose here is a simulated annealing algorithm for optimization problems involving continuous variables. It is a direct search method. The strengths of the new algorithm are: it does not require differentiability or any other properties of the function being optimized and it is memory-based. Therefore, the algorithm can be applied to noisy and/or not exactly known functions. Although the algorithm is stochastic in nature, it can memorise the best solution. The new simulated annealing algorithm has been shown to be reliable, fast, general purpose and efficient for solving some difficult global optimization problems.