Complexity Analysis Integrating Pure Adaptive Search (PAS) and Pure Random Search (PRS)

Abstract

Pure adaptive search (PAS) is a random search algorithm for global optimization that has promising complexity results. The complexity of pure adaptive search has been analyzed for both continuous and discrete, finite global optimization. Unfortunately, it is not possible at this time to implement pure adaptive search and achieve the ideal computational performance. To model practical random search algorithms more closely, we extend the complexity analysis to integrate pure adaptive search with pure random search. Many practical algorithms have some probability of sampling in the improving region, which is analogous to sampling according to PAS, and a probability of sampling outside the improving region, which is analogous to sampling according to PRS. Simulated annealing also has a probability of accepting a non-improving point. A Markov chain analysis is used to determine the expected number of iterations required to find the global optimum and to provide bounds for the expected number of iterations needed for a combination of PAS and PRS with acceptance probability. The analysis shows that one needs only a small probability of sampling in the improving region in order to dramatically improve performance.