Improved version of the McMurtry-Fu reinforcement learning scheme

Abstract

An improved version of the reinforcement scheme originally developed by McMurtry and Fu is presented. A projection procedure as well as a regularizing parameter are introduced to ensure the probability measure and uniqueness of the solution. To prevent degenerate situations where the realization of the function to be optimized is equal to zero, an auxiliary strictly positive regularizing parameter is introduced. A vector representation and a convergence analysis of this multimodal one-dimensional search technique are derived on the basis of the traditional convergence results on Robbins-Monro type of stochastic algorithms. Global maximization and minimization problems are discussed. Finally, some simulation results illustrate the performance and the feasibility of this self-learning optimization algorithm.