Analysis of associative reinforcement learning in neural networks using iterated function systems

Abstract

A mathematical theory of associative reinforcement learning in neural networks is developed in terms of random iterated function systems (IFSs), which are finite sets of random maps on metric spaces. In particular, the stochastic search for an associative mapping that maximizes the expected pay-off arising from reinforcement is formulated as a random IFS on weight-space. The dynamical evolution of the weights is described by a Markov process. If this process is ergodic then the limiting behavior of the system is described by an invariant probability measure on weight space that can have a fractal-like structure. A class of associative reinforcement learning algorithms is constructed that is an extension of the nonassociative schemes used in stochastic automata theory. The issue of generalization is discussed within the IFS framework and related to the stochastic and possibly fractal nature of the learning process.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>