Recursive algorithms, urn processes and chaining number of chain recurrent sets

Abstract

This paper investigates the dynamical properties of a class of urn processes and recursive stochastic algorithms with constant gain which arise frequently in control, pattern recognition, learning theory, and elsewhere. It is shown that, under suitable conditions, invariant measures of the process tend to concentrate on the Birkhoff center of irreducible (i.e. chain transitive) attractors of some vector field $F: {\Bbb R}^d \rightarrow {\Bbb R}^d$ obtained by averaging. Applications are given to simple situations including the cases where $F$ is Axiom A or Morse–Smale, $F$ is gradient-like, $F$ is a planar vector field, $F$ has finitely many alpha and omega limit sets.