Applications of a Kushner and Clark lemma to general classes of stochastic algorithms

Abstract

Two general classes of stochastic algorithms are considered, including algorithms considered by Ljung as well as algorithms of the form \theta_{n+1} = \theta_{n} - \gamma_{n+1} V_{n+1}(\theta_{n}, Z) , where Z is a stationary ergodic process. It is shown how one can apply a lemma of Kushner and Clark to obtain properties of these algorithms. This is done by using in particular Martingale arguments in the generalized Ljung case. In these various situations the convergence is obtained by the method of the associated ordinary differential equation, under the classical boundedness assumptions. In the case of linear algorithms, the boundedness assumptions are dropped.