Stochastic approximation with dependent disturbances—I

Abstract

A generalization of Robbins-Monro stochastic approximation is presented in the paper. It is shown that, if disturbances satisfy a sort of generalized law of large numbers then an appropriate stochastic approximation procedure converges almost surely or only in probability, depending on what kind of law of large numbers (strong or weak) is satisfied by disturbances. In that sense theorems presented in the paper generalize Robbins-Monro stochastic approximation schemes, because the law of large numbers can be satisfied, as is well-known, by sequences of dependent random variables. On the other hand, as theorem of Gladishev (a generalized version of Robbins-Monro theorem) can be obtained from the results presented in the paper (see Theorem 10), one can consider this paper as the one providing new proofs for different versions of stochastic approximation. The proofs of the theorems of the paper are different than usual proofs of stochastic approximation procedure. In particular, they are not based on the Martingale convergence theorem. Roughly speaking the proofs exploit the analogy between the stochastic approximation procedures of Robbins-Monro versions and deterministic numerical iterative procedures seeking zeros of the system of nonliner equations. As the results of the paper were thought to be applied to estimation of parameters of discrete stochastic processes (so called identification) special notation has been introduced. This notation is believed to be useful for the above purpose.