Asymptotic Properties of Distributed and Communicating Stochastic Approximation Algorithms

Abstract

The asymptotic properties of extensions of the type of distributed or decentralized stochastic approximation proposed in [1] are developed. Such algorithms have numerous potential applications in decentralized estimation, detection and adaptive control, or in decentralized Monte Carlo simulation for system optimization (where they can exploit the possibilities of parallel processing). The structure involves several isolated processors (recursive algorithms) that communicate to each other asynchronously and at random intervals. The asymptotic (small gain) properties are derived. The communication intervals need not be strictly bounded, and they and the system noise can depend on the (communicating) system state. State space constraints are also handled. In many applications, the dynamical terms are merely indicator functions, or have other types of discontinuities. The “typical” such case is also treated, as is the case where there is noise in the communication. The linear stochastic differential equation ...