Stochastic Approximation Based Consensus Dynamics over Markovian Networks

Abstract

This paper considers consensus problems with random networks. A key object of our analysis is a sequence of stochastic matrices which involve Markovian switches and decreasing step sizes. We establish ergodicity of the backward products of these stochastic matrices. The basic technique is to consider the second moment dynamics of an associated Markovian jump linear system and exploit its two-scale interaction property resulting from the decreasing step sizes. The mean square convergence rate of the backward products is also obtained. The ergodicity results are used to prove mean square consensus of stochastic approximation algorithms where agents collect noisy information. The approach is further applied to a token scheduled averaging model.