A Markovian Incremental Stochastic Subgradient Algorithm

Abstract

In this article, a stochastic incremental subgradient algorithm for the minimization of a sum of convex functions is introduced. The method sequentially uses partial subgradient information, and the sequence of partial subgradients is determined by a general Markov chain. This makes it suitable to be used in networks, where the path of information flow is stochastically selected. We prove convergence of the algorithm to a weighted objective function, where the weights are given by the Cesàro limiting probability distribution of the Markov chain. Unlike previous works in the literature, the Cesàro limiting distribution is general (not necessarily uniform), allowing for general weighted objective functions and flexibility in the method.