Online distributed stochastic learning algorithm for convex optimization in time-varying directed networks

Abstract

This paper investigates a online learning optimization problem in a distributed manner, where a set of agents aim to cooperatively minimize the sum of local time-varying cost functions while communications among agents depend on a sequence of time-varying directed graphs. For such a problem, we first propose a online distributed stochastic (sub)gradient push-sum algorithm by utilizing distributed optimization methods and push-sum protocols. Then we analyze the regret bounds for the proposed algorithm on the cases when the cost functions are convex and strongly convex, respectively. The bound on the expected regret for convex functions grows sub-linearly with order of O(T),where T is the time horizon. When the cost functions are strongly convex with Lipschitz gradients, the regret bound has an improved rate with order of O(lnT). Numerical simulations on the localization in wireless sensor networks are used to show the effectiveness of the proposed algorithm.