Distributed delayed dual averaging for distributed optimization over time-varying digraphs

Abstract

In this paper, a push-sum based distributed delayed dual averaging algorithm (PS-DDDA) is proposed to solve the distributed constrained optimization problem over the time-varying unbalanced directed graph (digraph). It considers the scenario in which each agent has delays while calculating the gradients and communicating. Both delays are assumed to be time-varying but bounded, which are more common in practice. Furthermore, we demonstrate that the cost function at the local state average converges to the optimal value and the local state average converges to the consensus with a rate of O (1/T) by utilizing the diminishing step size, where T is the total number of iterations. Moreover, to solve the distributed online constrained optimization problem, we propose the online version of PS-DDDA, and prove that its regret bound increases sublinearly with a rate of O (T). Finally, the effectiveness of the proposed algorithms is corroborated by solving logistic regression problems.