An accelerated exact distributed first-order algorithm for optimization over directed networks

Abstract

Distributed optimization over networked agents has emerged as an advanced paradigm to address large-scale control, optimization, and signal-processing problems. In the last few years, the distributed first-order gradient methods have witnessed significant progress and enrichment due to the simplicity of using only the first derivatives of local functions. An exact first-order algorithm is developed in this work for distributed optimization over general directed networks with only row-stochastic weighted matrices. It employs the rescaling gradient method to address unbalanced information diffusion among agents, where the weights on the received information can be arbitrarily assigned. Moreover, uncoordinated step-sizes are employed to magnify the autonomy of agents, and an error compensation term and a heavy-ball momentum are incorporated to accelerate convergency. A linear convergence rate is rigorously proven for strongly-convex objective functions with Lipschitz continuous gradients. Explicit upper bounds of step-size and momentum parameter are provided. Finally, simulations illustrate the performance of the proposed algorithm.