Multi-cluster distributed optimization via random sleep strategy

Abstract

In this paper, the distributed optimization problem over multi-cluster networks is considered. Different from the existing works, this paper studies the optimization algorithm under uncoordinated step sizes. More specifically, by combining a random sleep strategy and the round-robin communication among clusters, a new hierarchical algorithm is developed to solve the considered problem. In the proposed algorithm, the random sleep strategy enables each agent to independently choose either performing the projected subgradient descent, or keeping the previous estimate by a Bernoulli decision, based on which the step size of each agent is selected as an uncoordinated form that only relates to the independent Bernoulli decision variable. Technically, by introducing a key definition on the algorithm history, it is proven that the estimates of the proposed algorithm can converge to the optimal solution even with the uncoordinated step sizes. In addition, we also study the convergence performance of the proposed algorithm with simpler constant step sizes. In this case, it is proven that the random sleep strategy can efficiently improve the convergence accuracy of the algorithm. Finally, the theoretical findings are verified via simulation examples.