Randomized Gradient-Free Distributed Online Optimization with Time-Varying Cost Functions

Abstract

This paper considers a distributed online optimization problem in a multi-agent system, where the local cost functions of agents are time-varying. The value of the local cost function is only known to the local agent after the decision is made at each time-step. The objective of this multi-agent system is to collaboratively solve the problem by exchanging the information with the neighbors. An online randomized gradient-free distributed projected gradient descent (oRGF-DPGD) method is proposed, in which a local randomized gradient-free oracle is built locally to estimate the gradient in a random direction. Due to the time-varying setting of the cost functions, the optimal solution of the distributed optimization problem at each time-step is changing, which makes the analysis on the performance of the algorithm different from static distributed optimization problems. Hence, the concept of regret is introduced, which characterizes the gap between the total costs incurred by the agent’s actual state trajectory and the best fixed offline centralized optimal solution. With the proposed algorithm, we claim that the decision variable maintained by each agent is able to converge to the same trajectory, while its associated regret is bounded by a sublinear function of the time duration T . Specifically, by averaging the regret over the time duration, we obtain the approximate convergence to a small neighborhood of zero at a rate of $\mathcal{O}(1/\sqrt T )$ when the step-size at each time-step t is set to $1/\sqrt {t + 1} $.