Distributed Learning in Network Games: a Dual Averaging Approach

Abstract

In this paper, we propose a distributed no-regret learning algorithm for network games using a primal-dual method, i.e., dual averaging. With only locally available observations, we consider the scenario where each player optimizes a global objective, formed by local objective functions on the nodes of a given communication graph. Our learning algorithm for each player involves taking steps along their individual payoff gradients, dictated by local observations of the other player’s actions. The output is then projected back-again locally–to the set of admissible actions for each player. We provide the regret analysis of this distributed learning algorithm for the case of a deterministic network that is subjected to two teams with distinct objectives, and obtain an $O(\sqrt T \log (T))$ regret bound. Our analysis indicates the key correlation between the rate of convergence and network connectivity that also appears in the distributed optimization setup via dual averaging. Furthermore, we show that the point of convergence of the proposed algorithm would be a Nash Equilibrium of the game. Finally, we showcase by an illustrative example the performance of our algorithm in relation to the size and connectivity of the network.