Distributed No-Regret Learning for Stochastic Aggregative Games over Networks

Abstract

In this work, we propose a distributed no-regret learning for stochastic aggregative games over time-varying networks. We consider a finite set of players repeatedly playing the network aggregative game in stochastic regimes, where each player has an expectation-valued objective function depending on its own strategy and the aggregate of all player strategies, and the players can estimate gradients of their objective functions up to a zero-mean error with bounded variance. We consider the scenario in which players cannot directly obtain the aggregate value, but they are able to share their estimates of the aggregate with their neighbors without disclosing their own strategies. We design a distributed learning algorithm based on the mirror descent and dynamical averaging tracking. We then provide analysis for both the variational regret and the cost regret for aggregative games with player-specific problem being convex, and show that the expected regret bounds can be $O\left( {\sqrt K } \right)$ for specific step-sizes. These analyses indicate the key correlations between the regret bounds, the network connectivity, and game structures, etc. In addition, we validate the almost sure convergence to the Nash equilibrium for the class of strictly monotone games. Finally, we present preliminary numerics by applying the proposed scheme to the Nash-Cournot competition problem.