Nash, Social and Centralized Solutions to Consensus Problems via Mean Field Control Theory

Abstract

The purpose of this paper is to synthesize initial mean consensus behavior of a set of agents from the fundamental optimization principles of i) stochastic dynamic games, and ii) optimal control. In the stochastic dynamic game model each agent seeks to minimize its individual quadratic discounted cost function involving the mean of the states of all other agents. In this formulation we derive the limiting infinite population mean field equation system and explicitly compute its unique solution. The resulting mean field (MF) control strategies drive each agent to track the overall population's initial state distribution mean, and by applying these decentralized strategies, any finite population system reaches mean consensus asymptotically as time goes to infinity. Furthermore, these control laws possess an ε <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</i> -Nash equilibrium property where ε <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</i> goes to zero as the population size N goes to infinity. Finally, the analysis is extended to the case of random mean field couplings. In the social cooperative formulation the basic objective is to minimize a social cost as the sum of the individual cost functions containing mean field coupling. In this formulation we show that for any individual agent the decentralized mean field social (MF Social) control strategy is the same as the mean field Nash (MF Nash) equilibrium strategy. Hence MF-Nash Controls <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">U</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Nash</sub> <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</i> =MF - Social Controls <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">U</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Soc</sub> <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</i> . On the other hand, the solution to the centralized LQR optimal control formulation yields the Standard Consensus (SC) algorithm whenever the graph representing the corresponding topology of the network is Completely Connected (CC). Hence Cent. LQR Controls <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">U</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Cent</sub> <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</i> =SC-CC Controls <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">U</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">SC</sub> <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</i> . Moreover, a system with centralized control laws reaches consensus on the initial state distribution mean as time and population size N go to infinity. Hence, asymptotically in time MF-Nash Controls <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">U</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Nash</sub> <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</i> =MF-Social Controls <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">U</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Soc</sub> <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</i> = Cent. LQR Controls <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">U</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Cent</sub> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∞</sup> = SC-CC Controls <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">U</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">SC</sub> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∞</sup> . Finally, the analysis is extended to the long time average (LTA) cost functions case.