Stochastic adaptive Nash Certainty Equivalence control: Population parameter distribution estimation

Abstract

For noncooperative games the Nash Certainty Equivalence (NCE), or Mean Field (MF) methodology developed in previous work provides decentralized strategies which asymptotically yield Nash equilibria. The NCE (MF) control laws use only the local information of each agent on its own state evolution and knowledge of its own dynamical parameters, while the behaviour of the mass is precomputable from knowledge of the distribution of dynamical parameters throughout the mass population. Relaxing the a priori information condition introduces the methods of parameter estimation and stochastic adaptive control (SAC) into MF control theory. An initial problem on this path is that where each agent needs to estimate (i) its own dynamical parameters and (ii) the distribution of the population's dynamical parameters. In the present work, each agent observes a random subset of the population of agents. Each agent estimates its own dynamical parameters via the recursive weighted least squares (RWLS) algorithm and the distribution of the population's dynamical parameters via the maximum likelihood estimation (MLE). Under reasonable conditions on the population dynamical parameter distribution, we establish: (i) the strong consistency of the self-parameter estimates and the weak convergence of the distribution function; and that (ii) all agent systems are long run average L <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> stable; (iii) the set of controls yields a (strong) ϵ-Nash equilibrium for all e; and (iv) in the population limit the long run average cost obtained is equal to the non-adaptive long run average cost.