Stochastic Nash Equilibrium Seeking for Games with General Nonlinear Payoffs

Abstract

We introduce a multi-input stochastic extremum seeking algorithm to solve the problem of seeking Nash equilibria for a noncooperative game whose N players seek to maximize their individual payoff functions. The payoff functions are general (not necessarily quadratic), and their forms are not known to the players. Our algorithm is a nonmodel-based approach for asymptotic attainment of the Nash equilibria. Different from classical game theory algorithms, where each player employs the knowledge of the functional form of his payoff and the knowledge of the other players' actions, a player employing our algorithm measures only his own payoff values, without knowing the functional form of his or other players' payoff functions. We prove local exponential (in probability) convergence of our algorithms. For nonquadratic payoffs, the convergence is not necessarily perfect but may be biased in proportion to the third derivatives of the payoff functions and the intensity of the stochastic perturbations used in the algorithm. We quantify the size of these residual biases. Compared to the deterministic extremum seeking with sinusoidal perturbation signals, where convergence occurs only if the players use distinct frequencies, in our algorithm each player simply employs an independent ergodic stochastic probing signal in his seeking strategy, which is realistic in noncooperative games. As a special case of an N-player noncooperative game, the problem of standard multivariable optimization (when the players' payoffs coincide) for quadratic maps is also solved using our stochastic extremum seeking algorithm.