Nash Certainty Equivalence Principle Research Articles

Risk-sensitive mean field games with major and minor players

We investigate a class of mean field games containing a large number of major and minor players. Each player minimizes a quadratic-tracking type risk-sensitive cost functional, where the reference signal is a function of the state average term of the major and minor players. To reduce the complexity for solving the problem, we design a sequence of decentralized strategies by the Nash certainty equivalence principle. Firstly, for the optimal control problems with quadratic type risk-sensitive cost functionals, we propose a new verification theorem. Secondly, we apply the two-layer state aggregation method to construct the fixed-point equations for the estimations of the state average terms and give the conditions for the existence and uniqueness of the fixed points. Then, we design a sequence of decentralized strategies by the estimations of the state average terms based on local information. It is shown that the estimations of the state average terms are consistent with the true values for the closed-loop systems, and the sequence of strategies designed is a decentralized asymptotic Nash equilibrium. Finally, the effectiveness of the theoretical analysis is demonstrated by a numerical example.

Mixed Mean Field Games with Risk-Sensitive Cost Functionals

We investigate a class of mean field games containing a large number of major and minor players. Each player minimizes a quadratic-tracking type risk-sensitive cost functional, where the reference signal is a function of the state average term of the major and minor players. To reduce the complexity for solving the problem, we design a sequence of decentralized strategies by the Nash certainty equivalence principle. Firstly, we apply the two-layer state aggregation method to construct the fixed-point equations for the estimations of the state average terms and give the conditions for the existence and uniqueness of the fixed points. Then, we design a sequence of decentralized strategies by the estimations of the state average terms based on local information. It is shown that the estimations of the state average terms are consistent with the true values for the closed-loop systems, and the sequence of strategies designed is a decentralized asymptotic Nash equilibrium.

Adaptive Tracking Games for Coupled Stochastic Linear Multi-Agent Systems: Stability, Optimality and Robustness

Distributed adaptive tracking-type games are investigated for a class of coupled stochastic linear multi-agent systems with uncertainties of unknown structure parameters, external stochastic disturbances, unmodeled dynamics, and unknown agents' interactions. The control goal is to make the states of all the agents converge to a desired function of the population state average (PSA). Due to the fact that only local information is available for each agent, the control is distributed. For the time-invariant parameter case, the extended least-squares algorithm, Nash certainty equivalence (NCE) principle, and certainty equivalence (CE) principle are used to estimate the unknown parameters and the PSA term, and to design adaptive control, respectively. Under some mild conditions, it is shown that the closed-loop system is almost surely uniformly stable with respect to the population number N; the estimate for the PSA term is strongly consistent; the adaptive control is almost surely an asymptotic Nash equilibrium. When the dynamics of each agent contains time-varying parameters and unmodeled dynamics, the projected least mean square (LMS) algorithm, NCE principle, and CE principle are adopted to estimate the unknown time-varying parameters, and the unknown PSA term, and to design robust adaptive control, respectively. In addition to stability of the closed-loop system and consistency of the PSA estimate, the control law is shown to be robust Nash equilibrium with respect to the unmodeled dynamics, the variation of the unknown parameters, and the external disturbances. Two numerical examples are given to illustrate the methods and results of this paper.

Adaptive Mean Field Games for Large Population Coupled ARX Systems with Unknown Coupling Strength

This paper is concerned with decentralized tracking-type games for large population multi-agent systems with mean-field coupling. The individual dynamics are described by stochastic discrete-time auto-regressive models with exogenous inputs (ARX models), and coupled by terms of the unknown population state average (PSA) with unknown coupling strength. A two-level decentralized adaptive control law is designed. On the high level, the PSA is estimated based on the Nash certainty equivalence (NCE) principle. On the low level, the coupling strength is identified based on decentralized least squares algorithms and the estimate of the PSA. The decentralized control law is constructed by combining the NCE principle and Certainty equivalence (CE) principle. By probability limit theory, under mild conditions, it is shown that: (a) the closed-loop system is stable almost surely; (b) as the number of agents increases to infinity, the estimates of both the PSA and the coupling strength are asymptotically strongly consistent and the decentralized control law is an almost sure asymptotic Nash-equilibrium.

A Multi-Agent System for Controlled Charging of a Large Population of Electric Vehicles

The share of electric vehicles (EVs) is expected to increase significantly in the vehicle market sales in the forthcoming years. Uncontrolled charging of EVs can affect adversely the normal operation of the power system, especially at the distribution level. This paper proposes a distributed, multi-agent EV charging control method based on the Nash Certainty Equivalence Principle that considers network impacts. Convergence of the method when the EVs control agents are uncoupled and weakly-coupled is discussed. The efficiency of the proposed management system is evaluated through simulation of a realistic, urban distribution network.

Large-Population LQG Games Involving a Major Player: The Nash Certainty Equivalence Principle

We consider linear-quadratic-Gaussian (LQG) games with a major player and a large number of minor players. The major player has a significant influence on others. The minor players individually hav...

Large-Population LQG Games Involving a Major Player: The Nash Certainty Equivalence Principle

We consider linear-quadratic-Gaussian (LQG) games with a major player and a large number of minor players. The major player has a significant influence on others. The minor players individually have negligible impact, but they collectively contribute mean field coupling terms in the individual dynamics and costs. To overcome the dimensionality difficulty and obtain decentralized strategies, the so-called Nash certainty equivalence methodology is applied. The control synthesis is preceded by a state space augmentation via a set of aggregate quantities giving the mean field approximation. Subsequently, within the population limit the LQG game is decomposed into a family of limiting two-player games as each is locally seen by a representative minor player. Next, when solving these limiting two-player games, we impose certain interaction consistency conditions such that the aggregate quantities initially assumed coincide with the ones replicated by the closed loop of a large number of minor players. This procedure leads to a set of decentralized strategies for the original LQG game, which is an $\varepsilon$-Nash equilibrium.

Decentralized tracking-type games for multi-agent systems with coupled ARX models: Asymptotic Nash equilibria

A class of decentralized tracking-type games is considered for large population multi-agent systems (MAS). The agents are described by stochastic discrete-time auto-regressive models with exogenous inputs (ARX models), and coupled together through their individual dynamics and performance indexes by terms of the unknown population state average (PSA). The performance index of each agent to minimize is a stochastic long term averaged group-tracking-type functional, in which there is a nonlinear term of the unknown PSA. The control law is decentralized and implemented via the Nash certainty equivalence principle. By probability limit theory, under mild conditions it is shown that: (a) the estimate of the PSA is strongly consistent; (b) the closed-loop system is stable almost surely, and the stability is independent of the number N of agents; (c) the decentralized control law is an asymptotic Nash equilibrium almost surely or in probability according to the property of the nonlinear coupling function in the performance indexes.

Large population stochastic dynamic games: closed-loop McKean-Vlasov systems and the Nash certainty equivalence principle

We consider stochastic dynamic games in large population conditions where multiclass agents are weakly coupled via their individual dynamics and costs. We approach this large population game problem by the so-called Nash Certainty Equivalence (NCE) Principle which leads to a decentralized control synthesis. The McKean-Vlasov NCE method presented in this paper has a close connection with the statistical physics of large particle systems: both identify a consistency relationship between the individual agent (or particle) at the microscopic level and the mass of individuals (or particles) at the macroscopic level. The overall game is decomposed into (i) an optimal control problem whose Hamilton-Jacobi-Bellman (HJB) equation determines the optimal control for each individual and which involves a measure corresponding to the mass effect, and (ii) a family of McKean-Vlasov (M-V) equations which also depend upon this measure. We designate the NCE Principle as the property that the resulting scheme is consistent (or soluble), i.e. the prescribed control laws produce sample paths which produce the mass effect measure. By construction, the overall closed-loop behaviour is such that each agent’s behaviour is optimal with respect to all other agents in the game theoretic Nash sense.