Asymptotic Nash Equilibrium Research Articles

Contagion risks and security investment in directed networks.

We develop a model for contagion risks and optimal security investment in a directed network of interconnected agents with heterogeneous degrees, loss functions, and security profiles. Our model generalizes several contagion models in the literature, particularly the independent cascade model and the linear threshold model. We state various limit theorems on the final size of infected agents in the case of random networks with given vertex degrees for finite and infinite-variance degree distributions. The results allow us to derive a resilience condition for the network in response to the infection of a large group of agents and quantify how contagion amplifies small shocks to the network. We show that when the degree distribution has infinite variance and highly correlated in- and out-degrees, even when agents have high thresholds, a sub-linear fraction of initially infected agents is enough to trigger the infection of a positive fraction of nodes. We also demonstrate how these results are sensitive to vertex and edge percolation (intervention). We then study the asymptotic Nash equilibrium and socially optimal security investment. In the asymptotic limit, agents' risk depends on all other agents' investments through an aggregate quantity that we call network vulnerability. The limit theorems enable us to capture the impact of one class of agents' decisions on the overall network vulnerability. Based on our results, the vulnerability is semi-analytic, allowing for a tractable Nash equilibrium. We provide sufficient conditions for investment in equilibrium to be monotone in network vulnerability. When investment is monotone, we demonstrate that the (asymptotic) Nash equilibrium is unique. In the specific example of two types of core-periphery agents, we illustrate the strong effect of cost heterogeneity on network vulnerability and the non-monotonous investment as a function of costs.

Asymptotic Nash Equilibrium for the M-Ary Sequential Adversarial Hypothesis Testing Game

In this paper, we consider a novel <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$M$ </tex-math></inline-formula> -ary sequential hypothesis testing problem in which an adversary is present and perturbs the distributions of the samples before the decision maker observes them. This problem is formulated as a sequential adversarial hypothesis testing game played between the decision maker and the adversary. This game is a zero-sum and strategic one. We assume the adversary is active under all hypotheses and knows the underlying distribution of observed samples. We adopt this framework as it is the worst-case scenario from the perspective of the decision maker. The goal of the decision maker is to minimize the expectation of the stopping time to ensure that the test is as efficient as possible; the adversary’s goal is, instead, to maximize the stopping time. We derive a pair of strategies under which the asymptotic Nash equilibrium of the game is attained. We also consider the case in which the adversary is not aware of the underlying hypothesis and hence is constrained to apply the same strategy regardless of which hypothesis is in effect. Numerical results corroborate our theoretical findings.

Risk-sensitive mean field games with major and minor players

We investigate a class of mean field games containing a large number of major and minor players. Each player minimizes a quadratic-tracking type risk-sensitive cost functional, where the reference signal is a function of the state average term of the major and minor players. To reduce the complexity for solving the problem, we design a sequence of decentralized strategies by the Nash certainty equivalence principle. Firstly, for the optimal control problems with quadratic type risk-sensitive cost functionals, we propose a new verification theorem. Secondly, we apply the two-layer state aggregation method to construct the fixed-point equations for the estimations of the state average terms and give the conditions for the existence and uniqueness of the fixed points. Then, we design a sequence of decentralized strategies by the estimations of the state average terms based on local information. It is shown that the estimations of the state average terms are consistent with the true values for the closed-loop systems, and the sequence of strategies designed is a decentralized asymptotic Nash equilibrium. Finally, the effectiveness of the theoretical analysis is demonstrated by a numerical example.

Mixed Mean Field Games with Risk-Sensitive Cost Functionals

We investigate a class of mean field games containing a large number of major and minor players. Each player minimizes a quadratic-tracking type risk-sensitive cost functional, where the reference signal is a function of the state average term of the major and minor players. To reduce the complexity for solving the problem, we design a sequence of decentralized strategies by the Nash certainty equivalence principle. Firstly, we apply the two-layer state aggregation method to construct the fixed-point equations for the estimations of the state average terms and give the conditions for the existence and uniqueness of the fixed points. Then, we design a sequence of decentralized strategies by the estimations of the state average terms based on local information. It is shown that the estimations of the state average terms are consistent with the true values for the closed-loop systems, and the sequence of strategies designed is a decentralized asymptotic Nash equilibrium.

Convergence of Policy Gradient Methods for Nash Equilibria in General-sum Stochastic Games

We study Nash equilibria learning of a general-sum stochastic game with an unknown transition probability density function. Agents take actions at the current environment state and their joint action influences the transition of the environment state and their immediate rewards. Each agent only observes the environment state and its own immediate reward and is unknown about the actions or immediate rewards of others. We introduce the concept of weighted asymptotic Nash equilibrium with probability 1 and design a two-loop algorithm by the equivalence of Nash equilibrium and variational inequality problems. In the outer loop, we sequentially update a constructed strongly monotone variational inequality by updating a proximal parameter while employing a single-call extra-gradient algorithm in the inner loop for solving the constructed variational inequality. We show that if the associated Minty variational inequality has a solution, then the designed algorithm converges to the k1/2 -weighted asymptotic Nash equilibrium.

Finite-time 4-expert prediction problem

We explicitly solve the nonlinear PDE that is the continuous limit of dynamic programing equation of the expert prediction problem in finite horizon setting with N = 4 experts. The expert prediction problem is formulated as a zero sum game between a player and an adversary. By showing that the solution is we are able to show that the comb strategies, as conjectured by Gravin et al., form an asymptotic Nash equilibrium. We also prove the “Finite vs Geometric regret” conjecture proposed in arXiv:1409.3040 for N = 4, and show that this conjecture in fact follows from the conjecture that the comb strategies are optimal for all N.

Contagion Risks and Security Investment in Directed Networks

We develop a model for contagion risks and optimal security investment in a directed network of interconnected agents with heterogeneous degrees, loss functions and security profiles. Our model generalizes much of contagion models in the literature; in particular the independent cascade model and the linear threshold model. We state various limit theorems on the final size of infected agents in the case of random networks with given vertex degrees for finite and infinite variance degree distributions. The results allow us to derive a resilience condition of the network to the infection of a large group of agents and quantify how contagion amplifies small shocks to the network. We show that when the degree distribution has infinite variance and, highly correlated in- and out-degrees, then even when agents have high thresholds, a sub-linear fraction of initially infected agents is enough to trigger the infection of a positive fraction. We also show how these results are sensitive to vertex and edge percolation (immunization). We then study the asymptotic Nash equilibrium and socially optimal security investment. In the asymptotic limit, agents' risk depends on all other agents' investment through an aggregate quantity, that we call network vulnerability. The limit theorems allow us to capture the impact of one class of agents' decision on the overall network vulnerability. Based on our results, the vulnerability is semi-analytic allowing for a tractable Nash equilibrium. We give sufficient conditions for investment in equilibrium to be monotone in the network vulnerability. When investment is monotone, we show that the (asymptotic) Nash equilibrium is unique. In the particular example of two types core-periphery agents, we exhibit a strong effect of the cost heterogeneity and in particular non-monotonous investment as a function of costs.

On Tauberian theorem for stationary Nash equilibria

We consider general n-player nonzero-sum dynamic games, which is broader than differential games and could accommodate both discrete and continuous time. Assuming common dynamics, we study the long run average family and discounting average family of the running costs. For each of these game families, we investigate asymptotic properties of its Nash equilibria. We analyze asymptotic Nash equilibria—strategy profiles that are approximately optimal if the planning horizon tends to infinity in long run average games and if the discount tends to zero in discounting games. Moreover, we also assume that this strategy profile is stationary. Under a mild assumption on players’ strategy sets, we prove a uniform Tauberian theorem for stationary asymptotic Nash equilibrium. If a stationary strategy profile is an asymptotic Nash equilibrium and the corresponding Nash value functions converge uniformly for one of the families (when discount goes to zero for discounting games, when planning horizon goes to infinity in long run average games), then for the other family this strategy profile is also an asymptotic Nash equilibrium, and its Nash value functions converge uniformly to the same limit. As an example of application of this theorem, we consider Sorger’ model of competition of two firms.

On Nonzero-Sum Game Considered on Solutions of a Hybrid System with Frequent Random Jumps

We study a non-zero sum game considered on the solutions of a hybrid dynamical system that evolves in continuous time and that is subjected to abrupt changes of parameters. The changes of the parameters are synchronized with (and determined by) the changes of the states/actions of two Markov decision processes, each of which is controlled by a player that aims at minimizing his or her objective function. The lengths of the time intervals between the jumps of the parameters are assumed to be small. We show that an asymptotic Nash equilibrium of such hybrid game can be constructed on the basis of a Nash equilibrium of a deterministic averaged dynamic game.

Stochastic adaptive dynamical games

In this paper, we consider two players zero-sum dynamical games in linear stochastic systems described by input-output model with unknown parameters, and with an ergodic output-tracking quadratic index. We combine the ideas of game theory and control theory as a modeling method, and investigate the corresponding stochastic adaptive game problems. There is few existing results related to such stochastic adaptive game problems, since uncertainty raises the difficulty of making decisions for both players. We attempt to cope with this problem by the ideas and methods of adaptive control. We will firstly use the standard least squares to estimate the unknown parameters, and then construct the adaptive strategy profile for both players by using the so-called ``certainty equivalence principle. We will prove that, the adaptive strategic profile makes the system globally stable, and that the action profile is an asymptotic Nash equilibrium solution to our game problem in a certain sense.

Decentralised dynamic games for large population stochastic multi‐agent systems

In this study, the authors consider the large population dynamic games where each agent evolves according to a dynamic equation containing the input average of all agents. The long time average (LTA) cost that each agent aims to minimise is coupled with other agents’ states via a population state average (PSA), which is also known as the mean field term. In order to design decentralised controls, the Nash certainty equivalence is introduced. It is shown that the resulting decentralised mean field control laws lead the system to mean-consensus asymptotically as time goes to infinity. The stability property of the mass behaviour and the almost sure asymptotic Nash equilibrium property of the optimal controls are also guaranteed, and the case with non-linear system dynamics is also discussed. In addition, the influence of inaccurate mean field information on individual agent is analysed. Finally, they investigate the socially cooperative formulation where the objective is to minimise the social cost as the sum of all individual LTA costs containing the PSA. In this case, they show that the decentralised mean field social controls are the same as the mean field Nash controls for infinite population systems.

Adaptive Tracking Games for Coupled Stochastic Linear Multi-Agent Systems: Stability, Optimality and Robustness

Distributed adaptive tracking-type games are investigated for a class of coupled stochastic linear multi-agent systems with uncertainties of unknown structure parameters, external stochastic disturbances, unmodeled dynamics, and unknown agents' interactions. The control goal is to make the states of all the agents converge to a desired function of the population state average (PSA). Due to the fact that only local information is available for each agent, the control is distributed. For the time-invariant parameter case, the extended least-squares algorithm, Nash certainty equivalence (NCE) principle, and certainty equivalence (CE) principle are used to estimate the unknown parameters and the PSA term, and to design adaptive control, respectively. Under some mild conditions, it is shown that the closed-loop system is almost surely uniformly stable with respect to the population number N; the estimate for the PSA term is strongly consistent; the adaptive control is almost surely an asymptotic Nash equilibrium. When the dynamics of each agent contains time-varying parameters and unmodeled dynamics, the projected least mean square (LMS) algorithm, NCE principle, and CE principle are adopted to estimate the unknown time-varying parameters, and the unknown PSA term, and to design robust adaptive control, respectively. In addition to stability of the closed-loop system and consistency of the PSA estimate, the control law is shown to be robust Nash equilibrium with respect to the unmodeled dynamics, the variation of the unknown parameters, and the external disturbances. Two numerical examples are given to illustrate the methods and results of this paper.

Adaptive Mean Field Games for Large Population Coupled ARX Systems with Unknown Coupling Strength

This paper is concerned with decentralized tracking-type games for large population multi-agent systems with mean-field coupling. The individual dynamics are described by stochastic discrete-time auto-regressive models with exogenous inputs (ARX models), and coupled by terms of the unknown population state average (PSA) with unknown coupling strength. A two-level decentralized adaptive control law is designed. On the high level, the PSA is estimated based on the Nash certainty equivalence (NCE) principle. On the low level, the coupling strength is identified based on decentralized least squares algorithms and the estimate of the PSA. The decentralized control law is constructed by combining the NCE principle and Certainty equivalence (CE) principle. By probability limit theory, under mild conditions, it is shown that: (a) the closed-loop system is stable almost surely; (b) as the number of agents increases to infinity, the estimates of both the PSA and the coupling strength are asymptotically strongly consistent and the decentralized control law is an almost sure asymptotic Nash-equilibrium.

The Source Identification Game: An Information-Theoretic Perspective

We introduce a theoretical framework in which to cast the source identification problem. Thanks to the adoption of a game-theoretic approach, the proposed framework permits us to derive the ultimate achievable performance of the forensic analysis in the presence of an adversary aiming at deceiving it. The asymptotic Nash equilibrium of the source identification game is derived under an assumption on the resources on which the forensic analyst may rely. The payoff at the equilibrium is analyzed, deriving the conditions under which a successful forensic analysis is possible and the error exponent of the false-negative error probability in such a case. The difficulty of deriving a closed-form solution for general instances of the game is alleviated by the introduction of an efficient numerical procedure for the derivation of the optimum attacking strategy. The numerical analysis is applied to a case study to show the kind of information it can provide.

Asymptotically Optimal Decentralized Control for Large Population Stochastic Multiagent Systems

<para xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> The interaction of interest-coupled decision-makers and the uncertainty of individual behavior are prominent characteristics of multiagent systems (MAS). How to break through the framework of conventional control theory, which aims at single decision-maker and single decision objective, and to extend the methodology and tools in the stochastic adaptive control theory to analyze MAS are of great significance. In this paper, a preliminary exploration is made in this direction, and the decentralized control problem is considered for large population stochastic MAS with coupled cost functions. Different from the deterministic discounted costs in the existing differential game models, a time-averaged stochastic cost function is adopted for each agent. The decentralized control law is constructed based on the state aggregation method and tracking-like quadratic optimal control. By using probability limit theory, the stability and optimality of the closed-loop system are analyzed. The main contributions of this paper include the following points. 1) The concepts of asymptotic Nash-equilibrium in probability and almost surely, respectively, are introduced and the relationship between these concepts is illuminated, which provide necessary tools for analyzing the optimality of the decentralized control laws. 2) The closed-loop system is shown to be <emphasis emphasistype="boldital">almost surely uniformly stable</emphasis>, and bounded independently of the number of agents <formula formulatype="inline"><tex>$N$</tex> </formula>. 3) The population state average (PSA) is shown to converge to the infinite population mean (IPM) trajectory in the sense of both <formula formulatype="inline"><tex>${\cal L}_{2}$</tex></formula>-norm and time average almost surely, as <formula formulatype="inline"><tex>$N$</tex></formula> increases to infinity. 4) The decentralized control law is designed and shown to be <emphasis emphasistype="boldital">almost surely</emphasis> asymptotically optimal; the cost of each agent based on local measurements converges to that based on global measurements <emphasis emphasistype="boldital">almost surely</emphasis>, as <formula formulatype="inline"><tex>$N$</tex></formula> increases to infinity. </para>

Decentralized tracking-type games for multi-agent systems with coupled ARX models: Asymptotic Nash equilibria

A class of decentralized tracking-type games is considered for large population multi-agent systems (MAS). The agents are described by stochastic discrete-time auto-regressive models with exogenous inputs (ARX models), and coupled together through their individual dynamics and performance indexes by terms of the unknown population state average (PSA). The performance index of each agent to minimize is a stochastic long term averaged group-tracking-type functional, in which there is a nonlinear term of the unknown PSA. The control law is decentralized and implemented via the Nash certainty equivalence principle. By probability limit theory, under mild conditions it is shown that: (a) the estimate of the PSA is strongly consistent; (b) the closed-loop system is stable almost surely, and the stability is independent of the number N of agents; (c) the decentralized control law is an asymptotic Nash equilibrium almost surely or in probability according to the property of the nonlinear coupling function in the performance indexes.

An Asymptotic Solution for Sealed Bid Common-Value Auctions with Bidders Having Asymmetric Information

We investigate a first-price common-value auction where bidders have asymmetric information about an item of unknown value. We compute the unique Nash equilibrium when the bidders are constrained to translation-invariant bid functions. Further, this profile of bid functions is also an asymptotic Nash equilibrium (without the constraint on the bidders' strategies) as the a priori distribution of the true value becomes increasingly diffuse. All bidders have positive expected profits at equilibrium. In the second-price analogue with two bidders there is a continuum of Nash equilibria in which both bidders have positive expected profits. Journal of Economic Literature Classification Number: C7.