Nash Equilibrium Of Game Research Articles

Heterogeneous negotiation undermines cooperation in prisoner’s dilemma game

Contradictory findings have been reached on the effect of heterogeneous social ties and heterogeneous payoffs fostering cooperation, in which the former is positive while the latter is negative, due to the neglect of a common fact that both social ties and payoffs may be strongly impacted by individuals’ negotiation status. Thus, we here examine how the heterogeneity of negotiation status influences cooperation by acting jointly on both aspects. To this end, we design a two-stage model. In the first stage, the network dynamics and payoff structure are defined by a heterogeneous negotiation edge game, in which the share of payoff redistribution is associated with the negotiation status. The cooperative strategy dynamics of the second stage are contingent on the network and payoff structure established in the first. The Nash Equilibrium of the heterogeneous negotiation edge game reveals how the network and payoff are changed by the heterogeneous negotiation status. More specifically, the stronger the negotiation status of the relationship destroyer, the more payoffs are distributed to cooperators. Investing excessive negotiation status with the relationship destroyer, however, potentially disconnect links between cooperators. Furthermore, the simulation shows that the efficiency of fostering cooperation reduces as heterogeneity in negotiation status grows greater. In other words, we demonstrate equal negotiation status is the most effective for promoting cooperation. We claim that this work has proposed a potentially persuasive approach to explain why contradictory results were produced by relative studies and it contributes to a further understanding of the relationship between heterogeneity and cooperation.

Don’t Predict Counterfactual Values, Predict Expected Values Instead

Counterfactual Regret Minimization algorithms are the most popular way of estimating the Nash Equilibrium in imperfect-information zero-sum games. In particular, DeepStack -- the state-of-the-art Poker bot -- employs the so-called Deep Counterfactual Value Network (DCVN) to learn the Counterfactual Values (CFVs) associated with various states in the game. Each CFV is a multiplication of two factors: (1) the probability that the opponent would reach a given state in a game, which can be explicitly calculated from the input data, and (2) the expected value (EV) of a payoff in that state, which is a complex function of the input data, hard to calculate. In this paper, we propose a simple yet powerful modification to the CFVs estimation process, which consists in utilizing a deep neural network to estimate only the EV factor of CFV. This new target setting significantly simplifies the learning problem and leads to much more accurate CFVs estimation. A direct comparison, in terms of CFVs prediction losses, shows a significant prediction accuracy improvement of the proposed approach (DEVN) over the original DCVN formulation (relatively by 9.18-15.70% when using card abstraction, and by 3.37-8.39% without card abstraction, depending on a particular setting). Furthermore, the application of DEVN improves the theoretical lower bound of the error by 29.05-31.83% compared to the DCVN pipeline when card abstraction is applied. Additionally, DEVN is able to achieve the goal using significantly smaller, and faster to infer, networks. While the proposed modification may seem to be of a rather technical nature, it, in fact, presents a fundamentally different approach to the overall process of learning and estimating CFVs, since the distributions of the training signals differ significantly between DCVN and DEVN. The former estimates CFVs, which are biased by the probability of reaching a given game state, while training the latter relies on a direct EV estimation, regardless of the state probability. In effect, the learning signal of DEVN presents a better estimation of the true value of a given state, thus allowing more accurate CFVs estimation.

An Efficient Deep Reinforcement Learning Algorithm for Solving Imperfect Information Extensive-Form Games

One of the most popular methods for learning Nash equilibrium (NE) in large-scale imperfect information extensive-form games (IIEFGs) is the neural variants of counterfactual regret minimization (CFR). CFR is a special case of Follow-The-Regularized-Leader (FTRL). At each iteration, the neural variants of CFR update the agent's strategy via the estimated counterfactual regrets. Then, they use neural networks to approximate the new strategy, which incurs an approximation error. These approximation errors will accumulate since the counterfactual regrets at iteration t are estimated using the agent's past approximated strategies. Such accumulated approximation error causes poor performance. To address this accumulated approximation error, we propose a novel FTRL algorithm called FTRL-ORW, which does not utilize the agent's past strategies to pick the next iteration strategy. More importantly, FTRL-ORW can update its strategy via the trajectories sampled from the game, which is suitable to solve large-scale IIEFGs since sampling multiple actions for each information set is too expensive in such games. However, it remains unclear which algorithm to use to compute the next iteration strategy for FTRL-ORW when only such sampled trajectories are revealed at iteration t. To address this problem and scale FTRL-ORW to large-scale games, we provide a model-free method called Deep FTRL-ORW, which computes the next iteration strategy using model-free Maximum Entropy Deep Reinforcement Learning. Experimental results on two-player zero-sum IIEFGs show that Deep FTRL-ORW significantly outperforms existing model-free neural methods and OS-MCCFR.

RM-FSP: Regret minimization optimizes neural fictitious self-play

To compute the optimal strategy in competitive games, algorithms have been developed to achieve the Nash equilibrium. Current deep learning algorithms have succeeded in many games; however, optimizing the algorithms to approach the Nash equilibrium in imperfect-information games like StarCraft and Poker remains challenging.Neural Fictitious Self-Play (NFSP) is an effective end-to-end algorithm to learn an approximate Nash equilibrium in imperfect-information games. However, because a player in NFSP trains its best response according to its opponents’ past strategies, a discrepancy exists between the optimal strategy and the learned best response after the player updates its strategies. We call this discrepancy the optimality gap. During training, the optimality gap does not decay monotonically, which causes suboptimal results or unstable convergence of NFSP. We improve the performance of NFSP by allowing the optimality gap to decay monotonically. In this study, we propose Regret Minimization Fictitious Self-Play (RM-FSP), which applies a regret minimization method to compute NFSP’s best response. The regret minimization method effectively converges the optimality gap monotonically and faster than in NFSP. We prove there will be a better learning bound than the original NFSP after applying regret minimization methods to NFSP. Experiments on three typical environments in OpenSpiel demonstrate that RM-FSP outperforms NFSP in both exploitability (discrepancy between the learned policy profile and the Nash equilibrium) and time efficiency.

Consensus as a Nash Equilibrium of a Stochastic Differential Game

In this paper a consensus has been constructed in a social network which is modeled by a stochastic differential game played by agents of that network. Each agent independently minimizes a cost function which represents their motives. A conditionally expected integral cost function has been considered under an agent’s opinion filtration. The dynamic cost functional is minimized subject to a stochastic differential opinion dynamics. As opinion dynamics represents an agent’s differences of opinion from the others as well as from their previous opinions, random influences and stubbornness make it more volatile. An agent uses their rate of change of opinion at certain time point as a control input. This turns out to be a non-cooperative stochastic differential game which have a feedback Nash equilibrium. A Feynman-type path integral approach has been used to determine an optimal feedback opinion and control. This is a new approach in this literature. Later in this paper an explicit solution of a feedback Nash equilibrium opinion is determined.

Propagation of Chaos of Forward–Backward Stochastic Differential Equations with Graphon Interactions

In this paper, we study graphon mean field games using a system of forward–backward stochastic differential equations. We establish the existence and uniqueness of solutions under two different assumptions and prove the stability with respect to the interacting graphons which are necessary to show propagation of chaos results. As an application of propagation of chaos, we prove the convergence of n-player game Nash equilibrium for a general model, which is new in the theory of graphon mean field games.

On tightness of Tsaknakis-Spirakis descent methods for approximate Nash equilibria

This article explores the minimum approximation ratio for Nash equilibrium in bi-matrix games, focusing on the Tsaknakis and Spirakis (TS) methods. The previous SOTA, TS algorithm, achieved an approximation ratio of 0.3393, but efforts to improve the analysis of the TS algorithm have been unsuccessful. This work demonstrates that the bound of 0.3393 is tight for the TS algorithm and presents a theoretical worst-case analysis. A condition for identifying tight instances is provided, along with a generator. While most generated instances are unstable, indicating potential improvements, stable instances exist where perturbations cannot enhance the 0.3393 bound. Other approximate algorithms, such as regret-matching and fictitious play, achieve better ratios on these instances. The generated instances can serve as benchmarks for approximate Nash equilibrium algorithms. The article also mentions progress in the TS algorithm, achieving an approximation ratio of 1/3, which can be further studied using the presented techniques.

Game Theory of Universal Health Coverage for Sustainable Development

1. Aklilu Endalamaw, Charles F Gilks, Fentie Ambaw, Tesfa Dejenie Habtewold and Yibeltal Assefa. (2022). Universal Health Coverage for Antiretroviral Treatment:A Review. Retrieved from Researchgate website: https://www.researchgate.net/publication/366504047_Universal_Health_Coverage_for_Antiretroviral_Treatment_A_Review#fullTextFileContent 2. Badia Leonardo and Marchioro Thomas. (2022). Game Theory a handbook of problems and exercises. Società Editrice Esculapio s.r.l . Italy. 3. Barachini Franz and Stary Christian. (2022). From digital twins to digital selves and beyond. Engineering and social models for a trans- humanist world. Springer. Switzlerland. 4. Elena Parilina, Puduru Viswanadha Reddy, Georges Zaccour. (2022). Theory and Applications of Dynamic Games. A Course on Noncooperative and Cooperative Games Played over Event Trees. Springer. 5. Eoghan Brady a, Andy E. Carmone a, Sarthak Dasb, Raphael Hurleya, Maria Theresa Martinez Vergarac, and Address Malata. (2020). Harnessing the Power of Networks of Care for Universal Health Coverage Retrieved from Researchgate website: https://www.researchgate.net/publication/346521559_Harnessing_the_Power_of_Networks_of_Care_for_Universal_Health_Coverage 6. Fan Bai. (2016). Uniqueness of Nash equilibrium in vaccination games. Retrieved from tandfonline website: https://www.tandfonline.com/doi/epdf/10.1080/17513758.2016.1213319?needAccess=true&role=button 7. Hossain Golam and Sabiruzzaman (2021). Health expenditure, universal health coverage and mortality of mother andchild in South East Asia. Retrieved from Researchgate website: https://www.researchgate.net/publication/348906783_Health_expenditure_universal_health_coverage_and_mortality_of_mother_and_child_in_South_East_Asia#fullTextFileContent 8. Icddr,b. (2011). Science to Accelerate Universal Health Coverage. Retrieved from Researchgate website: https://www.researchgate.net/publication/363761888_Science_to_Accelerate_Universal_Health_Coverage#fullTextFileContent 9. O. Orjingene a, Ugo Okoli a, A. Adetunji-Hassan b and M. Ganama. (2022). Universal Health Coverage: A Catalyst for Achieving 2030 Sustainable Development Goals. Retrieved from Researchgate website: https://www.researchgate.net/publication/363255711_Universal_Health_Coverage_A_Catalyst_for_Achieving_2030_Sustainable_Development_Goals#fullTextFileContent 10. Prince Agwu, Obinna Onwujekwe, Benjamin Uzochukwu, Modest Mulenga. (2021). Improving service delivery at primary healthcare facilities for achievingUniversal Health Coverage: Examining the effects of insecurity in such fa-cilities in Enugu State, Nigeria Retrieved from Researchgate website: https://www.researchgate.net/publication/351022237_universal-health-coverage-insecurity#fullTextFileContent 11. Sandhya Venkateswaran, Shruti Slaria3 and Sampriti Mukherjee. (2022). Political motivation as akey driver for universal health coverage. Retrieved from DOI: 10.3389/fpubh.2022.922578 12. Sheryl L. Changa, Mahendra Piraveenan, Philippa PattisoncandMikhail Prokopenko. (2020). Game theoretic modelling of infectious disease dynamics and intervention methods: a review Retrieved from tandfonline website: https://www.tandfonline.com/doi/epdf/10.1080/17513758.2020.1720322?needAccess=true&role=button 13. Talwalkar P. (2016). The Joy of Game Theory: An Introduction to Strategic Thinking. Create space independent publishing platform. Standford University. UK. 14. Tjemkes Brian and Oli Mihalache. (2021). Transformative Strategies: Strategic Thinking in the Age of Globalization, Disruption, Collaboration and Responsibility. Routledge.USA. 15. Universal Health Coverage (UHC) Index. (2023) Retieve from website https://ourworldindata.org/grapher/universal-health-coverage-index?country=BWA~GMB~YEM 16. Yanhao Lin. (2022). Applications of Game Theory in Microeconomics. Retrieve online from website https://drpress.org/ojs/index.php/EHSS/article/view/777 17. WHO. (2021). Tracking UHC 2021 Global Monitoring Report. Retrieve online from websitehttps://www.who.int/news-room/fact-sheets/detail/universal-health-coverage-(uhc)

Pure-strategy equilibrium in Bayesian potential games with absolutely continuous information

We prove the existence of a pure-strategy Bayesian Nash equilibrium in Bayesian games with absolutely continuous information and a Bayesian potential that is upper semi-continuous in actions for any realization of the players' types. In particular, all Bayesian potential games with finitely many actions and absolutely continuous information possess a pure-strategy Bayesian Nash equilibrium.

Kdb-D2CFR: Solving Multiplayer imperfect-information games with knowledge distillation-based DeepCFR

Counterfactual regret minimization (CFR) is a popular method for finding approximate Nash equilibrium in imperfect-information games (IIG). However, CFR based methods for the IIG are either only designed for two-player IIGs or require much expert knowledge. In this paper, towards to solve the multiplayer IIG problem without utilizing much expert knowledge, we proposed a practical knowledge distillation based framework, which aims to transferring the knowledge from model of two-player IIG into the multiplayer one. By this framework, both of the training efficiency and performance is improved. To eliminate the requirement of expert knowledge, here we introduced a deep learning based CFR in the framework, by which the counterfactual value of CFR can be estimated in an end-to-end way without any expert knowledge and abstraction. We further propose kdb-DeepCFR and kdb-D2CFR based on DeepCFR and D2CFR respectively, which can effectively solve the strategy of multiplayer large-scale game problems. The extensive experiments conducted on 3-8 players poker games suggest that our method outperforms other baselines in the game performance.

Transient Impact from the Nash Equilibrium of a Permanent Market Impact Game

A large body of empirical literature has shown that market impact of financial prices is transient. However, from a theoretical standpoint, the origin of this temporary nature is still unclear. We show that an implied transient impact arises from the Nash equilibrium between a directional trader and one arbitrageur in a market impact game with fixed and permanent impact. The implied impact is the one that can be empirically inferred from the directional trader’s trading profile and price reaction to order flow. Specifically, we propose two approaches to derive the functional form of the decay kernel of the transient impact model, one of the most popular empirical models for transient impact, from the behavior of the directional trader at the Nash equilibrium. The first is based on the relationship between past order flow and future price change, while in the second we solve an inverse optimal execution problem. We show that in the first approach the implied kernel is unique, while in the second case infinite solutions exist and a linear kernel can always be inferred.

Computational mean-field games on manifolds

Conventional Mean-field games/control study the behavior of a large number of rational agents moving in Euclidean spaces. In this work, we explore the mean-field games on Riemannian manifolds. We formulate the mean-field game Nash Equilibrium on manifolds. We also establish the equivalence between the PDE system and the optimality conditions of the associated variational form on manifolds. Based on the triangular mesh representation of two-dimensional manifolds, we design a proximal gradient method for variational mean-field games. Our comprehensive numerical experiments on various manifolds illustrate the effectiveness and flexibility of the proposed model and numerical methods.

Pure Nash Equilibrium in a Two-Step Pricing Game: Covering a Tourist City with Retail Outlets

Pure Nash Equilibrium in a Two-Step Pricing Game: Covering a Tourist City with Retail Outlets

Differential Game-Based Optimal Control of Autonomous Vehicle Convoy

Group control of connected and autonomous vehicles on automated highways is challenging for the advanced driver assistance systems (ADAS) and the automated driving systems (ADS). This paper investigates the differential game-based approach to autonomous convoy control with the aim of deployment on automated highways. Under the noncooperative differential games, the coupled vehicles make their decisions independently while their states are interdependent. The receding horizon Nash equilibrium of the linear-quadratic differential game provides the convoy a distributed state-feedback control strategy. This approach suffers a fundamental issue that neither a Nash equilibrium’s existence nor the uniqueness is guaranteed. We convert the individual dynamics-based differential game to a relative dynamics-based optimal control problem that carries all the features of the differential game. The existence of a unique Nash control under the differential game corresponds to a unique solution to the optimal control problem. The latter is shown, as well as the asymptotic stability of the closed-loop system. Simulations illustrate the effectiveness of the presented convey control scheme and how it well suits automated highway driving scenarios.

The Computation of Approximate Generalized Feedback Nash Equilibria

We present the concept of a generalized feedback Nash equilibrium (GFNE) in dynamic games, extending the feedback Nash equilibrium concept to games in which players are subject to state and input constraints. We formalize necessary and sufficient conditions for (local) GFNE solutions at the trajectory level, which enable the development of efficient numerical methods for their computation. Specifically, we propose a Newton-style method for finding game trajectories which satisfy necessary conditions for an equilibrium, which can then be checked against sufficiency conditions. We show that the evaluation of the necessary conditions in general requires computing a series of nested, implicitly defined derivatives, which quickly becomes intractable. To this end, we introduce an approximation to the necessary conditions which is amenable to efficient evaluation and, in turn, computation of solutions. We call the solutions to the approximate necessary conditions the generalized feedback quasi-Nash equilibria, and we introduce numerical methods for their computation. In particular, we develop a sequential linear-quadratic (LQ) game approach, in which an LQ local approximation of the game is solved at each iteration. The development of this method relies on the ability to compute a GFNE to inequality- and equality-constrained LQ games, and therefore specific methods for the solution of these special cases are developed in detail. We demonstrate the effectiveness of the proposed solution approach on a dynamic game arising in an autonomous driving application.

Neuro-adaptive control for searching generalized Nash equilibrium of multi-agent games: A two-stage design approach

This paper investigates the generalized Nash equilibrium searching problem of multi-agent games over weight-unbalanced directed networks. In this problem, the feasible action set of each agent is not only constrained by private convex sets and shared coupling equality, but also constrained by its own uncertain dynamics. Moreover, the local cost function is related to both his own actions and others, and each agent is only allowed access to neighbor information. To address this problem, we propose a neuro-adaptive control strategy based on two-stage design. In the first stage of design process, an approximator with adaptive-gain is designed to generate a virtual trajectory that converges to a necessary Nash equilibrium, the compensator based on consensus-tracking is used to obtain non-neighbor information and several auxiliary state variables are used to exchange information with local neighborhoods on a digraphs to deal with equality constraints. In the second stage of the strategy, we further developed a neuro-adaptive tracking controller based on one-step design for tracking the virtual trajectory. In the controller design, the virtual control variables and the actual control laws are obtained in a collective way, without repeating the design process. We introduce a variable called the minimum learning parameter to reduce the number of online learning parameters of the neural network. By using tools from variational inequality theory and Lyapunov stability theory, we prove that the proposed control strategy can drive all agents reach Nash equilibrium. Finally, the effectiveness of control strategy is verified by simulation example.

Approximating Nash equilibrium for anti-UAV jamming Markov game using a novel event-triggered multi-agent reinforcement learning

In the downlink communication, it is currently challenging for ground users to cope with the uncertain interference from aerial intelligent jammers. The cooperation and competition between ground users and unmanned aerial vehicle (UAV) jammers leads to a Markov game problem of anti-UAV jamming. Therefore, a model-free method is adopted based on multi-agent reinforcement learning (MARL) to handle the Markov game. However, the benchmark MARL strategies suffer from dimension explosion and local optimal convergence. To solve these issues, a novel event-triggered multi-agent proximal policy optimization algorithm with Beta strategy (ETMAPPO) is proposed in this paper, which aims to reduce the dimension of information transmission and improve the efficiency of policy convergence. In this event-triggering mechanism, agents can learn to obtain appropriate observation in different moment, thereby reducing the transmission of valueless information. Beta operator is used to optimize the action search. It expands the search scope of policy space. Ablation simulations show that the proposed strategy achieves better global benefits with fewer dimension of information than benchmark algorithms. In addition, the convergence performance verifies that the well-trained ETMAPPO has the capability to achieve stable jamming strategies and stable anti-jamming strategies. This approximately constitutes the Nash equilibrium of the anti-jamming Markov game.

Solving Two-Person Zero-Sum Stochastic Games With Incomplete Information Using Learning Automata With Artificial Barriers.

Learning automata (LA) with artificially absorbing barriers was a completely new horizon of research in the 1980s (Oommen, 1986). These new machines yielded properties that were previously unknown. More recently, absorbing barriers have been introduced in continuous estimator algorithms so that the proofs could follow a martingale property, as opposed to monotonicity (Zhang et al., 2014), (Zhang et al., 2015). However, the applications of LA with artificial barriers are almost nonexistent. In that regard, this article is pioneering in that it provides effective and accurate solutions to an extremely complex application domain, namely that of solving two-person zero-sum stochastic games that are provided with incomplete information. LA have been previously used (Sastry et al., 1994) to design algorithms capable of converging to the game's Nash equilibrium under limited information. Those algorithms have focused on the case where the saddle point of the game exists in a pure strategy. However, the majority of the LA algorithms used for games are absorbing in the probability simplex space, and thus, they converge to an exclusive choice of a single action. These LA are thus unable to converge to other mixed Nash equilibria when the game possesses no saddle point for a pure strategy. The pioneering contribution of this article is that we propose an LA solution that is able to converge to an optimal mixed Nash equilibrium even though there may be no saddle point when a pure strategy is invoked. The scheme, being of the linear reward-inaction ( LR-I ) paradigm, is in and of itself, absorbing. However, by incorporating artificial barriers, we prevent it from being "stuck" or getting absorbed in pure strategies. Unlike the linear reward- ϵ penalty ( LR-ϵP ) scheme proposed by Lakshmivarahan and Narendra almost four decades ago, our new scheme achieves the same goal with much less parameter tuning and in a more elegant manner. This article includes the nontrial proofs of the theoretical results characterizing our scheme and also contains experimental verification that confirms our theoretical findings.

CCGnet: A deep learning approach to predict Nash equilibrium of chance-constrained games

This paper proposes a novel method for efficiently finding the Nash equilibrium in a chance-constrained game (CCG). Conventional numerical solution methods require significant computational time when solving multiple instances of CCG. We introduce CCGnet, a deep learning approach that is capable of efficiently solving multiple instances of CCG in a one-shot manner. CCGnet employs a specialized network structure and training algorithm based on neurodynamic optimization. We demonstrate the strong performance of CCGnet in practice and show that our proposed method outperforms conventional methods.

On Approximate and Weak Correlated Equilibria in Constrained Discounted Stochastic Games

In this paper, we consider constrained discounted stochastic games with a countably generated state space and norm continuous transition probability having a density function. We prove existence of approximate stationary equilibria and stationary weak correlated equilibria. Our results imply the existence of stationary Nash equilibrium in ARAT stochastic games.