Nash Equilibrium Of Game Research Articles

Deep Reinforcement Learning from Self-Play in No-limit Texas Hold'em Poker

Imperfect information games describe many practical applications found in the real world as the information space is rarely fully available. This particular set of problems is challenging due to the random factor that makes even adaptive methods fail to correctly model the problem and find the best solution. Neural Fictitious Self Play (NFSP) is a powerful algorithm for learning approximate Nash equilibrium of imperfect information games from self-play. However, it uses only crude data as input and its most successful experiment was on the in-limit version of Texas Hold’em Poker. In this paper, we develop a new variant of NFSP that combines the established fictitious self-play with neural gradient play in an attempt to improve the performance on large-scale zero-sum imperfect information games and to solve the more complex no-limit version of Texas Hold’em Poker using powerful handcrafted metrics and heuristics alongside crude, raw data. When applied to no-limit Hold’em Poker, the agents trained through self-play outperformed the ones that used fictitious play with a normal-form single-step approach to the game. Moreover, we showed that our algorithm converges close to a Nash equilibrium within the limited training process of our agents with very limited hardware. Finally, our best self-play-based agent learnt a strategy that rivals expert human level.

Off-policy Q-learning: Solving Nash equilibrium of multi-player games with network-induced delay and unmeasured state

In the framework of adaptive dynamic programming combined with Q-learning, this paper investigates networked multi-player games, in which the common state of the plant is transmitted to all players via a network, for finding the Nash equilibrium solution without requiring the system matrices to be known, even though there exists network-induced delay and system state cannot be directly measured. By adding an observer and a virtual Smith predictor for estimating system state and predicting system state, the control policies of players can be successfully designed. Then, a novel off-policy Q-learning algorithm is proposed to learn the Nash equilibrium solution via solving the coupled algebraic Riccati equations using available data, followed by the rigorous proof of convergence of the proposed algorithm. Finally, an example is given to show the effectiveness of the proposed method.

Ky-Fan inequality, Nash equilibria in some idempotent and harmonic convex structure

B-convexity is defined as a suitable Peano-Kuratowski limit of linear convexities. An alternative idempotent convex structure called inverse B-convexity was recently proposed in the literature. This paper continues and extends some investigation started in these papers. In particular we focus on the Ky-Fan inequality and prove the existence of a Nash equilibrium for inverse B-convex games. This we do by considering a suitable “harmonic” topological structure which allows to establish a KKM theorem as well as some important related properties. Among other things a coincidence theorem is established. The paper also establishes fixed point results and Nash equilibriums properties in the case where two different convex topological structures are merged. It follows that one can consider a large class of games where the players may optimize their payoff subject to different forms of convexity. Among other things an inverse B-convex version of the Debreu-Gale-Nikaido theorem is proposed.

Distributed Nash equilibrium seeking for aggregative games with second-order nonlinear players

The aggregative games of multi-agent systems are studied in this paper, where the players have second-order nonlinear dynamics with uncertain parameters and the communication networks among the players are weight-balanced digraphs. In order to control these heterogeneous second-order nonlinear players to autonomously seek the Nash equilibrium of aggregative games, two distributed algorithms are developed. Moreover, the convergence of the two algorithms is analyzed via Lyapunov theorems and variational analysis. By the two algorithms, the players globally exponentially converge to the Nash equilibrium, although the second-order nonlinear dynamics contain uncertain parameters. Finally, the effectiveness of the two algorithms are verified by a numerical example.

Strict pure strategy Nash equilibrium in large finite-player games when the action set is a manifold

We present results on the relationship between non-atomic games (in distributional form) and approximating games with a large but finite number of players. Specifically, in a setting with differentiable payoff functions, we show that: (1) The set of all non-atomic games has an open dense subset such that any finite-player game that is sufficiently close (in terms of distributions of players’ characteristics) to a game in this subset and has sufficiently many players has a strict pure strategy Nash equilibrium (Theorem 1), and (2) any equilibrium distribution of any non-atomic game is the limit of equilibrium distributions defined from strict pure strategy Nash equilibria of finite-player games (Theorem 2). This supplements our paper Carmona and Podczeck (2020b), where analogous results are established for the case where the action set of players is a subset of some Euclidean space, with non-empty interior, and payoff functions are such that equilibrium actions are in the interior of the action set. The goal of the present paper is to remove these assumptions.

On some general scheme of constructing iterative methods for searching the Nash equilibrium in concave games

The subject of the paper is finite-dimensional concave games id est noncooperative $n$-person games with objective functionals concave with respect to `their own' variables. For such games we investigate the problem of designing iterative algorithms for searching the Nash equilibrium with convergence guaranteed without requirements concerning objective functionals such as smoothness and as convexity in `strange' variables or another similar hypotheses (in the sense of weak convexity, quasiconvexity and so on). In fact, we prove some assertion analogous to the theorem on convergence of $M$-Fej\'{er iterative process for the case when an operator acts in a finite-dimensional compact and nearness to an objective set is measured with the help of arbitrary continuous function. Then, on the base of this assertion we generalize and develop the approach suggested by the author formerly to searching the Nash equilibrium in concave games. The last one can be regarded as "a cross between" the relaxation algorithm and the Hooke-Jeeves method of configurations (but taking into account a specific character of the the residual function being minimized). Moreover, we present results of numerical experiments with their discussion.

A payoff-based learning approach for Nash equilibrium seeking in continuous potential games

This work addresses the Nash equilibrium seeking problem in continuous potential games in a situation that the payoff functions and actions of players are blind to each other due to privacy security or communication limitation. To this end, a payoff-based learning dynamics is proposed with the limitation that only available information for each agent is its own past actions and several observed payoffs. In detail, we formulate a trial-and-error learning protocol to search for proper moving direction and length for action adjustment of each agent, and prove that following the proposed learning dynamics, the action profile of agents will be guaranteed to converge to a Nash equilibrium of the continuous potential game. For illustration of the theoretical development, the proposed payoff-based learning dynamics is further utilized to design an absolute distance-based consensus protocol for multi-agent systems. It is shown that agents can eventually reach a consensus point even when they do not know the relative positions with other agents.

Nash equilibrium computation of two-network zero-sum games with event-triggered communication

In this paper, a zero-sum game Nash equilibrium computation problem with event-triggered communication is investigated under an undirected weight-balanced multi-agent network. A novel distributed event-triggered projection subgradient algorithm is developed to reduce the communication burden within the subnetworks. In the proposed algorithm, when the difference between the current state of the agent and the state of the last trigger time exceeds a given threshold, the agent will be triggered to communicate with its neighbours. Moreover, we prove that all agents converge to Nash equilibrium by the proposed algorithm. Finally, two simulation examples verify that our algorithm not only reduces the communication burden but also ensures that the convergence speed and accuracy are close to that of the time-triggered method under the appropriate threshold.

Games in possibility capacities with payoff expressed by fuzzy integral

This paper studies non-cooperative games where players are allowed to play their mixed non-additive strategies. Expected payoffs are expressed by so-called fuzzy integrals: Choquet integral, Sugeno integral and generalizations of Sugeno integral obtained by using triangular norms. We consider the existence problem of Nash equilibrium for such games. Positive results for Sugeno integral and its generalizations are obtained. However we provide some example of a game with Choquet payoffs which have no Nash equilibrium. Such example demonstrates that fuzzy integrals based on the maximum operation are more suitable for possibility capacities than Choquet integral which is based on the addition operation.

Towards a systematic computational framework for modeling multi-agent decision-making at micro level for smart vehicles in a smart world

We propose a multi-agent based computational framework for modeling decision-making and strategic interaction at micro level for smart vehicles in a smart world. The concepts of Markov game and best response dynamics are heavily leveraged. Our aim is to make the framework conceptually sound and computationally practical for a range of realistic applications, including micro path planning for autonomous vehicles. To this end, we first convert the would-be stochastic game problem into a closely related deterministic one by introducing risk premium in the utility function for each individual agent. We show how the sub-game perfect Nash equilibrium of the simplified deterministic game can be solved by an algorithm based on best response dynamics. In order to better model human driving behaviors with bounded rationality, we seek to further simplify the solution concept by replacing the Nash equilibrium condition with a heuristic and adaptive optimization with finite look-ahead anticipation. In addition, the algorithm corresponding to the new solution concept drastically improves the computational efficiency. To demonstrate how our approach can be applied to realistic traffic settings, we conduct a simulation experiment: to derive merging and yielding behaviors on a double-lane highway with an unexpected barrier. Despite assumption differences involved in the two solution concepts, the derived numerical solutions show that the endogenized driving behaviors are very similar. We also briefly comment on how the proposed framework can be further extended in a number of directions in our forthcoming work, such as behavioral calibration using real traffic video data, and computational mechanism design for traffic policy optimization.

A Monte Carlo Neural Fictitious Self-Play approach to approximate Nash Equilibrium in imperfect-information dynamic games

A Monte Carlo Neural Fictitious Self-Play approach to approximate Nash Equilibrium in imperfect-information dynamic games

Coalition-proof stability and optimal surplus sharing in international environmental agreements

In a Nash equilibrium of voluntary participation games in international environmental agreements, the optimal surplus sharing rules, proposed by Eyckmans and Finus (2004) and Weikard (2009), incentivize more countries to participate in the agreements. We examine how robust this desirability of the rules is to multilateral deviations.

Nash Equilibrium Sequence in a Singular Two-Person Linear-Quadratic Differential Game

A finite-horizon two-person non-zero-sum differential game is considered. The dynamics of the game is linear. Each of the players has a quadratic functional on its own disposal, which should be minimized. The case where weight matrices in control costs of one player are singular in both functionals is studied. Hence, the game under the consideration is singular. A novel definition of the Nash equilibrium in this game (a Nash equilibrium sequence) is proposed. The game is solved by application of the regularization method. This method yields a new differential game, which is a regular Nash equilibrium game. Moreover, the new game is a partial cheap control game. An asymptotic analysis of this game is carried out. Based on this analysis, the Nash equilibrium sequence of the pairs of the players’ state-feedback controls in the singular game is constructed. The expressions for the optimal values of the functionals in the singular game are obtained. Illustrative examples are presented.

A Two-Stage Closed-Loop Supply Chain Pricing Decision: Cross-Channel Recycling and Channel Preference

This paper focuses on the pricing problem of a two-stage closed-loop supply chain (CLSC) considering the cross-channel recycling and channel preference based on a single manufacturer and a single traditional retailer. The pricing decision problem raises from the manufacturer’s direct sales and the retailer’s retailing including recycling. Managers need to focus on intelligible management considering consumer channel preferences, cross-channel recovery and pricing strategies. According to game theory, centralized and decentralized CLSC decision models are used to provide an efficient solution to managers for the pricing problem. The centralized model consists of differential and uniform pricing strategy and the decentralized model consists of manufacturer-led Stackelberg, retailer-led Stackelberg and Nash equilibrium game, respectively. The impact of cross-channel recycling rate and channel preference on pricing and profitability in a two-stage CLSC system is explained elaborately in this study. The results show that cross-channel recovery rates and consumer channel preferences have a direct significant impact on pricing strategies including profit allocation decisions in CLSC. It demonstrated that different channel preferences leading to different pricing strategies and decision for manufacturers and retailers choices. Manufacturer’s pricing decreases when channel preferences are constant and cross-channel recovery rates increase. Retailer’s pricing remains stable as the cross-channel recovery rate has less affected on it. Furthermore, if the cross-channel recovery rates increase, then the manufacturers pricing decreases and retailers pricing increases. This information will be a helpful guideline for the manager to select suitable pricing strategies based on the company scenario.

Linear model for coputation of equilibrium point in bimatrix game

The methods for finding Nash equilibrium in bimatrix game are introduced in this paper. Linear optimization model with binary variables for coputation of all equilibrium points in nongenerate bimatrix game is proposed.

Similarity as an extension of symmetry and its application to superrationality

In this paper we present a concept of similarity in games, on which to ground alternative solution concepts, some of which differ from the classical notions in the field. In order to do this we impose a constraint on players’ beliefs that amounts to a variant of the well-known symmetry principle in classical bargaining theory. We show how this similarity relation helps to identify different Nash equilibria in games, and how these “similar Nash equilibria” can be extended to non-symmetric games. While the notion is normative, it is nonetheless inspired by phenomena in which similarities between players lead to outcomes detected in behavioral studies. We study the strategic properties of the concept of similarity and discuss its relationships with Hofstadter’ notion of superrationality.

Stochastic Superiority Equilibrium in Game Theory

The definition of best response for a player in the Nash equilibrium is based on maximizing the expected utility given the strategy of the rest of the players in a game. In this work, we consider stochastic games, that is, games with random payoffs, in which a finite number of players engage only once or at most a limited number of times. In such games, players may choose to deviate from maximizing their expected utility. This is because maximizing expected utility strategy does not address the uncertainty in payoffs. We instead define a new notion of a stochastic superiority best response. This notion of best response results in a stochastic superiority equilibrium in which players choose to play the strategy that maximizes the probability of them being rewarded the most in a single round of the game rather than maximizing the expected received reward, subject to the actions of other players. We prove the stochastic superiority equilibrium to exist in all finite games, that is, games with a finite number of players and actions, and numerically compare its performance to Nash equilibrium in finite-time stochastic games. In certain cases, we show the payoff under the stochastic superiority equilibrium is 70% likely to be higher than the payoff under Nash equilibrium.

Algorithm for Computing Approximate Nash Equilibrium in Continuous Games with Application to Continuous Blotto

Successful algorithms have been developed for computing Nash equilibrium in a variety of finite game classes. However, solving continuous games—in which the pure strategy space is (potentially uncountably) infinite—is far more challenging. Nonetheless, many real-world domains have continuous action spaces, e.g., where actions refer to an amount of time, money, or other resource that is naturally modeled as being real-valued as opposed to integral. We present a new algorithm for approximating Nash equilibrium strategies in continuous games. In addition to two-player zero-sum games, our algorithm also applies to multiplayer games and games with imperfect information. We experiment with our algorithm on a continuous imperfect-information Blotto game, in which two players distribute resources over multiple battlefields. Blotto games have frequently been used to model national security scenarios and have also been applied to electoral competition and auction theory. Experiments show that our algorithm is able to quickly compute close approximations of Nash equilibrium strategies for this game.

An axiomatic and non-cooperative approach to the multi-step Shapley value

Inspired by the two-step Shapley value, in this paper we introduce and axiomatize the multi-step Shapley value for cooperative games with levels structures. Moreover, we design a multi-step bidding mechanism, which implements the value strategically in subgame perfect Nash equilibrium for superadditve games.

Expressiveness and Nash Equilibrium in Iterated Boolean Games

We define and investigate a novel notion of expressiveness for temporal logics that is based on game theoretic equilibria of multi-agent systems. We use iterated Boolean games as our abstract model of multi-agent systems [Gutierrez et al. 2013, 2015a]. In such a game, each agent has a goal , represented using (a fragment of) Linear Temporal Logic ( ) . The goal captures agent ’s preferences, in the sense that the models of represent system behaviours that would satisfy . Each player controls a subset of Boolean variables , and at each round in the game, player is at liberty to choose values for variables in any way that she sees fit. Play continues for an infinite sequence of rounds, and so as players act they collectively trace out a model for , which for every player will either satisfy or fail to satisfy their goal. Players are assumed to act strategically, taking into account the goals of other players, in an attempt to bring about computations satisfying their goal. In this setting, we apply the standard game-theoretic concept of (pure) Nash equilibria. The (possibly empty) set of Nash equilibria of an iterated Boolean game can be understood as inducing a set of computations, each computation representing one way the system could evolve if players chose strategies that together constitute a Nash equilibrium. Such a set of equilibrium computations expresses a temporal property—which may or may not be expressible within a particular fragment. The new notion of expressiveness that we formally define and investigate is then as follows: What temporal properties are characterised by the Nash equilibria of games in which agent goals are expressed in specific fragments of ? We formally define and investigate this notion of expressiveness for a range of fragments. For example, a very natural question is the following: Suppose we have an iterated Boolean game in which every goal is represented using a particular fragment of : is it then always the case that the equilibria of the game can be characterised within ? We show that this is not true in general.