Information Compression in Dynamic Games

Two of the most important refinements of the Nash equilibrium concept for extensive form games with perfect recall are Selten's (1975) perfect equilibrium and Kreps and Wilson's (1982) more inclusive sequential equilibrium. These two equilibrium refinements are motivated in very different ways. Nonetheless, as Kreps and Wilson (1982, Section 7) point out, the two concepts lead to similar prescriptions for equilibrium play. For each particular game form, every perfect equilibrium is sequential. Moreover, for almost all assignments of payoffs to outcomes, almost all sequential equilibrium strategy profiles are perfect equilibrium profiles, and all sequential equilibrium outcomes are perfect equilibrium outcomes. We establish a stronger result: For almost all assignments of payoffs to outcomes, the sets of sequential and perfect equilibrium strategy profiles are identical. In other words, for almost all games each strategy profile which can be supported by beliefs satisfying the rationality requirement of sequential equilibrium can actually be supported by beliefs satisfying the stronger rationality requirement of perfect equilibrium. We obtain this result by exploiting the algebraic/geometric structure of these equilibrium correspondences, following from the fact that they are semi-algebraic sets; i.e., they are defined by finite systems of polynomial inequalities. That the perfect and sequential equilibrium correspondences have this semi-algebraic structure follows from a deep result from mathematical logic, the Tarski-Seidenberg Theorem; that this structure has important game-theoretic consequences follows from deep properties of semi-algebraic sets.

We analyze a class of stochastic dynamic games among teams with asymmetric information, where members of a team share their observations internally with a delay of d. Each team is associated with a controlled Markov Chain, whose dynamics are coupled through the players’ actions. These games exhibit challenges in both theory and practice due to the presence of signaling and the increasing domain of information over time. We develop a general approach to characterize a subset of Nash equilibria where the agents can use a compressed version of their information, instead of the full information, to choose their actions. We identify two subclasses of strategies: sufficient private information-Based (SPIB) strategies, which only compress private information, and compressed information-based (CIB) strategies, which compress both common and private information. We show that SPIB-strategy-based equilibria exist and the set of payoff profiles of such equilibria is the same as that of all Nash equilibria. On the other hand, we show that CIB-strategy-based equilibria may not exist. We develop a backward inductive sequential procedure, whose solution (if it exists) provides a CIB strategy-based equilibrium. We identify some instances where we can guarantee the existence of a solution to the above procedure. Our results highlight the tension among compression of information, ability of compression-based strategies to sustain all or some of the equilibrium payoff profiles, and backward inductive sequential computation of equilibria in stochastic dynamic games with asymmetric information.

Modeling, analysis, and dynamics of Bayesian games via matrix-based method

In this study, a continuous model of stochastic dynamic game for water allocation from a reservoir system was developed. The continuous random variable of inflow in the state transition function was replaced with a discrete approximant rather than using the mean of the random variable as is done in a continuous model of deterministic dynamic game. As a result, a new solution method was used to solve the stochastic model of game based on collocation method. The collocation method was introduced as an alternative to linear-quadratic (LQ) approximation methods to resolve a dynamic model of game. The collocation method is not limited to the first and second degree approximations, compared to LQ approximation, i.e. Ricatti equations. Furthermore, in spite of LQ related problems, consideration of the stochastic nature of game on the action variables in the collocation method would be possible. The proposed solution method was applied to the real case of reservoir operation, which typically requires considering the effect of uncertainty on decision variables. The results of the solution of the stochastic model of game are compared with the results of a deterministic solution of game, a classical stochastic dynamic programming model (e.g. Bayesian Stochastic Dynamic Programming model, BSDP), and a discrete stochastic dynamic game model (PSDNG). By comparing the results of alternative methods, it is shown that the proposed solution method of stochastic dynamic game is quite capable of providing appropriate reservoir operating policies.

This chapter is an introduction to game theory, and to its application to situations of asymmetric information. It covers simultaneous and sequential games, dominant strategies, Nash equilibrium, backward induction, subgame perfection, repeated games, Bayesian games, sequential rationality, behavioural strategies, perfect Bayesian equilibrium, sequential equilibrium. The Centipede game is taken as an incentive to be critical of the usual definition of rationality; some doubts are also raised about the universal certainty that dominant strategies will always be played: some examples of Prisoners’ Dilemma suggest as plausible that players may choose ‘Cooperate’. The treatment of auctions is limited but rigorous, it is proved that a first-price sealed-bid (independent-private-values) auction and a second-price sealed-bid auction generate the same expected revenue for the seller, and on this basis the revenue equivalence theorem is sketched; for common-value auctions the winner’s curse is examined in detail. Then the chapter passes to asymmetric information, and discusses principal-agent, moral hazard, adverse selection, screening, signalling, and the intuitive criterion of Cho and Kreps. The dependence of all these analyses on the assumption that agents choose on the basis of VNM expected utility, which does not seem to correspond to how people actually choose, is noticed.

Subgame-consistent cooperative solutions in randomly furcating stochastic dynamic games

In some economic processes in discrete-time, uncertainty may also arise. For instance, Smith and Zenou (Rev. Econ. Dyn. 6(1):54–79, 2003) considered a discrete-time stochastic job search model. Esteban-Bravo and Nogales (Comput. Oper. Res. 35:226–240, 2008) analyzed mathematical programming for stochastic discrete-time dynamics arising in economic systems, including examples in a stochastic national growth model and international growth model with uncertainty. The discrete-time counterpart of stochastic differential games is known as stochastic dynamic games. Basar and Ho (J. Econ. Theory 7:370–387, 1974) examined informational properties of the Nash solutions of stochastic nonzero-sum games. The elimination of the informational nonuniqueness in a Nash equilibrium through a stochastic formulation was first discussed in Basar (Int. J. Game Theory 5:65–90, 1976) and further examined in Basar (Automatica 11:547–551, 1975; In: New trends in dynamic system theory and economics, pp. 3–5, 1979; In: Dynamic policy games in economics, pp. 9–54, 1989). Basar and Mintz (In: Proceedings of the IEEE 11th conference on decision and control, pp. 188–192, 1972; Stochastics 1:25–69, 1973) and Basar (IEEE Trans. Autom. Control AC-23:233–243, 1978) developed an equilibrium solution of linear-quadratic stochastic dynamic games with noisy observation. Again, the SIAM Classics on Dynamic Noncooperative Game Theory by Basar and Olsder (Dynamic noncooperative game theory, 2nd edn. Academic Press, London, 1995) gave a comprehensive treatment of noncooperative stochastic dynamic games. Yeung and Petrosyan (J. Optim. Theory Appl. 145(3):579–596, 2010) provided the techniques in characterizing subgame consistent solutions for stochastic dynamic. Furthermore, they also presented a stochastic dynamic game in resource extraction. Analyses of noncooperative and cooperative discrete-time dynamic games with random game horizons were presented in Yeung and Petrosyan (J. Optim. Theory Appl. forthcoming, 2011). The recently emerging robust control techniques in discrete time along the lines of Hansen and Sargent (Robustness. Princeton University Press, Princeton, 2008) should prove to be fruitful in developing into stochastic dynamic interactive economic models.

Development of stochastic dynamic Nash game model for reservoir operation II. The value of players’ information availability and cooperative behaviors

This chapter deals with games in extensive form. Here an explicit evolution of the interaction is given, describing precisely when each player plays, what actions are available and what information is available to each player when he makes a decision. We start with games with perfect information (such as chess) and prove Zermelo’s theorem for finite games. We then consider infinite games a la Gale–Stewart: we show that open games are determined and that under the axiom of choice, there exists an undetermined game. Next we introduce games with imperfect information and prove Kuhn’s theorem, which states that mixed and behavioral strategies are equivalent in games with perfect recall. We present the standard characterization of Nash equilibria in behavioral strategies and introduce the basic refinements of Nash equilibria in extensive-form games: subgame-perfection, Bayesian perfect and sequential equilibria, which impose rational behaviors not only on the equilibrium path but also off-path. We prove the existence of sequential equilibrium (Kreps and Wilson). For normal form games as in Chap. 4 we introduce the standard refinements of Nash equilibrium: perfect equilibrium (Selten) and proper equilibrium (Myerson). We prove that a proper equilibrium of a normal form game G induces a sequential equilibrium in every extensive-form game with perfect recall having G as normal form. Finally we discuss forward induction and stability (Kohlberg and Mertens).

This paper studies generic properties of Markov perfect equilibria in dynamic stochastic games. We show that almost all dynamic stochastic games have a finite number of locally isolated Markov perfect equilibria. These equilibria are essential and strongly stable. Moreover, they all admit purification. To establish these results, we introduce a notion of regularity for dynamic stochastic games and exploit a simple connection between normal form and dynamic stochastic games.

We formulate cyber security problems with many strategic attackers and defenders as stochastic dynamic games with asymmetric information. We discuss solution approaches to stochastic dynamic games with asymmetric information and identify the difficulties/challenges associated with these approaches. We present a solution methodology for stochastic dynamic games with asymmetric information that resolves some of these difficulties. Our main results are based on certain key assumptions about the game model. Therefore, our methodology can solve only specific classes of cyber security problems. We identify classes of cyber security problems that our methodology cannot solve and connect these problems to open problems in game theory.

We study repeated games with imperfect public monitoring and unequal dis- counting. We characterize the limit set of perfect and public equilibrium payoffs as discount factors converge to 1 with the relative patience between players fixed. We show that the pairwise and individual full rank conditions are sufficient for the folk theorem. In this paper, we characterize the equilibrium payoffs in repeated games with imperfect public monitoring and unequal discounting as discount factors converge to 1 with rel- ative patience fixed. In particular, we show that the pairwise and individual full rank conditions are sufficient for the folk theorem. Lehrer and Pauzner (1999) (henceforth LP) analyze two-player repeated games with perfect monitoring and unequal discounting. They define the set of feasible and sequen- tially individually rational (henceforth SIR) payoffs and show that in two-player games with perfect monitoring, the limit set of subgame perfect equilibrium payoffs coincides with that of SIR payoffs as discount factors converges to 1 with the relative patience fixed (the folk theorem). Recently, Chen and Takahashi (2012) extend the result to n-player games with perfect monitoring. This paper extends their results to imperfect public monitoring. While the proofs of both Lehrer and Pauzner (1999 )a ndChen and Takahashi (2012) are constructive, we em- ploy a nonconstructive approach using the recursive structure of the perfect and public equilibrium (henceforth PPE). Specifically, we attain a characterization of the set of PPE payoffs as discount factors converge to1. In addition, we characterize SIR payoffs. Given these characterizations, we show that if the pairwise and individual full rank conditions are satisfied, these two sets coincide, that is, the folk theorem holds.

Modelling the strategic petroleum reserves of China and India by a stochastic dynamic game

Cooperative packet delivery can improve the data delivery performance in wireless networks by exploiting the mobility of the nodes, especially in networks with intermittent connectivity, high delay and error rates such as wireless mobile delay-tolerant networks (DTNs). For such a network, we study the problem of rational coalition formation among mobile nodes to cooperatively deliver packets to other mobile nodes in a coalition. Such coalitions are formed by mobile nodes which can be either well behaved or misbehaving in the sense that the well-behaved nodes always help each other for packet delivery, while the misbehaving nodes act selfishly and may not help the other nodes. A Bayesian coalitional game model is developed to analyze the behavior of mobile nodes in coalition formation in presence of this uncertainty of node behavior (i.e., type). Given the beliefs about the other mobile nodes' types, each mobile node makes a decision to form a coalition, and thus the coalitions in the network vary dynamically. A solution concept called Nash-stability is considered to find a stable coalitional structure in this coalitional game with incomplete information. We present a distributed algorithm and a discrete-time Markov chain (DTMC) model to find the Nash-stable coalitional structures. We also consider another solution concept, namely, the Bayesian core, which guarantees that no mobile node has an incentive to leave the grand coalition. The Bayesian game model is extended to a dynamic game model for which we propose a method for each mobile node to update its beliefs about other mobile nodes' types when the coalitional game is played repeatedly. The performance evaluation results show that, for this dynamic Bayesian coalitional game, a Nash-stable coalitional structure is obtained in each subgame. Also, the actual payoff of each mobile node is close to that when all the information is completely known. In addition, the payoffs of the mobile nodes will be at least as high as those when they act alone (i.e., the mobile nodes do not form coalitions).

We formulate and analyze dynamic games with d-step (d ≥ 1) delayed sharing information structure. The resulting game is a dynamic game of asymmetric information with hidden actions, imperfect observations, and controlled and interdependent system dynamics. We adopt common information based perfect Bayesian equilibrium (CIB-PBE) as the solution concept, and provide a sequential decomposition of the dynamic game. Such a decomposition leads to a backward induction algorithm to compute CIB-PBEs. We discuss the features of our approach to the above class of games and address the existence of CIB-PBEs.