Zero-sum Case Research Articles

Zero and non-zero sum risk-sensitive Semi-Markov games

In this article we consider zero and non-zero sum risk-sensitive average criterion games for semi-Markov processes with a finite state space. For the zero-sum case, under suitable assumptions we show that the game has a value. We also establish the existence of a stationary saddle point equilibrium. For the non-zero sum case, under suitable assumptions we establish the existence of a stationary Nash equilibrium.

Dynamic Set Values for Nonzero-Sum Games with Multiple Equilibriums

Nonzero sum games typically have multiple Nash equilibriums (or no equilibrium), and unlike the zero-sum case, they may have different values at different equilibriums. Instead of focusing on the existence of individual equilibriums, we study the set of values over all equilibriums, which we call the set value of the game. The set value is unique by nature and always exists (with possible value [Formula: see text]). Similar to the standard value function in control literature, it enjoys many nice properties, such as regularity, stability, and more importantly, the dynamic programming principle. There are two main features in order to obtain the dynamic programming principle: (i) we must use closed-loop controls (instead of open-loop controls); and (ii) we must allow for path dependent controls, even if the problem is in a state-dependent (Markovian) setting. We shall consider both discrete and continuous time models with finite time horizon. For the latter, we will also provide a duality approach through certain standard PDE (or path-dependent PDE), which is quite efficient for numerically computing the set value of the game.

Monotone bargaining is Nash-solvable

Given two finite ordered sets A and B, let O=A×B denote the set of outcomes of the following game: Two players, Alice and Bob, have the sets of strategies X and Y that consist of all monotone non-decreasing mappings x:A→B and y:B→A, respectively. It is easily seen that each pair (x,y)∈X×Y produces at least one deal, that is, an outcome (a,b)∈O such that x(a)=b and y(b)=a. Denote by G(x,y)⊆O the set of all such deals related to (x,y). The obtained mapping G:X×Y→2O is a game correspondence. Choose an arbitrary deal g(x,y)∈G(x,y) to obtain a mapping g:X×Y→O, which is a game form. We show that each such game form is tight and, hence, Nash-solvable, that is, for any pair u=(uA,uB) of utility functions of Alice and Bob, the obtained monotone bargaining game (g,u) has at least one Nash equilibrium (x,y) in pure strategies. Moreover, |G(x,y)|=1 and, hence, (x,y) is a Nash equilibrium in game (g,u) for all g∈G. We also obtain an efficient algorithm that determines such an equilibrium in time quadratic in |O|, although the numbers of strategies are exponential in |O|. Our results show that, somewhat surprisingly, the players have no need to hide or randomize their bargaining strategies, even in the zero-sum case.

Backward induction in presence of cycles

For the classical backward induction algorithm, the input is an arbitrary $n$-person positional game with perfect information modeled by a finite acyclic directed graph (digraph) and the output is a profile $(x_1, \ldots, x_n)$ of pure positional strategies that form some special subgame perfect Nash equilibrium. We extend this algorithm to work with digraphs that may have directed cycles. Each digraph admits a unique partition into strongly connected components, which will be treated as the outcomes of the game. Such a game will be called a {\em deterministic graphical multistage}(DGMS) game. If we identify the outcomes corresponding to all strongly connected components, except terminal positions, we obtain the so-called {\em deterministic graphical}(DG) games, which are frequent in the literature. The outcomes of a DG game are all terminal positions and one special outcome $c$ that is assigned to all infinite plays. We modify the backward induction procedure to adapt it for the DGMS games. However, by doing so, we lose two important properties: the modified algorithm always outputs a {\em Nash equilibrium} (NE) only when $n = 2$ and, even in this case, this NE may be not {\em subgame perfect}. (Yet, in the zero-sum case it is.) The lack of these two properties is not a fault of the algorithm, just (subgame perfect) Nash equilibria in pure positional strategies may fail to exist in the considered game. {\bf Keywords:} deterministic graphical (multistage) game, game in normal and in positional form, saddle point, Nash equilibrium, Nash-solvability, game form, positional structure, directed graph, digraph, directed cycle, acyclic digraph.

A Tauberian Theorem for Nonexpansive Operators and Applications to Zero-Sum Stochastic Games

We prove a Tauberian theorem for nonexpansive operators and apply it to the model of zero-sum stochastic game. Under mild assumptions, we prove that the value of the λ-discounted game converges uniformly when λ goes to zero if and only if the value of the n-stage game converges uniformly when n goes to infinity. This generalizes the Tauberian theorem of Lehrer and Sorin [Lehrer E, Sorin S (1992) A uniform Tauberian theorem in dynamic programming. Math. Oper. Res. 17(2):303–307] to the two-player zero-sum case. We also provide the first example of a stochastic game with public signals on the state and perfect observation of actions, with finite state space, signal sets, and action sets, in which for some initial state known by both players, the value of the λ-discounted game and the value of the n-stage game starting at that initial state converge to distinct limits.

Forward–Backward Stochastic Differential Games and Stochastic Control under Model Uncertainty

We study optimal stochastic control problems with jumps under model uncertainty. We rewrite such problems as stochastic differential games of forward---backward stochastic differential equations. We prove general stochastic maximum principles for such games, both in the zero-sum case (finding conditions for saddle points) and for the nonzero sum games (finding conditions for Nash equilibria). We then apply these results to study robust optimal portfolio-consumption problems with penalty. We establish a connection between market viability under model uncertainty and equivalent martingale measures. In the case with entropic penalty, we prove a general reduction theorem, stating that a optimal portfolio-consumption problem under model uncertainty can be reduced to a classical portfolio-consumption problem under model certainty, with a change in the utility function, and we relate this to risk sensitive control. In particular, this result shows that model uncertainty increases the Arrow---Pratt risk aversion index.

Orderfield property of mixtures of stochastic games

We consider certain mixtures, Γ, of classes of stochastic games and provide sufficient conditions for these mixtures to possess the orderfield property. For 2-player zero-sum and non-zero sum stochastic games, we prove that if we mix a set of states S1 where the transitions are controlled by one player with a set of states S2 constituting a sub-game having the orderfield property (where S1 ∩ S2 = ∅), the resulting mixture Γ with states S = S1 ∪ S2 has the orderfield property if there are no transitions from S2 to S1. This is true for discounted as well as undiscounted games. This condition on the transitions is sufficient when S1 is perfect information or SC (Switching Control) or ARAT (Additive Reward Additive Transition). In the zero-sum case, S1 can be a mixture of SC and ARAT as well. On the other hand,when S1 is SER-SIT (Separable Reward — State Independent Transition), we provide a counter example to show that this condition is not sufficient for the mixture Γ to possess the orderfield property. In addition to the condition that there are no transitions from S2 to S1, if the sum of all transition probabilities from S1 to S2 is independent of the actions of the players, then Γ has the orderfield property even when S1 is SER-SIT. When S1 and S2 are both SERSIT, their mixture Γ has the orderfield property even if we allow transitions from S2 to S1. We also extend these results to some multi-player games namely, mixtures with one player control Polystochastic games. In all the above cases, we can inductively mix many such games and continue to retain the orderfield property.

Nonzero-sum games for continuous-time Markov chains with unbounded discounted payoffs

In this paper, we study two-person nonzero-sum games for continuous-time Markov chains with discounted payoff criteria and Borel action spaces. The transition rates are possibly unbounded, and the payoff functions might have neither upper nor lower bounds. We give conditions that ensure the existence of Nash equilibria in stationary strategies. For the zero-sum case, we prove the existence of the value of the game, and also provide arecursiveway to compute it, or at least to approximate it. Our results are applied to a controlled queueing system. We also show that if the transition rates areuniformly bounded, then a continuous-time game is equivalent, in a suitable sense, to a discrete-time Markov game.

Nonzero-sum games for continuous-time Markov chains with unbounded discounted payoffs

In this paper, we study two-person nonzero-sum games for continuous-time Markov chains with discounted payoff criteria and Borel action spaces. The transition rates are possibly unbounded, and the payoff functions might have neither upper nor lower bounds. We give conditions that ensure the existence of Nash equilibria in stationary strategies. For the zero-sum case, we prove the existence of the value of the game, and also provide arecursiveway to compute it, or at least to approximate it. Our results are applied to a controlled queueing system. We also show that if the transition rates areuniformly bounded, then a continuous-time game is equivalent, in a suitable sense, to a discrete-time Markov game.

EQUILIBRIUM STRATEGIES IN STOCHASTIC GAMES WITH ADDITIVE COST AND TRANSITION STRUCTURE

Two-person stochastic games with additive transition and cost structure and the criterion of expected total costs are treated. State space and action spaces are standard Borel, and unbounded costs are allowed. For the zero-sum case, it is shown that there are stationary deterministic εη-optimal strategies for every ε>0 and a certain weight function η if some semi-continuity and compactness conditions are fulfilled. Using these results, the existence of so-called quasi-stationary deterministic εη-equilibrium strategy pairs under corresponding conditions is proven.

Stochastic differential games with multiple modes

We have studied two person stochastic differential games with multiple modes. For the zero-sum game we have established the existence of optimal strategies for both players. For the non zerosum case we have proved the existence of a Nash equilibrium

Nonzero-sum stochastic games with unbounded costs: Discounted and average cost cases

We treat non-cooperative stochastic games with countable state space and with finitely many players each having finitely many moves available in a given state. As a function of the current state and move vector, each player incurs a nonnegative cost. Assumptions are given for the expected discounted cost game to have a Nash equilibrium randomized stationary strategy. These conditions hold for bounded costs, thereby generalizing Parthasarathy (1973) and Federgruen (1978). Assumptions are given for the long-run average expected cost game to have a Nash equilibrium randomized stationary strategy, under which each player has constant average cost. A flow control example illustrates the results. This paper complements the treatment of the zero-sum case in Sennott (1993a).

Algorithms for stochastic games ? A survey

We consider finite state, finite action, stochastic games over an infinite time horizon. We survey algorithms for the computation of minimax optimal stationary strategies in the zerosum case, and of Nash equilibria in stationary strategies in the nonzerosum case. We also survey those theoretical results that pave the way towards future development of algorithms.

A class of stochastic games with ordered field property

It is shown that discounted general-sum stochastic games with two players, two states, and one player controlling the rewards have the ordered field property. For the zero-sum case, this result implies that, when starting with rational data, also the value is rational and that the extreme optimal stationary strategies are composed of rational components.

Portfolio decisions as games

Efficient decisions in portfolio models are modelled here as two-person games, which may be both zero-sum and non-zero-sum. In the zero-sum case, mixed strategy solutions are characterized in different ways in both static and dynamic games and it is shown that the traditional solutions of mean variance portfolio theory can be generalized considerably.

Noncooperative stochastic games under a n-stage local contraction assumption

We consider a class of noncooperative stochastic games with general state and action spaces and with a state dependent discount factor. The expected time duration between any two stages of the game is not bounded away from zero, so that the usual N-stage contraction assumption, uniform over all admissible strategies, does not hold. We propose milder sufficient regularity conditions, allowing strategies that give rise with probability one to any number of simultaneous stages. We give sufficient conditions for the existence of equilibrium and ∈-equilibrium stationary strategies in the sense of Nash. In the two-player zero-sum case, when an equilibrium strategy exists, the value of the game is the unique fixed point of a specific functional operator and can be computed by dynamic programming.

Sequential Stackelberg equilibria in two-person games

The concept of sequential Stackelberg equilibrium is introduced in the general framework of dynamic, two-person games defined in the Denardo contracting operator formalism. A relationship between this solution concept and the sequential Nash equilibrium for an associated extended game is established. This correspondence result, which can be related to previous results obtained by Basar and Haurie (1984), is then used for studying the existence of such solutions in a class of sequential games. For the zero-sum case, the sequential Stackelberg equilibrium corresponds to a sequential maxmin equilibrium. An algorithm is proposed for the computation of this particular case of equilibrium.

Noncooperative Stochastic Games

We introduce a sequential competitive decision process that is a generalization of noncooperative finite games and of two-person zero-sum stochastic games (hence, of Markovian decision processes). We prove the existence of equilibrium points under criteria of discounted gain and of average gain. Two person zero-sum stochastic games and noncooperative finite games were introduced in elegant papers by Shapley [22] and Nash [16], [17]. Shapley's work prompted a series of papers [1], [4], [5], [10], [11], [12], [14], [18], [26] concerned with the existence of minimax solutions and algorithms for their computation. Even for the two-person zero-sum case, no finite algorithm yet exists. Nash's papers led to a sizeable literature in both mathematics and economics. Mills' [15] work, for example, is related to our characterization of equilibrium points in Section 4. Noncooperative stochastic games may yield fruitful models for several phenomena in the social sciences. Theories of economic markets, for example, have increasingly sought to encompass sequential economic decision processes. Some recent research in social psychology has taken an analogous direction [19], [25]. I became aware of recent work by Rogers [20] shortly after completing this paper. His results and ours nearly coincide with our Theorem 2 being slightly stronger than the comparable results in his paper. The basic difference between the papers is that Rogers relies on the Kakutani fixed point theorem whereas we use Brouwer's theorem. Our arguments are somewhat simpler as a consequence.