Equilibria In Stochastic Games Research Articles

$\varepsilon$-Equilibria for Stochastic Games with Uncountable State Space and Unbounded Costs

We study a class of noncooperative stochastic games with unbounded cost functions and an uncountable state space. It is assumed that the transition law is absolutely continuous with respect to some probability measure on the state space. Undiscounted stochastic games with expected average costs are considered first. It is shown under a uniform geometric ergodicity assumption that there exists a stationary $\varepsilon$-equilibrium for each $\varepsilon > 0 $. The proof is based on recent results on uniform bounds for convergence rates of Markov chains [S. P. Meyn and R. L. Tweedie, Ann. Appl. Probab., 4 (1994), pp. 981--1011] and on an approximation method similar to that used in [A. S. Nowak, J. Optim. Theory Appl., 45 (1985), pp. 591--602], where an $\varepsilon$-equilibrium in stationary policies was shown to exist for the bounded discounted costs. The stochastic game is approximated by one with a countable state space for which a stationary Nash equilibrium exists (see [E. Altman, A. Hordijk, and F. M. Spieksma, Math. Oper. Res., 22 (1997), pp. 588--618]); this equilibrium determines an $\epsilon$-equilibrium for the original game. Finally, new results for the existence of stationary $\varepsilon$-equilibrium for discounted stochastic games are given.

Characterization of correlated equilibria in stochastic games

A general communication device is a device that at every stage of the game receives a private message from each player, and in return sends a private signal to each player; the signals the device sends depend on past play, past signals it sent, and past messages it received. An autonomous correlation device is a general communication device where signals depend only on past signals the device sent, but not on past play or past messages it received. We show that the set of all equilibrium payoffs in extended games that include a general communication device coincides with the set of all equilibrium payoffs in extended games that include an autonomous correlation device. A stronger result is obtained when the punishment level is independent of the history.

Average-discounted equilibria in stochastic games

In stochastic games with finite state and action spaces, we examine existence of equilibria where player 1 uses the limiting average reward and player 2 a discounted reward for the evaluations of the respective payoff sequences. By the nature of these rewards, the far future determines player 1's reward, while player 2 is rather interested in the near future. This gives rise to a natural cooperation between the players along the course of the play. First we show the existence of stationary ε-equilibria, for all ε>0, in these games. However, besides these stationary ε-equilibria, there also exist ε-equilibria, in terms of only slightly more complex ultimately stationary strategies, which are rather in the spirit of these games because, after a large stage when the discounted game is not interesting any longer, the players cooperate to guarantee the highest feasible reward to player 1. Moreover, we analyze an interesting example demonstrating that 0-equilibria do not necessarily exist in these games, not even in terms of history dependent strategies. Finally, we examine special classes of stochastic games with specific conditions on the transition and payoff structures. Several examples are given to clarify all these issues.

Cyclic Markov equilibria in stochastic games

We examine a three-person stochastic game where the only existing equilibria consist of cyclic Markov strategies. Unlike in two-person games of a similar type, stationary ε-equilibria (ε > 0) do not exist for this game. Besides we characterize the set of feasible equilibrium rewards.

Cyclic Markov Equilibria in Stochastic Games

We examine a three-person stochastic game where the only existing equilibria consist of cyclic Markov strategies. Unlike in two-person games of a similar type, stationary ε-equilibria (ε > 0) do not exist for this game. Besides we characterize the set of feasible equilibrium rewards.

Optimality and Equilibria in Stochastic Games

"Optimality and Equilibria in Stochastic Games." Journal of the Operational Research Society, 45(1), pp. 116–117

Optimality and Equilibria in Stochastic Games.

Optimality and Equilibria in Stochastic Games.

Optimality and Equilibria in Stochastic Games (F. Thuijsman)

Optimality and Equilibria in Stochastic Games (F. Thuijsman)

Perfect equilibria in stochastic games

We examine stochastic games with finite state and action spaces. For the β-discounted case, as well as for the irreducible limiting average case, we show the existence of trembling-hand perfect equilibria and give characterizations of those equilibria. In the final section, we give an example which illustrates that the existence of stationary limiting average equilibria in a nonirreducible stochastic game does not imply the existence of a perfect limiting average equilibrium.

Nonlinear programming and stationary equilibria in stochastic games

Stationary equilibria in discounted and limiting average finite state/action space stochastic games are shown to be equivalent to global optima of certain nonlinear programs. For zero sum limiting average games, this formulation reduces to a linear objective, nonlinear constraints program, which finds the “best” stationary strategies, even whene-optimal stationary strategies do not exist, for arbitrarily smalle.