$\varepsilon$-Equilibria for Stochastic Games with Uncountable State Space and Unbounded Costs

Abstract

We study a class of noncooperative stochastic games with unbounded cost functions and an uncountable state space. It is assumed that the transition law is absolutely continuous with respect to some probability measure on the state space. Undiscounted stochastic games with expected average costs are considered first. It is shown under a uniform geometric ergodicity assumption that there exists a stationary $\varepsilon$-equilibrium for each $\varepsilon > 0 $. The proof is based on recent results on uniform bounds for convergence rates of Markov chains [S. P. Meyn and R. L. Tweedie, Ann. Appl. Probab., 4 (1994), pp. 981--1011] and on an approximation method similar to that used in [A. S. Nowak, J. Optim. Theory Appl., 45 (1985), pp. 591--602], where an $\varepsilon$-equilibrium in stationary policies was shown to exist for the bounded discounted costs. The stochastic game is approximated by one with a countable state space for which a stationary Nash equilibrium exists (see [E. Altman, A. Hordijk, and F. M. Spieksma, Math. Oper. Res., 22 (1997), pp. 588--618]); this equilibrium determines an $\epsilon$-equilibrium for the original game. Finally, new results for the existence of stationary $\varepsilon$-equilibrium for discounted stochastic games are given.