Nonzero-Sum Expected Average Discrete-Time Stochastic Games: The Case of Uncountable Spaces

Abstract

In this paper we study nonzero-sum discrete-time stochastic games with an uncountable state space and Borel action spaces under the expected average payoff criterion. The reward functions can be possibly unbounded and the transition law is a convex combination of finitely many probability measures dependent on the state variable and dominated by some probability measure on the state space. We introduce several auxiliary static game models and obtain their properties. Moreover, by a technique of extending the space state, we introduce auxiliary stochastic game models and derive the uniform geometric ergodicity of Markov chains taking values in the extended state space. Furthermore, we show the existence of a stationary almost Markov Nash equilibrium via an approximation method. Finally, we use a resource extraction model to illustrate the main results.