On the Existence and Uniqueness of Equilibrium Situations in Markovian Games with Discounting

Abstract

Markovian (stochastic) two-person games with discounting are considered. It is proved that if the set of states of such a game and the set of decisions of the players are finite, then the game has values and both players have optimal stationary strategies. The proof, which is based on the principle of contracting mappings, is constructive and leads to a recurrent algorithm of finding solutions of the game. The question of uniqueness of an equilibrium situation in the game considered is also discussed. In addition to the Markovian game with discounting in the context of the principle of contracting mappings, its subgame, namely, a Markovian decision process with discounting is also studied.