A two-person zero-sum Markov game with a stopped set

Abstract

We consider a two-person zero-sum Markov game with continuous time up to the time that the game process goes into a fixed subset of a countable state space, this subset is called a stopped set of the game. We show that such a game with a discount factor has optimal value function and both players will have their optimal stationary strategies. The same result is proved for the case of a nondiscounted Markov game under some additional conditions, that is a reward rate function is nonnegative and the first time τ (entrance time) of the game process going to the stopped set is finite with probability one (i.e., p( τ < ∞) = 1). It is remarkable that in the case of a nondiscounted Markov game, if the expectation of the entrance time is bounded, and the reward rate function need not be nonnegative, then the same result holds.