Almost Stationary ∈-Equilibria in Zero-Sum Stochastic Games

Abstract

We show the existence of almost stationary ∈-equilibria, for all ∈ > 0, in zero-sum stochastic games with finite state and action spaces. These are ∈-equilibria with the property that, if neither player deviates, then stationary strategies are played forever with probability almost 1. The proof is based on the construction of specific stationary strategy pairs, with corresponding rewards equal to the value, which can be supplemented with history-dependent δ-optimal strategies, with small δ > 0, in order to obtain almost stationary ∈-equilibria.