Comparing recent assumptions for the existence of average optimal stationary policies

Abstract

We consider discrete time average cost Markov decision processes with countable state space and finite action sets. Conditions recently proposed by Borkar, Cavazos-Cadena, Weber and Stidham, and Sennott for the existence of an expected average cost optimal stationary policy are compared. The conclusion is that the Sennott conditions are the weakest. We also give an example for which the Sennott axioms hold but the others fail.