Existence of optimal stationary policies in discounted Markov decision processes: approaches by occupation measures

Abstract

We consider discounted Markov decision processes with general state and action spaces and bounded Borel measurable cost functions. We extend the idea of occupation measures given in Borkar [1]. And constructing the occupation measure associated with a sequence of policies under a system-assumption, we prove the existence theorem of optimal stationary policies. Here, the assumptions of continuity and compactness are excluded and the discussion is done in the class of Borel measurable policies.