Blackwell Optimality in Borelian Continuous-in-Action Markov Decision Processes

Abstract

We prove the existence of stationary Blackwell optimal policies in Markov decision processes with a Borel state space, compact action sets, and continuous-in-action and bounded transition densities and rewards, satisfying a simultaneous Doeblin-type condition. The proof is based on a compactification of the randomized stationary policy space in a weak-strong topology, on the continuity of Laurent coefficients of the discounted rewards in this topology, and on a lexicographical policy improvement. Until now similar results were obtained for the models with a denumerable state space or with a Borel state space and finite action sets.