Finite-state approximation of Markov decision processes with unbounded costs and Borel spaces

Abstract

Computing optimal policies for stochastic control problems with general state and action spaces is often intractable. This paper studies finite-state approximations of discrete time Markov decision processes (MDPs) with Borel state and action spaces and unbounded one-stage cost function, for both discounted and average cost criteria. Under mild technical assumptions, it is shown that stationary optimal policies obtained from the solutions to finite-state models can approximate an optimal stationary policy with arbitrary precision. A simulation example is provided.