The asymptotic behaviour of the minimal total expected cost for the denumerable state Markov decision model

Abstract

This paper considers the discrete time Markov decision model with a denumerable state space and finite action space. Under certain conditions it is proved that the minimal total expected cost for a planning horizon of n epochs minus n times the minimal long-run average expected cost per unit time has a finite limit as n → ∞ for each initial state.