Denumerable Markov Decision Processes Research Articles

Conditions for existence of average and Blackwell optimal stationary policies in denumerable Markov decision processes

Sufficiency conditions are given for the existence of stationary Blackwell optimal policies. These conditions concern only the one step transition probability matrices and their resolvents and therefore appear to be more natural than those recently introduced by Dekker and Hordijk. We use the same operator-theoretic approach and standard results about spectral theory of bounded linear operators.

Strong 1-optimal stationary policies in denumerable Markov decision processes

Usual conditions for existence of stationary average optimal policies in denumerable MDPs with general bounded rewards are shown to be also sufficient for strong 1-optimality. Moreover, we prove that all limit points of discounted optimal stationary policies when the discount factor goes to 1 are strong 1-optimal.