Value iteration for average cost Markov decision processes inBorel spaces

Abstract

This paper deals with the value iteration algorithm (VIA) for average cost Markov decision processes in Borel state and action spaces. The costs may have neither upper nor lower bounds, instead of the case of nonnegative (or bounded below) costs widely used in the previous literature. We propose aset of conditions which is weaker than those in the previous literature. Under these conditions, we first establish the average cost optimality equation. Then under an additional condition, we show that the VIA yields the optimal (minimum) average cost, anaverage optimal stationary policy, and a solution to the average cost optimality equation. Finally, we use an example to illustrate our conditions.