Semi-Markov decision processes with a reachable state-subset

Abstract

We consider the problem of minimizing the long-run average expected cost per unit time in a semi-Markov decision process with arbitrary state and action space, Assuming the existence .of a Borel subset of state space called a reachable state-subset, we derive the optimality equation for the unbounded costs. The contraction property [8; 9] for the average case is used, so that the assumptions of both continuity of the one-step cost function and compactness of state and action space are excluded.