Continuous time markov decision processes with nonuniformly bounded transition rate: expected total rewards

Abstract

This paper deals with a stationary continuous time Markov decision process with countable state space, nonuniformly bounded transition rate and extended real valued reward, under the criterion of expected total rewards. The model is transformed into an equivalent discrete time Markov decision process within stochastic stationary policies when the model is well-defined. Thus most results, for example, the optimality equation and the existence of ε- optimal policies, in the discrete time Markov decision process may be generalized directly to hold for the continuous time Markov decision process. Furthermore, the state space is partitioned into three subsets: where the optimal value equals to positive infinity, negative infinity, or is finite, respectively