Mean–semivariance optimality for continuous-time Markov decision processes

Abstract

In this paper we study the mean–semivariance problem for continuous-time Markov decision processes with Borel state and action spaces and unbounded cost and transition rates. The optimality criterion is to minimize the semivariance of the discounted total cost over the set of all policies satisfying the constraint that the mean of the discounted total cost is equal to a given function. Under reasonable conditions, we show that the semivariance optimal value function is a solution to the optimality equation of the mean–semivariance criterion by an iteration approach. Moreover, we obtain the existence of mean–semivariance optimal policies from the optimality equation. Furthermore, we give a value iteration algorithm to compute approximately an optimal policy and the optimal value, and analyze the convergence of the algorithm.