A mean-variance optimization problem for continuous-time Markov decision processes

Abstract

This paper deals with the mean-variance optimization problem for continuous-time Markov decision processes in Polish spaces. The transition and reward rates are allowed to be unbounded, and the paper focuses on an optimality criterion that improves the usual excepted (or mean) discounted reward criterion. Especially, we aim to nd the conditions for the existence of a mean-variance optimal policy under the Polish spaces. First, under suitable conditions, we prove that the variance minimization problem can be transformed into an equivalent discounted-cost optimization problem by using the so-called rst passage decomposition method. Then, we obtain the so-called mean-variance optimality equation and the existence of a mean-variance optimal policy that minimizes the variance over the set of policies with optimal reward. Finally, we present some examples to illustrate our results.