Notes on variance in randomized reward Markov decision processes

Abstract

In Markov decision processes with randomized reward function, countable states and finite actions, this paper studies total variance of average rewards. The variance is represented explicitly in a different form from the deterministic reward case. By parallel arguments to the case, we show the existence of a stationary deterministic policy which is mean-variance optimal. The recurrent condition used in the existence theorem is examined.