Mean Variance Optimization in Markov Decision Processes with State-Dependent Risk Aversion

Abstract

In many revenue management applications risk aversion is crucial. In dynamic decision problems it is challenging to find the right balance between maximizing expected profits and minimizing their variance. In this paper, we present an efficient dynamic programming approach to leverage mean variance optimization in discounted Markov decision processes with finite time horizon. We show how to derive optimal risk-sensitive policies which allow for state-dependent mean variance balancing. We compare our approach to risk-neutral policies as well as risk-averse policies of the exponential utility model. Based on state-probabilities, we analytically evaluate the expected evolution of accumulated profits as well as profit variances. Our general concepts are demonstrated for dynamic pricing applications.