Continuity of the optimal average cost in Markov decision chains with small risk-sensitivity

Abstract

This note concerns discrete-time controlled Markov chains driven by a decision maker with constant risk-sensitivity \(\lambda \). Assuming that the system evolves on a denumerable state space and is endowed with a bounded cost function, the paper analyzes the continuity of the optimal average cost with respect to the risk-sensitivity parameter, a property that is promptly seen to be valid at each no-null value of \(\lambda \). Under standard continuity-compactness conditions, it is shown that a general form of the simultaneous Doeblin condition allows to establish the continuity of the optimal average cost at \(\lambda = 0\), and explicit examples are given to show that, even if every state is positive recurrent under the action of any stationary policy, the above continuity conclusion can not be ensured under weaker recurrence requirements, as the Lyapunov function condition.