Optimal and Nearly Optimal Policies in Markov Decision Chains with Nonnegative Rewards and Risk-Sensitive Expected Total-Reward Criterion

Abstract

This work considers Markov decision processes with discrete state space. Assuming that the decision maker has a non-null constant risk-sensitivity, which leads to grade random rewards via the expectation of an exponential utility function, the performance index of a control policy is the risk-sensitive expected total-reward criterion corresponding to a nonnegative reward function. Within this framework, the existence of optimal and approximately optimal stationary policies in the absolute sense is studied. The main results can be summarised as follows: (i) An optimal stationary policy exists if the state and actions sets are finite, whereas an e-optimal stationary policy is guaranteed when just the state space is finite. (ii) This latter fact is used to obtain, for the general denumerable state space case, that e-optimal stationary policies exist if the controller is risk-seeking and the optimal value function is bounded.