Stationary Markovian Decision Problems and Perturbation Theory of Quasi-Compact Linear Operators

Abstract

In this paper stationary Markov decision problems are considered with arbitrary state space and compact space of strategies. Conditions are given for the existence of an average optimal strategy. This is done by using the fact that a continuous function on a compact space attains its minimum. To prove the continuity of the average costs as function on the space of strategies some perturbation results for quasi-compact linear operators are used. In a first set of conditions the boundedness of the one-period cost functions and the quasi-compactness of the Markov processes are assumed. In more general conditions the boundedness of the cost functions is replaced by the boundedness, on a subset A of the state space, of the recurrence time and costs until A and the quasi-compactness of the Markov processes are replaced by the quasi-compactness of the embedded Markov processes on A.