Finite state Markov decision models with average reward criteria

Abstract

This paper deals with a discrete time Markov decision model with a finite state space, arbitrary action space, and bounded reward function under the average reward criteria. We consider four average reward criteria and prove the existence of persistently nearly optimal strategies in various classes of strategies for models with complete state information. We show that such strategies exist in any class of strategies satisfying the following condition: along any trajectory at different epochs the controller knows different information about the past. Though neither optimal nor stationary nearly optimal strategies may exist, we show that for some nonempty set of states the described nearly optimal strategies may be chosen either stationary or optimal.