On the Existence of Average Optimal Policies in Semiregenerative Decision Models

Abstract

This paper gives sufficient conditions for the existence of average optimal policies in a decision model, which is a generalization of a denumerable state space semi-Markov decision model. The conditions extend most of those previously known; especially, neither uni-chainedness nor communicatingness is assumed. An average optimal policy can be obtained as a limit of discount optimal policies. If structured policies are discount optimal, there exists a structured average optimal policy. The results are applicable to more general continuous time models such as the one by Yushkevich and Fainberg (Yushkevich, A. A., E. A. Fainberg. 1979. On homogeneous Markov models with continuous time and finite or countable state space. Theory Probab. Appl. 24 156–161.).