Optimal Average Value Convergence in Nonhomogeneous Markov Decision Processes

Abstract

We address the undiscounted nonhomogeneous Markov decision process with average reward criterion and prove two structural results. First, we establish equivalence of this problem to a discounted Markov decision process by means of an ergodic coefficient embedded in the original problem. Second, we prove, for the original problem, that the optimal finite horizon average values converge to the infinite horizon optimal average value under an ergodic condition.