Algorithms for singularly perturbed limiting average Markov control problems

Abstract

A singularly perturbed Markov decision process with the limiting average reward criterion is considered. It is assumed that the underlying process is composed of n separate irreducible processes, and that the small perturbation is such that it unites these processes into a single irreducible process. Two algorithms for the solution of the underlying limit Markov control problem are presented. The first of these is a linear program possessing the Wolfe-Dantzig structure inherited from the ergodic 'nearly decomposable' assumption in the model. The second is an aggregation-disaggregation policy improvement algorithm. >