Ergodic control of continuous-time Markov chains with pathwise constraints

Abstract

We deal with unichain ergodic continuous-time controlled Markov chains with a denumerable state space, and unbounded reward and transition rates. We are concerned with finding control policies that maximize a sample-path average reward over the family of policies for which a certain pathwise average cost is below a given value with probability one. We illustrate our results with examples on the control of a queueing system and an epidemic process.