Constrained Continuous-Time Markov Control Processes with Discounted Criteria

Abstract

In this paper we study constrained continuous-time Markov control processes with a denumerable state space and unbounded reward/cost and transition rates. The criterion to be maximized is an expected discounted reward, and the constraint is imposed on an expected discounted cost. We give conditions that ensure the existence of constrained-optimal policies. We also show that a constrained-optimal policy may be a stationary policy or a randomized stationary policy that randomizes between two stationary policies which differ in at most one state. Our results are illustrated with a controlled queueing system.