On the existence of strict optimal controls for constrained, controlled Markov processes in continuous time

Abstract

Closedness and convexity conditions are identified under which optimal controls in the class of strict controls exist for a large class of stochastic processes under infinite-horizon discounted, long-term average, first exit, finite-horizon and discretionary stopping criteria in the presence of hard and/or soft constraints. The results are more general than results obtained by Haussmann and Lepeltier (SIAM J. Control Optim. 28 (1990), pp. 851–902) for a controlled diffusion under a mixed optimal-stopping/finite-horizon/first exit criterion. The approach taken in this paper is to utilize equivalent linear programming (LP) formulations of the control problems which provides a unified LP formulation for the problems. The conditions of Haussmann and Lepeltier are shown to imply the sufficient conditions of this paper when the process is a controlled diffusion. Simpler conditions are also identified for Markov chains, simple Markov jump processes, diffusions with jumps, regime-switching diffusions and solutions to Lévy stochastic differential equations.