Discretisation of continuous-time stochastic optimal control problems with delay

Abstract

In the present work, we study discretisation schemes for continuous-time stochastic optimal control problems with time delay. The dynamics of the control problems to be approximated are described by controlled stochastic delay (or functional) differential equations. The value functions associated with such control problems are defined on an infinite-dimensional function space. The discretisation schemes studied are obtained by replacing the original control problem by a sequence of approximating discrete-time Markovian control problems with finite or finite-dimensional state space. Such a scheme is convergent if the value functions associated with the approximating control problems converge to the value function of the original problem. Following a general method for the discretisation of continuous-time control problems, sufficient conditions for the convergence of discretisation schemes for a class of stochastic optimal control problems with delay are derived. The general method itself is cast in a formal framework. A semi-discretisation scheme for a second class of stochastic optimal control problems with delay is proposed. Under standard assumptions, convergence of the scheme as well as uniform upper bounds on the discretisation error are obtained. The question of how to numerically solve the resulting discrete-time finite-dimensional control problems is also addressed.