Empirical Dynamic Programming for Controlled Diffusion Processes

Abstract

We consider Markov chain approximation for optimal control of diffusion processes under infinite horizon discounted cost optimality and apply the simulation-based Empirical Value Iteration to estimate the value function of each approximating chain. We follow a nested multi-grid discretization of the state space to establish weak convergence of the value function sequence to the value function of the original controlled diffusion. We illustrate the convergence performance of the model on the popular Benes’ bang-bang control problem [Beneš (1974)].