Neuro-dynamic programming overview and a case study in optimal stopping

Abstract

We discuss some of the main ideas underlying neuro-dynamic programming. This methodology has significant potential as a general approach to approximately solving a wide variety of complex stochastic control problems. However, though neuro-dynamic programming algorithms have generated promising results in a number of applications, the algorithms that have been most successful are not well-understood at a theoretical level. As a case study in the development of theory in support of such algorithms, we propose an algorithm for solving optimal stopping problems, and we provide theoretical results concerning convergence and approximation error. Though this algorithm is customized for optimal stopping problems, it retains the key features seen in the most popular neuro-dynamic programming methods, and its analysis might therefore serve as a starting point for the study of methods of broader scope. The applicability of the algorithm is illustrated through a computational case study involving the pricing of a path-dependent financial derivative security that gives rise to an optimal stopping problem with a one-hundred-dimensional state space.