A remark on the Bellman equation for optimal control problems with exit times and noncoercing dynamics

Abstract

This note continues my work (1997) on uniqueness questions for viscosity solutions of Hamilton-Jacobi-Bellman equations (HJBs) arising from deterministic control problems with exit times. I prove a general uniqueness theorem characterizing the value functions for a class of problems of this type for nonlinear systems as the unique solutions of the corresponding HJBs among continuous functions with appropriate boundary conditions when the dynamical law is non-Lipschitz and noncoercing. The class includes Sussmann's (1996) reflected brachystochrone problem (RBP), as well as problems with unbounded nonlinear running cost functionals. I show that the RBP value function is the unique viscosity solution of the corresponding HJB among the continuous functions which vanish on the target and which are bounded below. Value function characterizations of this kind have been studied by many authors for a large number of stochastic and deterministic optimal control problems. However, these earlier characterizations assume the dynamics are coercing and positive lower bounds on the running cost functionals and therefore do not apply to many standard problems. Our work is part of a larger research program which extends uniqueness results from viscosity theory to versions covering well-known optimal control problems with unbounded cost functionals or dynamics that do not have uniqueness of solutions.