A direct method for solving stochastic control problems

Abstract

In this paper a method motivated by completion of squares is used to describe explicit optimal controls for some stochastic control problems that include the linear-quadratic control problem for systems with a general noise process, the linear exponential quadratic Gaussian control problem for systems with Brownian motion, and the control of Brownian motion in the two sphere and the real hyperbolic plane with both finite and infinite time horizons. 1. Introduction. While the study of stochastic control problems can be traced from the evolution of the calculus of variations and deterministic control and from various stochastic control problems in statistics, the first major problem solution for the formally defined area of optimal stochastic control was the linear-quadratic Gaus- sian (LQG) problem (e.g. (2), (3)) that is also called the stochastic regulator problem (e.g. (13)). The area of stochastic control has developed significantly in breadth and depth from the solution of the LQG problem approximately fifty years ago. Two general methods have developed for solving stochastic control problems. They are the Hamilton-Jacobi-Belllman (HJB) equation and the stochastic maximum (or min- imum) principle. The HJB equation can be considered as a natural generalization of the Hamilton-Jacobi equation of classical mechanics and the stochastic maximum principle can be considered as a natural generalization of the maximum principle of deterministic control (e.g. (18)). For a stochastic control problem the Hamilton-Jacobi-Bellman equation is (typ- ically) a nonlinear second order partial differential equation. The control system is assumed to generate a continuous time Markov process for any control from the family of admissible controls. Clearly the questions of existence and uniqueness of solutions for HJB equations are difficult in general. Therefore the notion of solutions is of- ten relaxed to viscosity solutions (14). However an optimal control that arises from the solution of an HJB equation may not be in the family of admissible controls. Thus significant difficulties often arise with this method though it provides sufficient conditions for optimality. The stochastic maximum principle method is another important approach which yields necessary conditions for optimality. With some suitable convexity conditions � Dedicated to Professor Hanfu Chen on the occasion of his 75th birthday. Research supported by NSF grants DMS 0808138 and DMS 1108884, AFOSR grants FA9550-09-12-1-0384 and FA9550-12-