Stochastic bang-bang controls that maximize the expectation of first passage time†

Abstract

The stochastic bang-bang control problem of maximizing the expectation of the first passage time of the state to the boundary of a certain safe region is considered. It is assumed that the dynamics of the system is described by a linear stochastic differential equation. By use of dynamic programming, the problem is reduced to a boundary-value problem of Dirichlet type for the Bellman equation. A difference scheme is applied in order to obtain the numerical solution of the boundary-value problem. It is found that for first-order systems the difference scheme gives excellent numerical results. Some switching curves are also obtained for a second-order plant l/s 2 with additive Gaussian white noises.