Girsanov functionals and optimal bang-bang laws for final value stochastic control

Abstract

Girsanov's theorem is a generalization of the Cameron-Martin formula for the derivative of a measure induced by a translation in Wiener space. It states that for ϕ a nonanticipative Brownian functional with ∫|ϕ| 2 d s < ∞ a.s. and d P ̃ =exp[ζ(ϕ)] d P with E ̃ {1}=1 , where ζ(ϕ) = ∫ϕ dw- 1 2 ∫|ϕ| 2 ds , the translated functions (Tw)(t) = w t - ∫ 0 t ϕ ds are a Wiener process under P̃. The Girsanov functionals exp [ζ(ϕ)] have been used in stochastic control theory to define measures corresponding to solutions of stochastic DEs with only measurable control laws entering the right-hand sides. The present aim is to show that these same concepts have direct practical application to final value problems with bounded control. This is done here by an example, the noisy integrator: Make E{ x 2 1}∣small, subject to d x t = u t d t + d w t , | u|⩽ 1, x t observed. For each control law there is a definite cost v(1− t, x) of starting at x, t and using that law till t = 1, expressible as an integral with respect to (a suitable) P̃. By restricting attention to a dense set of smooth laws, using Itô's lemma, Kac's theorem, and the maximum principle for parabolic equations, it is possible to calculate sgn v x for a critical class of control laws, then to compare control laws, “solve” the Bellman-Hamilton-Jacobi equation, and thus justify selection of the obvious bang-bang law as optimal.