Maximum Principle and Stochastic Hamiltonian Systems

Abstract

One of the principal approaches in solving optimization problems is to derive a set of necessary conditions that must be satisfied by any optimal solution. For example, in obtaining an optimum of a finite-dimensional function, one relies on the zero-derivative condition (for the unconstrained case) or the Kuhn-Tucker condition (for the constrained case), which are necessary conditions for optimality. These necessary conditions become sufficient under certain convexity conditions on the objective/constraint functions. Optimal control problems may be regarded as optimization problems in infinite-dimensional spaces; thus they are substantially difficult to solve. The maximum principle, formulated and derived by Pontryagin and his group in the 1950s, is truly a milestone of optimal control theory. It states that any optimal control along with the optimal state trajectory must solve the so-called (extended) Hamiltonian system, which is a two-point boundary value problem (and can also be called a forward-backward differential equation, to be able to compare with the stochastic case), plus a maximum condition of a function called the Hamiltonian. The mathematical significance of the maximum principle lies in that maximizing the Hamiltonian is much easier than the original control problem that is infinite-dimensional. This leads to closed-form solutions for certain classes of optimal control problems, including the linear quadratic case.