Statistical control of control-affine nonlinear systems with nonquadratic cost functions: HJB and verification theorems

Abstract

In statistical control, the cost function is viewed as a random variable and one optimizes the distribution of the cost function through the cost cumulants. We consider a statistical control problem for a control-affine nonlinear system with a nonquadratic cost function. Using the Dynkin formula, the Hamilton–Jacobi–Bellman equation for the n th cost moment case is derived as a necessary condition for optimality and corresponding sufficient conditions are also derived. Utilizing the n th moment results, the higher order cost cumulant Hamilton–Jacobi–Bellman equations are derived. In particular, we derive HJB equations for the second, third, and fourth cost cumulants. Even though moments and cumulants are similar mathematically, in control engineering higher order cumulant control shows a greater promise in contrast to cost moment control. We present the solution for a control-affine nonlinear system using the derived Hamilton–Jacobi–Bellman equation, which we solve numerically using a neural network method.