Optimal controller design for control-affine stochastic systems using neural networks and path integrals

Abstract

For deterministic nonlinear dynamical systems, approximate dynamic programming based on Pontryagin's maximum principle provides a systematic way to solve optimal control problems. However, in the presence of noise, this approach becomes cumbersome. Hence, in current optimal control solution methodologies noise effect is typically ignored in the adjoint equations. Alternatively, in the Hamilton-Jacobi Bellman (HJB) framework, presence of noise results in the second order stochastic HJB equation. Furthermore, through a unique exponential transformation, the stochastic HJB equation of control-affine nonlinear stochastic systems with quadratic control cost function can be transformed into a path integral. In this paper, an offline approximate dynamic programming approach using neural networks and path integrals is proposed for solving the above class of finite horizon stochastic optimal control problems. Simulation results using Vanderpol oscillator model are presented to demonstrate the potential of the proposed approach.