Computational method for nonlinear stochastic optimal control

Abstract

Nonlinear stochastic optimal control problems are treated such that they are nonlinear in the state dynamics, but are linear in the control. The cost functional is a general function of the state, but the costs are quadratic in the control. The system is subject to random fluctuations due to discontinuous Poisson noise that depends on both the state and control, as well as due to discontinuous Gaussian noise. This general framework provides a comprehensive model for numerous applications that are subject to random environments. A stochastic dynamic programming approach is used and the theory for an iterative algorithm is formulated utilizing a least squares equivalent of a genuine LQG Poisson (LQGP) problem to approximate the nonlinear state space dependence of the LQGP problem in control only in order to accelerate the convergence of the nonlinear control problem. A Poisson jump process that is linear in the control vector, within the context of a nonlinear control problem, is treated in detail.