A finite step scheme for general near-optimal stochastic control

Abstract

This paper reports a finite step scheme for the computation of near-optimal control of general stochastic systems with their diffusion terms affected by control. It is developed based on second order estimation of the stochastic system and the associated two adjoint variational equations. The search is an extension of the standard steepest descent method to the functional case with a random step size based on the variation of an auxiliary function H. Convergence analysis are included to show this scheme does converge to a desired admissible control in finite step. Consistency of the approximation of the associated adjoint equations is also discussed. A linear quadratic control example is included for illustration purpose.