Temporal deep unfolding for constrained nonlinear stochastic optimal control

Abstract

This paper proposes a computational technique to solve discrete-time optimal control problems for nonlinear systems subject to stochastic disturbances, hard input constraints, and soft constraints. The approach employs the idea of deep unfolding, which is a recently developed model-based deep learning method that is applicable to iterative algorithms. By regarding each state transition of the dynamical system as an iteration step, these state transitions are unfolded into the layers of a deep neural network. This produces a computational graph that contains trainable parameters that determine control inputs. These parameters are optimized by training the deep neural network using the standard deep learning technique of backpropagation and incremental training. Then, the optimal control inputs are obtained from the optimized parameters. This provides a way to optimize the control inputs that otherwise would be difficult to obtain. The effectiveness of the proposed technique is demonstrated by numerical experiments for the maximum hands-off control problem and constrained model predictive control problems using a highly nonlinear model of a continuous stirred tank reactor.