Closed-loop optimisation of neural networks for the design of feedback policies under uncertainty

Abstract

Solving model predictive control (MPC) problems online can be computationally intractable, especially when considering uncertainty and nonlinear systems. One approach to avoid this is to train a neural network on a data-set of solutions of MPC problems (potentially nonlinear) offline, and to evaluate the trained control policy online. However, due to the separation of these optimisation problems the neural network controller must be carefully verified to ensure adequate closed loop performance. In this paper we propose a method to train a neural network in closed loop for control systems, in continuous or discrete time, while allowing for flexible consideration of parametric uncertainty. This method does not require off-line solutions of the NMPC problem, and instead directly optimises the desired closed loop performance. We prove that our method can approximate the optimal closed-loop control policy to arbitrary tolerance and in numerical examples demonstrate its performance compared to explicit MPC, imitation learning and nominal MPC.