Discrete LQR and ILQR methods based on high order Runge-Kutta discretizations

Abstract

In this paper, discrete linear quadratic regulator (DLQR) and iterative linear quadratic regulator (ILQR) methods based on high-order Runge-Kutta (RK) discretizations are proposed for solving linear and nonlinear quadratic optimal control problems respectively. As discovered in Hager (2000) [11], direct approaches with RK discretization are equivalent with indirect approaches based on symplectic partitioned Runge-Kutta (SPRK) integration. In this paper, we will reconstruct this equivalence by the analogue of continuous and discrete dynamic programming. Then, based on the equivalence, we discuss the issue that the internal-stage controls produced by direct approaches may have lower order accuracy than the RK method used. We propose order conditions for internal-stage controls and then demonstrate that third or fourth order explicit RK methods cannot avoid the order reduction phenomenon. To overcome this obstacle, we calculate node control instead of internal-stage controls in DLQR and ILQR methods. Another advantage of our methods is high computational efficiency which comes from the usage of feedback technique. A simple numerical example will illustrate the validity of ILQR in solving nonlinear optimal control problems with high-order RK discretizations. In this paper, we also demonstrate that ILQR is essentially a quasi-Newton method with linear convergence rate.