Reinforcement Learning Based Optimal Control of Linear Singularly Perturbed Systems

Abstract

This brief studies the optimal control problem of linear singularly perturbed systems (SPSs) via reinforcement learning (RL). We first present an offline model-based algorithm on the basis of Kleinman algorithm to solve the underlying algebraic Riccati equation with singular perturbation parameter and its convergence is proven. This revised version of Kleinman algorithm is the key to develop the subsequent learning algorithm. Then, by means of two time-scale characteristics of SPSs, we present a novel model-free learning algorithm relating to the online measurement of state and input to design the optimal controller. Compared with the existing learning approaches, the obtained RL algorithm is free of ill-conditioned numerical issues caused by the co-existence of fast and slow modes in SPSs and thus is more robust with respect to the computation and measurement errors. Finally, we verify the effectiveness of the developed results by a simulation example.