Adaptive optimal output feedback tracking control for unknown discrete‐time linear systems using a combined reinforcement Q‐learning and internal model method

Abstract

This study addresses the novel output feedback-based reinforcement Q-learning algorithms for optimal linear quadratic tracking problem of unknown discrete-time systems. An augmented system composed of the original controlled system and the reference trajectory dynamic is first constructed. Then, learning algorithms including on-policy and off-policy approaches are both developed to solve the optimal tracking control problem with unknown augmented system dynamics. In both the optimal tracking control policies, a two-stage framework is proposed composed of two controllers, where the internal model controller is used to collect some data for the next process, and then the output feedback Q-learning scheme is able to learn the optimal tracking controller online by using the past input, output, and reference trajectory data of the augmented system. Finally, simulation results are provided to verify the effectiveness of the proposed scheme.