$H_{\infty}$ Tracking Control of Unknown Discrete- Time Linear Systems via Output-Data-Driven Off-policy Q-learning Algorithm

Abstract

In the paper, an off-policy Q-learning algorithm is provided for tackling the <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$H_{\infty}$</tex> tracking control problem of discrete-time linear systems with unknown dynamics. First, the discrete-time linear system is augmented when taken by its reference trajectory generator and we transform the considered <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$H_{\infty}$</tex> tracking control problem into a min-max optimization problem with a discounted performance cost function. Second, we established a discounted game algebraic Riccati equation (GARE) based on a former cost function and proposed the existence conditions to assure the stability of the discounted GARE. Third, a Q-function Bellman equation is constructed to obtain the learning algorithm. As such, an output data-driven Q-learning algorithm is developed for optimal control where the matrices of the corresponding Bellman equations are full rank and it is proved to satisfy the persistent excitation condition. An application to the current tracking control of a grid-connected three-phase PV power inverter is derived by simulation to validate the effectiveness and superiority of the proposed Qlearning algorithm.