H∞ Control for Discrete-time Linear Systems by Integrating Off-policy Q-learning and Zero-sum Game

Abstract

This paper focuses on solving H∞ control problem by proposing a novel off-policy Q-learning algorithm under the framework of zero-sum game for discrete-time (DT) linear systems, using only the measured data along the system trajectories. Firstly, H∞ control problem is formulated, followed by the standard on-policy Q-learning algorithm in order to learn the H∞ controller gain. Secondly, behavior control policy and behavior disturbance policy are introduced, and an off-policy Q-function based game Bellman equation (OPQ-GBE) is derived. Consequently, an off-policy Q-learning algorithm is developed for the first time for discrete-time linear systems subject to the external disturbance, and the convergence and no bias of solution of OPQ-BE are proven. Finally, a F-16 aircraft autopilot is given to verify the effectiveness of the proposed method.