Game Theoretical Reinforcement Learning for Robust H∞ Tracking Control of Discrete-Time Linear Systems with Unknown Dynamics

Abstract

Robust H <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∞</inf> control has been widely studied to improve control performance for industrial process control systems against disturbances. However, most of existing robust H <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∞</inf> control are model-based, and their deployment in some industrial facilities may greatly increase the installation and maintenance costs due to requiring system identification. Towards this end, a model-free robust H <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∞</inf> tracking control scheme is developed based on game theoretical reinforcement learning (RL) for discrete-time linear systems with unknown dynamics. The normal robust H <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∞</inf> tracking control problem is first modeled as a two-player zero-sum game with the controller and disturbance as the two players. A model-based solution by solving game discrete-time differential Riccati equation (GDARE) is introduced to show the solvability of the robust H <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∞</inf> tracking control problem, and then a novel off-policy RL algorithm is developed to replace the GDARE method for model-free robust H <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∞</inf> tracking control of the discrete-time linear systems with unknown dynamics. Stability of the learning algorithm is analyzed. Finally, a simulation study upon a de-oiling hydrocyclone system is conducted to demonstrate the effectiveness of the proposed algorithm.