Improved model-free H∞ control for batch processes via off-policy 2D game Q-learning

Abstract

To eliminate the requirement of the precise model for model-based control methods, an improved modelfree H∞ control method is designed for batch processes with unknown dynamics and disturbances. Firstly, both the zero-sum game value function and the Q-function are presented as two-dimensional (2D) forms, and their relation is analyzed to obtain the model-free Bellman equation. Secondly, an on policy 2D game Q-learning method is proposed for learning the optimal gains of the designed H∞ controller. On this basis, the behavior control policy and the behavior disturbance policy are individually applied by developing an off-policy 2D game Q-learning method. Subsequently, the strict proof about the convergence and the unbiasedness of the off-policy approach are given. Finally, the simulation results of injection velocity manifest the validity and effectivity of the proposed algorithm.