Two-dimensional model-free Q-learning-based output feedback fault-tolerant control for batch processes

Abstract

For batch processes with partial actuator failures and unknown system dynamics, an innovative two-dimensional (2D) model-free Q-learning algorithm is proposed to obtain the optimal controller's gains, achieving output feedback fault-tolerant control. First, a 2D linear model is constructed to describe batch processes with partial actuator failures. Then, the state increments in the batch direction and the output errors in the time direction are used as novel state variables to construct a multi-degree-of-freedom model. Second, a 2D Bellman equation is proposed through a connection between a 2D value and a 2D Q functions. Next, a 2D off-policy model-free Q-learning algorithm is highlighted, which incorporates target policies into a multi-degree-of-freedom model and focuses on using policy iteration to solve the fault-tolerant tracking control problem. The robustness analysis rigorously proves the stability of the closed-loop system. Lastly, the simulation results of the holding stage prove the feasibility and effectiveness of the presented algorithm.