Optimal tracking control of mechatronic servo system using integral reinforcement learning

Abstract

Recent years, there has been an increasing interest in the application of reinforcement learning (RL) to nonlinear system control. And many published studies have shown that reusing historical data can improve the learning efficiency of intelligent agents. However, reusing historical data leads to a large increase in the data to be processed. This paper proposes a novel adaptive law for the critic network in the RL framework to address this problem. The innovation of the proposed method is the introduction of an auxiliary matrix containing information about the historical estimation errors instead of the historical data itself. Moreover, the addition of the auxiliary matrix allows the persistent excitation (PE) conditions to be checked online. As a validation, the proposed approach is applied to a motor servo system containing unknown dynamics. Simulation and experimental results demonstrate the effectiveness of the proposed controller in the end.