Reinforcement learning based-adaptive tracking control for a class of semi-markov non-Lipschitz uncertain system with unmatched disturbances

Abstract

A reinforcement learning based-adaptive tracking control method has been proposed for a class of semi-markov non-Lipschitz uncertain system with unmatched disturbances. Not only does the considered system contain unknown nonlinear functions and multisource disturbances, but parameter matrix in the system varies with the semi-Markov progress, which create difficulties in the design of the controller. Firstly, to handle the non-Lipschitz nonlinear terms, an adaptive control law based on reinforcement learning has been designed. Secondly, according to the reinforcement learning result, an adaptive disturbance observer has been constructed to offset the adverse influence generated by the multiple disturbances in the considered system. Finally, aiming at the parameter matrixes in the system varying with semi-Markov progress, a novel adaptive tracking controller integrating the reinforcement learning approximation, the adaptive disturbance observer and the tracking control law, has been proposed. With the given adaptive tracking control law based on reinforcement learning, through the Lyapunov direct method, it has been proven that the closed-loop system are stable and convergent and the tracking errors are ultimately uniformly bounded. In addition, several numerical experiments are provided to illustrate the feasibility and effectiveness of the proposed reinforcement learning based adaptive tracking control law.