Data-Driven Tracking Control for Multi-Agent Systems With Unknown Dynamics via Multithreading Iterative Q-Learning

Abstract

This article addresses the tracking control problem of multiagent systems (MASs) with unknown dynamics. First, by designing a compensator, an augmented neighborhood error system is proposed. Then, a multithreading iterative <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$Q$ </tex-math></inline-formula> -learning algorithm is developed, which can transform the tracking control problem into the optimal regulation of the augmented error system. In this algorithm, the agent controller is multithread, which consists of thread unit, global unit, and replay buffer. The thread unit is used to create the thread to collect the transitions with different control policies, which can reduce data relativity and improve data collection speed and exploration capability. The replay buffer is used to store the generated transitions. The global unit is used to generate the control policy. Moreover, the convergence analysis of the proposed algorithm is given, which shows that the neighborhood error converges to zero and the tracking control problem can be solved. Finally, the numerical example not only verifies the effectiveness of the obtained results but also shows the proposed algorithm performs better than the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$Q$ </tex-math></inline-formula> -iterative algorithm.