Optimal consensus control for unknown second-order multi-agent systems: Using model-free reinforcement learning method

Abstract

In this paper, the optimal consensus control problem with second-order dynamics consisting of leader and follower agents is discussed. For optimal consensus problem, the optimal control policies rely on algebraic Riccati equations (AREs) equation, which are difficult to solve. Furthermore, both the follower agents’ and the leader agent’s dynamics are assumed to be completely unknown. As the consensus problem based on feedback control, the second-order discrete-time multi-agent systems (DT-MASs) model with directed topology is formulated to the optimal tracking control problem via online deep reinforcement learning method. Based on graph theory, matrix analysis, Lyapunov stability, deep learning and optimal control, the optimality of value function and the stability of the consensus error systems for the unknown second-order systems are guaranteed for each agent. The results show that the designed policy iteration algorithm not only stabilizes the distributed dynamic systems, but also makes all agents’ position and velocity states reach consensus, respectively. Finally, the correctness of our theoretical results is illustrated under two numerical simulations based on the designing model-free actor-critic networks.