Optimal Consensus Control for Second-Order Discrete-Time Multi-Agent Systems: Using Online Policy Iteration Algorithm

Abstract

In this paper, the optimal consensus control problem for multi-agent systems is discussed via a model-free reinforcement learning algorithm. Considering the second-order discretetime multi-agent systems (MASs) with directed topology which is composed of the leader and the follower agents, a novel solution to the Hamilton-Jacobi-Bellman (HJB) equation combining the optimal consensus problem with the system characteristics including the second-order discrete-time systems with cooperative interactions under directed topology is presented. By graph theory, matrix analysis, deep learning and optimal control method, the necessary conditions and strategies for solving the optimal consensus problem of the dynamic systems are obtained. The results show that the online policy iteration algorithm not only stabilizes the distributed second-order dynamic systems, but also achieves the optimal consensus. Finally, the correctness of our theoretical results are illustrated by the designing actor-critic networks through several numerical simulations.