Graph attention mechanism based reinforcement learning for multi-agent flocking control in communication-restricted environment

Abstract

To solve the poor performance of reinforcement learning (RL) in the multi-agent flocking cooperative control under the communication-restricted environments, we propose a multi-agent cooperative RL (MACRL) method based on the equivalent characteristics of the agents in the flocking task. A distance graph attention (GAT) mechanism is introduced into the policy network of the proposed MACRL to change the agent’s attention weights related to neighbors and reduce the influence of remote neighbors which have poor communication quality on the agent’s behavioral decision-making. Furthermore, a distance GAT-based MACRL (DGAT-MACRL) algorithm is proposed for multi-agent flocking control in the communication-restricted environment. The simulation results show that the proposed flocking algorithm has good adaptability to the communication delay and the communication distance constraint environments. The flocking control effect is significantly better than other RL-based flocking algorithms and traditional flocking algorithms. In addition, the good experimental performance has also confirmed that the proposed DGAT-MACRL is an effective solution to the problem of how to improve the adaptability of traditional RL to the flocking control system with a dynamic scale. Our algorithm provides a novel and practical algorithm offering an effective method for the cooperative task accomplished by multi-agent in non-ideal environments.