A multi-agent flocking collaborative control method for stochastic dynamic environment via graph attention autoencoder based reinforcement learning

Abstract

The environmental adaptability of the multi-agent flocking collaborative control system is vital to practical applications. Focusing on the adaptive problem of multi-agent flocking collaborative control system in stochastic dynamic environment, this paper proposes a distributed multi-agent flocking collaborative control algorithm based on a graph attention autoencoder (GAE) based multi-agent reinforcement learning (MARL). In our algorithm, a distance-based graph attention (GAT) mechanism is introduced into the networks of MARL to improve the non-stationarity problem of state transition in MARL caused by stochastic dynamic environment and enhance agents’ comprehension to the observation state. Based on the distance-based GAT, a GAE is designed to adapt to dynamic scale scenes. In addition, a global reward based strategy evaluation method is used to minimize the system loss of the flocking collaborative control system. The experimental results demonstrate that the proposed flocking algorithm has better environmental adaptability and better global control strategy than other RL-based flocking algorithms. The conclusion that the control strategies learned by our algorithm can be well transferred to the scenes with different agents and obstacles is also verified. This paper provides a novel and effective solution scheme to the multi-agent flocking collaborative control problem in a stochastic dynamic environment, which is conducive to promoting the application of flocking algorithm.