Multi-agent cooperative area coverage: A two-stage planning approach based on reinforcement learning

Abstract

Multi-agent area coverage aims to accomplish the complete traversal of the target area through cooperation between agents. Focusing on the problems of low coverage efficiency and weak practicability in the existing methods, we propose a two-stage area coverage method based on multi-agent deep reinforcement learning. In the first stage, we convert the coverage path planning problem into an optimal grid selection problem, and according to the equivalence of agents in cooperative tasks, we propose a distributed coverage path planning algorithm based on QMIX and a grid coverage map. The second stage is to realize the cooperative navigation control in a constrained environment with obstacles and non-ideal communication conditions. To implement the stage, we design a hybrid attention mechanism to adaptively aggregate important feature information of adjacent agents and obstacles, which efficiently exploits the limited local perception and communication capabilities of agents to perform cooperative control. The experimental results show that the proposed two-stage multi-agent area coverage method can accomplish the area coverage task in the environment with random obstacles, and the area coverage efficiency and robustness are significantly better than other reinforcement learning based or traditional coverage algorithms. In addition, the results also verify that the proposed method has the advantage of adapting to the dynamic changes in the number of agents and the communication range.