Non-communicating Decentralized Multi-robot Collision Avoidance in Grid Graph Workspace based on Dueling Double Deep Q-Network

Abstract

This paper designs a motion rule suitable for grid environment and proposes a multi-robot autonomous obstacle avoidance method based on deep reinforcement learning. The training and validation of the method are done under the Stage simulation platform based on ROS operating system. During the training process, the robot uses Lidar to obtain the surrounding state information and generates actions based on the state information to obtain rewards, and the robot is guided by the rewards to optimize the strategy. Based on the D3QN algorithm, a new reward function is designed, a proximity penalty is introduced to reduce the collision between robots, a distance reward is added to guide the robot to complete the task, a step reward is added to improve the efficiency of the robot to complete the task, and an illegal action penalty is added to avoid the robot to choose an illegal action; the input is 5 frames of Lidar data, and in the network structure, the agent can better learn the correlation between the data by introducing Long Short Term Memory(LSTM) layer, and introducing Convolutional Block Attention Module(CBAM), a hybrid attention mechanism to allow the robot to pay more attention to the information of the surrounding robots. By designing experiments, we demonstrate that the learned strategy can effectively guide the robot through obstacle avoidance and complete the task.