Collision avoidance decision-making strategy for multiple USVs based on Deep Reinforcement Learning algorithm

Abstract

—To address the issue of flexible decision-making for Unmanned Surface Vehicles (USVs) in emergency collision avoidance scenarios, this paper proposes a Collision Avoidance Decision-making Strategy (CADMS) for multiple USVs based on Deep Reinforcement Learning (DRL). The novelty of this method is: (1) The Preferential Experience Replay Mechanism (PERM) and the Gate Recurrent Unit (GRU) network are employed to enhance the Deep Deterministic Policy Gradient (DDPG) algorithm. The PERM facilitates faster convergence by increasing the sampling probability, thereby aiding in data assimilation. Meanwhile, the GRU enables self-assessment of sensor input significance, leading to more accurate action prediction. (2) In the design of the reward function, the departure from COLREGs has been fully considered, including the two-ship encounter situations and the multi-ship encounter situations. This study also provides a comprehensive analysis of neural network training and generalization verification. The experimental results demonstrate the effectiveness of the proposed approach in effectively avoiding collisions with multiple USVs in complex water environments. It is noteworthy that in comparison to conventional collision avoidance methods, the proposed CADMS model exhibits a superior collision avoidance success rate and enables safer and more efficient decision-making behaviors when confronted with intricate encounter situations.