Intelligent collision avoidance algorithms for USVs via deep reinforcement learning under COLREGs

Abstract

In the field of unmanned surface vehicles, intelligent collision avoidance technology is essential to ensure the safety of navigating. In this paper, the problem of avoiding moving boats for USVs under the constraints of COLREGs is studied. A COLREGs intelligent collision avoidance (CICA) algorithm based on deep reinforcement learning is proposed, which can automatically extract state features by using powerful deep neural networks. The reward function is designed, which ensures that the USV navigates to the target while obeying COLREGs to avoid dynamic obstacles. A method is proposed to track the current network weight to update the target network weight, which improves the stability of the algorithm in learning the optimal strategy. It is shown that the CICA algorithm converges with fewer training times through ε-greedy with both decaying ε and reward threshold than other three strategies. By comparing the CICA algorithm with the artificial potential field method and the velocity obstacle method, it is concluded that the CICA algorithm is superior to the other two algorithms.