A human-like collision avoidance method for autonomous ship with attention-based deep reinforcement learning

Abstract

Reinforcement learning has the characteristics of simple structure and strong adaptability, which has been widely used in the field of ship autonomous collision avoidance. In order to solve the problem of collision avoidance in multi-ship encounter situation, a novel collision avoidance method for autonomous ship with attention-based deep reinforcement learning (ADRL) is proposed, it consists of two parts, risk assessment module and motion planning module, the difference between the former and the existing collision risk calculation method is that from the officer's attention distribution, it encode the ship's information through the local map, and calculate each ship's collision avoidance decision in the form of attention score in real time under the constraints of the COLREGS. In addition, a composite learning method is designed, which integrates supervised learning into the common direct environmental exploration model, which accelerates the exploration efficiency of the model and shows excellent learning performance. Finally, based on the Open AI Gym platform, static obstacle situation, dynamic multi-ship encounter situation, dynamic and static obstacle coexistence situation are designed, and the rationality and effectiveness of collision avoidance decision are analyzed from the perspectives of collision risk and the closest safety distance respectively.