COLREGs-Compliant Multi-Ship Collision Avoidance Based on Multi-Agent Reinforcement Learning Technique

Abstract

The congestion of waterways can easily lead to traffic hazards. Moreover, according to the data, the majority of sea collisions are caused by human error and the failure to comply with the Convention on the International Regulation for the preventing Collision at Sea (COLREGs). To avoid this situation, ship automatic collision avoidance has become one of the most important research issues in the field of marine engineering. In this study, an efficient method is proposed to solve multi-ship collision avoidance problems based on the multi-agent reinforcement learning (MARL) algorithm. Firstly, the COLREGs and ship maneuverability are considered for achieving multi-ship collision avoidance. Subsequently, the Optimal Reciprocal Collision Avoidance (ORCA) algorithm is utilized to detect and reduce the risk of collision. Ships can operate at the safe velocity computed by the ORCA algorithm to avoid collisions. Finally, the Nomoto three-degrees-of-freedom (3-DOF) model is used to simulate the maneuvers of ships. According to the above information and algorithms, this study designs and improves the state space, action space and reward function. For validating the effectiveness of the method, this study designs various simulation scenarios with thorough performance evaluations. The simulation results indicate that the proposed method is flexible and scalable in solving multi-ship collision avoidance, complying with COLREGs in various scenarios.