A human-like collision avoidance method for USVs based on deep reinforcement learning and velocity obstacle

Abstract

Collision avoidance is critical in unmanned surface vehicles (USVs) and is especially challenging in scenarios involving ships of different intelligence levels. To address this problem, inspired by risk analysis and collision avoidance habits of seafarers, this paper proposes a human-like collision avoidance method based on deep reinforcement learning (DRL) and velocity obstacle (VO). It first introduces a navigation impact factor (NIF) calculation module based on fuzzy theory to simulate human beings’ attention mechanisms when facing multiple ships. To be compatible with human beings’ regulations, the convention on the International Regulations for Preventing Collisions at Sea (COLREGs) is first to be integrated into the calculation of the NIF calculation module, and the VO algorithm is incorporated into the reward function, which can reduce collision risks in paths and improve safety requirements. In addition, a series of reward functions are carefully designed to balance safety, smoothness, and driving operation. To validate the performance, experiments are conducted on our virtual simulation platform. The results show that our algorithm can accurately assess the impact of target ships (TS) on own ship (OS) in complex environments and can obey the COLREGs. Collision avoidance can be achieved effectively and the path is smoother.