COLREGs-abiding hybrid collision avoidance algorithm based on deep reinforcement learning for USVs

Abstract

In order to realize the autonomous collision avoidance of unmanned surface vehicles (USVs), an intelligent hybrid collision avoidance algorithm based on deep reinforcement learning is proposed. First, the navigation situation model of the USV is designed, the geometric model of the encounter between two ships is established based on navigation practice. According to static and dynamic obstacles, hybrid risk assessment and collision avoidance model are proposed, the risk factor is calculated. Then, for static obstacles, the collision cone is introduced, for dynamic ships, the COLREGS is observed, the encounter situation is quantified into five types. Collision avoidance strategy is formulated. Finally, the state, action, reward function and network structure are designed. Aiming at the problem of low utilization for samples in random sampling, this paper improves the original sampling mechanism of DDPG, and the priority sampling mechanism with cumulative pruning is proposed in this paper. Simulation experiments are carried out in several typical encounter scenarios. The results show that this algorithm can accurately judge the encounter situation, give reasonable collision avoidance actions, and realize effective collision avoidance in a complex environment with dynamic and static obstacles. The research can provide theoretical basis and method reference for autonomous navigation of USVs.