Research on Collision Avoidance Algorithm of Unmanned Surface Vehicle Based on Deep Reinforcement Learning

Abstract

To solve the problem of active collision avoidance for unmanned surface vehicle (USV) in complex maritime environment, a method of deep reinforcement learning (DRL) based on proximal policy optimization (PPO) is proposed. In order to master the collision avoidance policy by self-learning, the mathematical model of USV, dynamic obstacle generation model and reward mechanism were established. Using the relative position between obstacle and USV and the information of closest point of approach (CPA), high-dimensional state features including the collision track layer and collision threat layer, were constructed respectively. On this basis, combined with low-dimensional states such as navigation state and path error, a deep convolutional neural network(CNN) structure fused in multi-feature and multi-scale was designed. The proposed DRL network was trained through repeated collision avoidance simulations in random environments. Simulation results reveal that the proposed algorithm can output effective decisions in complex scenarios and avoid dynamic obstacles quickly and safely.