Efficient Reinforcement Learning for Autonomous Ship Collision Avoidance under Learning Experience Reuse

Abstract

In this paper, a learning experience reuse - reinforcement learning collision avoidance (LER-RLCA) method is proposed, which can synthesize near-optimal collision avoidance policy with efficient sampling and good seamanship, to solve the local safety sailing of autonomous ship in a multi-obstacle environment. Lying on the general reinforcement learning (RL), using learning experience reuse, the hidden features of historical training data were mined. Meanwhile, a new reward function combining external revenue signal with internal incentive signal was designed to encourage search the environment with a low probability of state transition. We further applied LER-RLCA algorithm to the simulation of autonomous ship collision avoidance. The results show that the proposed LER-RLCA algorithm can well realize the collision-free and safe navigation of autonomous ships, to avoid falling into local iteration, greatly improve the convergence speed of the algorithm, and improve the performance of online collision avoidance decision-making.