Deep reinforcement learning with dynamic window approach based collision avoidance path planning for maritime autonomous surface ships

Abstract

Automatic obstacle avoidance technology is one of the key technologies for ship intelligence. The purpose of this paper is to investigate the obstacle avoidance problem of maritime autonomous surface ships(MASS) in a complex offshore environment, and an obstacle avoidance strategy based on deep reinforcement learning and a dynamic window algorithm was proposed. To solve the collision avoidance problems that may occur during intelligent ship navigation, the action space of the proximal policy optimization (PPO) algorithm is defined according to the description of ship motion by linear and angular velocity in the dynamic window approach (DWA). The maximum detection distance of the MASS is utilized to construct the ship safety domain, which determines the state space containing the information of this ship and the nearest obstacle. To solve the problem of sparse reward, the reward function of the PPO is improved by combining the evaluation functions for distance, velocity and heading in the DWA. To verify the effectiveness of the algorithm, simulation experiments are performed in various situations. It is also shown that the improved algorithm can make the optimal collision avoidance decision from the complex environment and can effectively realize autonomous collision avoidance path planning for the MASS.