Navigation behavioural decision-making of MASS based on deep reinforcement learning and artificial potential field

Abstract

To realize intelligent obstacle avoidance and local path decisions for maritime autonomous surface ships (MASS) in uncertain environments, a navigation behavioural decision-making model based on deep reinforcement learning (DRL) algorithm improved by artificial potential field (APF) is proposed. Based on the analysis of navigation decision system and perception principle, the action space, reward function, motion search strategy and action value function are designed respectively for the purpose of steering to collision avoidance. The navigation behavioural decision-making model for MASS is improved by adding the prior information, the gravitational potential field and the obstacle repulsion potential field to update the initial action state value function and search path. Python and Pygame modules are used to build a simulation chart. Effectiveness of the algorithm is verified, with Tianjin Xingang port as a study case. The simulation results show that the APF-DRL algorithm is better than the DRL algorithm in training iteration time and piloting decision path, which improves the self-learning ability of MASS, and can meet the requirements of MASS path decision and adaptive obstacle avoidance.