MASS autonomous navigation system based on AIS big data with dueling deep Q networks prioritized replay reinforcement learning

Abstract

Ship autonomous navigation constitutes the most critical step in intelligent ships and is the major prerequisite for the all task completion of the marine autonomous surface ship (MASS). Reinforcement learning (RL) has been widely used in unmanned transport vehicles because of its excellent performance in solving continuous handling problems. This study proposes a MASS autonomous navigation system using dueling deep Q networks prioritized replay (Dueling-DQNPR) based on the ship automatic identification system (AIS) big data. A navigation environment with three difficulty levels were established to train the Dueling-DQNPR network in sequence by setting a reward mechanism. Moreover, the Dueling-DQNPR was improved by combining the prioritized experience replay, dueling structure and long–short-term memory unit to increase the network depth and ability to process continuous data. Finally, simulation training for the AIS trajectory data was carried out in waters near Zhoushan port. The results demonstrated that, through trial and error, the MASS could be controlled to arrive at its destination without collision. As a result, the proposed method may be applicable to MASS on-duty technology and intelligent ships.