Decision Model of Ship Intelligent Collision Avoidance Based on Automatic Information System Data and Generic Adversary Imitation Learning-Deep Deterministic Policy Gradient

Abstract

Aiming at the problems that the current decision-making model of ship collision avoidance does not consider International Regulations for Preventing Collisions at Sea (COLREGS), ship maneuverability, and the need for a lot of training time, combined with the advantages of reinforcement learning and imitation learning, a ship intelligent collision avoidance decision-making model based on Generic Adversary Imitation Learning (GAIL) is proposed: Firstly, the collision avoidance data in Automatic Information System (AIS) data is extracted as expert data; Secondly, in the generator part, the environment model is established based on Mathematical Model Group (MMG) and S-57 chart rendering, and the state space, behaviour space and reward function of reinforcement learning are constructed. The deep deterministic policy gradient (DDPG) is used to interact with the environment model to generate ship trajectory data. At the same time, the generator can constantly learn expert data; Finally, a discriminator can distinguish the expert data from the data generated by the generator is constructed and trained. The model training is completed when the discriminator cannot distinguish the two. In order to verify the performance of the model, AIS data near the South China Sea is used to process and extract collision avoidance decision data, and a ship intelligent collision avoidance decision model based on GAIL is established. After the model converges, the final generated data is compared with the expert data. The experimental results verify that the model proposed in this paper can reproduce the expert collision avoidance trajectory and is a practical decision model of ship collision avoidance.