Global path planning algorithm based on double DQN for multi-tasks amphibious unmanned surface vehicle

Abstract

It is a key to making path planning for an amphibious unmanned surface vehicle (USV). A global path planning algorithm based on double deep Q networks (DDQN) is proposed. Firstly, an environment model is constructed by an electronic nautical chart and elevation map to train and verify the algorithm. Secondly, based on the kinematics of amphibious USV, a Markov decision process (MDP) framework is built, and various reward functions are designed for diverse tasks. During the training, obstacles and water depth information of the environment are used, the amphibious USV agent is guided to the target area. Meanwhile, based on the prior knowledge, an action mask approach is integrated to deal with the invalid actions generated by the amphibious USV. Path smoothing is also integrated to smooth the path. According to different criteria, reasonable paths can be generated and adjusted by the weights of the reward function. To verify our algorithm, a small-scale simulation environment is established, and two scenarios are introduced. The results show that our DDQN algorithm can generate reasonable global paths for diverse tasks. Moreover, compared with DQN, A*, and RRT algorithms, the paths generated by our method have better performance.