Autonomous Underwater Vehicle Based Chemical Plume Tracing via Deep Reinforcement Learning Methods

Abstract

This article presents two new chemical plume tracing (CPT) algorithms for using on autonomous underwater vehicles (AUVs) to locate hydrothermal vents. We aim to design effective CPT navigation algorithms that direct AUVs to trace emitted hydrothermal plumes to the hydrothermal vent. Traditional CPT algorithms can be grouped into two categories, including bio-inspired and engineering-based methods, but they are limited by either search inefficiency in turbulent flow environments or high computational costs. To approach this problem, we design a new CPT algorithm by fusing traditional CPT methods. Specifically, two deep reinforcement learning (RL) algorithms, including double deep Q-network (DDQN) and deep deterministic policy gradient (DDPG), are employed to train a customized deep neural network that dynamically combines two traditional CPT algorithms during the search process. Simulation experiments show that both DDQN- and DDPG-based CPT algorithms achieve a high success rate (>90%) in either laminar or turbulent flow environments. Moreover, compared to traditional moth-inspired method, the averaged search time is improved by 67% for the DDQN- and 44% for the DDPG-based CPT algorithms in turbulent flow environments.