Reinforcement Learning-Based Trajectory Optimization for Data Muling With Underwater Mobile Nodes

Abstract

This paper addresses trajectory optimization problems for underwater data muling with mobile nodes. In the underwater data muling scenario, multiple autonomous underwater vehicles (AUVs) sample a mission area, and autonomous surface vehicles (ASVs) visit the navigating AUVs to retrieve the collected data. The optimization objectives are to simultaneously maximize fairness in data transmissions and minimize the travel distance of the surface nodes. We propose an nearest- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$K$ </tex-math></inline-formula> reinforcement learning algorithm, which chooses only from the nearest- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$K$ </tex-math></inline-formula> AUVs as candidates for the next node for data transmissions. We use the distance between AUVs and the ASV as the state, selected AUVs as the action. A reward is designed as the function of both the data volume transmitted and the ASV travel distance. In the scenario with multiple ASVs, an AUV association strategy is presented to support the use of multiple surface nodes. We conduct computer simulations for performance evaluation. The effects from the number of AUVs, the size of the mission area, and the state number are investigated. The simulation results show that the proposed algorithm outperforms traditional methods in terms of the fairness and ASV travel distance.