Reinforcement Learning-based path tracking for underactuated UUV under intermittent communication

Abstract

This paper studies the path control of a six-degree-of-freedom underactuated Unmanned Underwater Vehicle (UUV) under limited communication conditions. Considering the large number of coupling between six-degree-of-freedom underactuated UUV of unknown dynamic models, traditional model-based control methods are difficult to effectively solve the three-dimensional path control problem. A self-attention based soft actor and critic (A-SAC) algorithm is designed to learn effective control policy from random paths. The problem of limited target acquisition by UUV in the actual underwater environment is effectively overcome, which is mainly caused by the inability of UUV to consistently receive information about their expected path. A new state space is designed and a self-attention mechanism is introduced to improve the efficiency of using discontinuous path information. Furthermore, the validation experiment compares classical Reinforcement Learning methods such as DDPG, PPO, and etc. Compared to other existing methods, the proposed A-SAC algorithm can more quickly and effectively learn the path control policy for a six-degree-of-freedom UUV that operates in a complex environment.