Model-Free Recurrent Reinforcement Learning for AUV Horizontal Control

Abstract

In this paper, aiming at the problems of 2-DOF horizontal motion control with high precision for autonomous underwater vehicle(AUV) trajectory tracking tasks, deep reinforcement learning controllers are applied to these conditions. These control problems are considered as a POMDP (Partially Observable Markov Decision Process). Model-free reinforcement learning(RL) algorithms for continuous control mission based on Deterministic Policy Gradient(DPG) allows robots learn from received delayed rewards when interacting with environments. Recurrent neural networks LSTM (Long Short-Term Memory) are involved into the reinforcement learning algorithm. Through this deep reinforcement learning algorithm, AUVs learn from sequences of dynamic information. The horizontal trajectory tracking tasks are described by LOS method and the motion control are idealized as a SISO model. Tanh-estimators are presented as data normalization. Moreover, AUV horizontal trajectory tracking and motion control simulation results demonstrate this algorithm gets better accuracy compared with the PID method and other non-recurrent methods. Efforts show the efficiency and effectiveness of the improved deep reinforcement learning algorithm.