AUV path tracking with real-time obstacle avoidance via reinforcement learning under adaptive constraints

Abstract

In this paper, the methods are proposed for underactuated autonomous underwater vehicle (AUV) to address three-dimensional (3D) path tracking and real-time obstacle avoidance. The errors of path tracking are generated based on the Serret–Frenet frame and line-of-sight (LOS) guidance law, while the errors in obstacle avoidance are obtained based on the carrier coordinate system to filter irrelevant environment information. On this basis, to deal with the complicated target path and unknown obstacles, the controller is designed by deep deterministic policy gradient (DDPG) algorithm and adaptive multi-constraints. The safety constraints are adopted in reward functions to avoid useless explorations and facilitate convergence. The training proceeds for path tracking and obstacle avoidance respectively. Compared to the original DDPG algorithm in the training, the proposed algorithm shows faster convergence. Various simulations are conducted under different initial conditions, and the results demonstrate the effectiveness of the proposed algorithm.