Path-following optimal control of autonomous underwater vehicle based on deep reinforcement learning

Abstract

In this paper, optimal control is applied in the path-following control problem for the autonomous underwater vehicle (AUV) when the geometry information of path is known and will not change over time. Aiming at the problem that the optimization time is too long in one control step for complex nonlinear problems, a path-following control method based on Simplified Deep Deterministic Policy Gradient (S-DDPG) algorithm is proposed. In S-DDPG, only the reward in the current state is considered, and the future reward does not need to be predicted, which avoids generating amount of meaningless failed samples and simplifies the training process of neural networks (NNs). The training is performed and completed offline before the beginning of path-following where the NNs are directly used as the controller. The simulation results show that the S-DDPG can make the AUV complete path-following tasks and has obvious advantages compared with other methods.