Research on path tracking control method of unmanned surface vehicle based on deep reinforcement learning

Abstract

Due to the nonlinear and underactuated characteristics of unmanned surface vehicle system and the uncertainty of environmental model, it is hard to establish accurate dynamic model and control law obtained by traditional algorithm which is too complex and has no engineering practice realization. In this paper, based on deep reinforcement learning algorithm of deep deterministic policy gradients, the line of sight algorithm is used firstly to obtains the expected value of heading angle of USV according to the current time position and the expected trajectory of USV. Meanwhile, we adopt the double Gaussian reward function to evaluate the training action, so as to obtain the optimal control action to realize the accurate tracking control. Finally, compared with explicit model predictive controller and linear quadratic regulator, the designed track controller based on DDPG has shorter adjusting time and smaller overshoot than explicit model predictive controller and linear quadratic regulator.