Straight-Path Following for Underactuated Marine Vessels using Deep Reinforcement Learning

Abstract

We propose a new framework, based on reinforcement learning, for solving the straight-path following problem for underactuated marine vessels under the influence of unknown ocean current. A dynamic model from the Marine Systems Simulator is employed to simulate the motion of a mariner-class vessel, however the policy search algorithm has no prior knowledge of the system it is assigned to control. A deep neural network is used as function approximator and the deep deterministic policy gradients method is employed to extract a suitable policy that minimizes the cross-track error. Two intuitive reward functions, which in addition prevent noisy rudder behavior, are proposed and compared. The simulation results demonstrate excellent performance, also in comparison with the line-of-sight guidance law.