Reducing Controller Effort in a Deep Reinforcement Learning-based Autopilot for an Underactuated ASV

Abstract

Listen

Translate article

With the growing interest in applications of Deep Reinforcement Learning for control of Autonomous Surface Vessels, it becomes vital that one ensures a smooth and efficient behavior of trained action policies. Excessive variation in control can lead to unnecessary wear and tear of actuators, passenger discomfort, inefficient and unstable vessel motions, and other complications. While several ways to handle this issue are documented in the literature, it is unclear how well these approaches can be tailored to applications in underactuated and fully actuated Autonomous Surface Vessels. In this study, we compare some of these methods in the task of Deep Reinforcement Learning-based rudder control of an underactuated vessel for path-following. The results indicate that introducing flexibility while enforcing controller smoothness yields a more stable behavior from the trained policy. In addition, we find that regularizing action policies yields a more efficient control behavior, but with a potential trade-of on smoothness and difficulties in implementation and analysis.