Reinforcement learning based robot navigation using illegal actions for autonomous docking of surface vehicles in unknown environments

Abstract

Autonomous Surface Vehicles (ASVs) are bound to play a fundamental role in the maintenance of offshore wind farms. Robust navigation for inspection vehicles should take into account the operation of docking within a harbouring structure, which is a critical and still unexplored maneuver. This work proposes an end-to-end docking approach for ASVs, based on Reinforcement Learning (RL), which teaches an agent to tackle collision-free navigation towards a target pose that allows the berthing of the vessel. The developed research presents a methodology that introduces the concept of illegal actions to facilitate the vessel’s exploration during the learning process. This method improves the adopted Actor-Critic (AC) framework by accelerating the agent’s optimization by approximately 38.02%. A set of comprehensive experiments demonstrate the accuracy and robustness of the presented method in scenarios with simulated environmental constraints (Beaufort Scale and Douglas Sea Scale), and a diversity of docking structures. Validation with two different real ASVs in both controlled and real environments demonstrates the ability of this method to enable safe docking maneuvers without prior knowledge of the scenario.