Obstacle avoidance for environmentally-driven USVs based on deep reinforcement learning in large-scale uncertain environments

Abstract

This paper focuses on obstacle avoidance for the environmentally-driven unmanned surface vehicles (USVs) in large-scale and uncertain environments. A novel speed adaptive robust obstacle avoidance (SAROA) approach is proposed with the deep reinforcement learning (DRL). A feature enhanced dynamic training method for the DRL is proposed, which significantly improves the sampling efficiency and accelerates convergence. The sensory cues, the executed action and the reward feedback function are properly designed for reinforcement learning to realize robust obstacle avoidance in uncertain environment. Moreover, the obstacle perception domain and the line-of-sight (LOS) based target tracking method are proposed to enable the USVs to avoid collision as well as to follow the path in large-scale environment. In addition, regarding that the environmentally-driven USV is normally super under-actuated and its speed is uncontrollable, a speed adaptive zone is proposed to adapt the obstacle avoidance policies to various navigation speeds, which significantly improves the adaptability and robustness of the proposed obstacle avoidance strategy. Extensive obstacle avoidance tests for the environmentally-driven USV with different navigation speed are conducted, which demonstrates that the DRL based SAROA approach shows excellent practicability and robustness for the environmentally-driven USVs in large-scale and uncertain environments.