Abstract
Unmanned surface vehicle (USV) has witnessed a rapid growth in the recent decade and has been applied in various practical applications in both military and civilian domains. USVs can either be deployed as a single unit or multiple vehicles in a fleet to conduct ocean missions. Central to the control of USV and USV formations, path planning is the key technology that ensures the navigation safety by generating collision free trajectories. Compared with conventional path planning algorithms, the deep reinforcement learning (RL) based planning algorithms provides a new resolution by integrating a high-level artificial intelligence. This work investigates the application of deep reinforcement learning algorithms for USV and USV formation path planning with specific focus on a reliable obstacle avoidance in constrained maritime environments. For single USV planning, with the primary aim being to calculate a shortest collision avoiding path, the designed RL path planning algorithm is able to solve other complex issues such as the compliance with vehicle motion constraints. The USV formation maintenance algorithm is capable of calculating suitable paths for the formation and retain the formation shape robustly or vary shapes where necessary, which is promising to assist with the navigation in environments with cluttered obstacles. The developed three sets of algorithms are validated and tested in computer-based simulations and practical maritime environments extracted from real harbour areas in the UK.
Highlights
By witnessing the advance of technologies in robotics and autonomous systems (RAS) in recent decades, an growing interest has been cast on the development of unmanned surface vehicles (USVs) to support complex maritime missions
The input layer and each hidden layer are applied with Rectified Linear Unit (ReLU) activation functions
A large yellow squared area in the bottom right part of the simulation environment demonstrates the dimension of a destination area, and any movement steps into this area will be regarded as successfully reaching the goal point
Summary
By witnessing the advance of technologies in robotics and autonomous systems (RAS) in recent decades, an growing interest has been cast on the development of unmanned surface vehicles (USVs) to support complex maritime missions. Reference [9] designed a DRL based controller using deep deterministic policy gradient (DDPG) to achieve a self-learning capability to robustly follow a guidance trajectory. Initial studies have been undertaken to investigate USV formation path planning These methods require a holistic navigation environment modelling and complex mathematical calculations for target point assignment. Most of the present studies only validate the algorithms in a simple 2D grid map with obstacles been modelled with regular shapes and the performance of RL based path planning in practical maritime environments needs to be investigated. In this paper, to resolve the above-mentioned issues, new deep reinforcement learning based path planning algorithms have been proposed and designed for single USV and USV formations applications.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
