Coordination of distributed unmanned surface vehicles via model-based reinforcement learning methods

Abstract

This article presents a coordination algorithm for organizing a fleet of unmanned surface vehicles (USVs) to search multiple moving object targets in the ocean environment. During the fleet maneuver, USVs can exchange local sensing information through a wireless communication network. Based on both received and self-perceived information, a USV constructs a grid confidence map, which reflects how well the USV fleet perceives on every region of the search area. Then, the USV coordination is modeled as a reinforcement learning (RL) problem, where reward functions are defined based on the information obtained from the grid confidence map. Therefore, USVs are encouraged to explore new regions and prevented visiting the already searched areas. Search routes of USVs are calculated via a policy-iteration based path planning algorithm, while inter-vehicle collisions are avoided by applying policy constraints. Real-world experiments were conducted in the ocean environment to evaluate the validity of the proposed method. Compared to the conventional formation control strategy and the uncoordinated algorithm, experiment results show that the proposed method is more intelligent and efficient for searching object targets.