Abstract
Unmanned surface vehicle (USV) is a robotic system with autonomous planning, driving, and navigation capabilities. With the continuous development of applications, the missions faced by USV are becoming more and more complex, so it is difficult for a single USV to meet the mission requirements. Compared with a single USV, a multi-USV system has some outstanding advantages such as fewer perceptual constraints, larger operation ranges, and stronger operation capability. In the search mission about multiple stationary underwater targets by a multi-USV system in the environment with obstacles, we propose a novel cooperative search algorithm (CSBDRL) based on reinforcement learning (RL) method and probability map method. CSBDRL is composed of the environmental sense module and policy module, which are organized by the “divide and conquer” policy-based architecture. The environmental sense module focuses on providing environmental sense values by using the probability map method. The policy module focuses on learning the optimal policy by using RL method. In CSBDRL, the mission environment is modeled and the corresponding reward function is designed to effectively explore the environment and learning policies. We test CSBDRL in the simulation environment and compare it with other methods. The results prove that compared with other methods, CSBDRL makes the multi-USV system have a higher search efficiency, which can ensure targets are found more quickly and accurately while ensuring the USV avoids obstacles in time during the mission.
Highlights
Unmanned surface vehicle (USV) is a robot system with capabilities of autonomous planning, driving, and navigation [1]
A novel cooperative search learning algorithm (CSBDRL) based on reinforcement learning (RL) method and probability map method is proposed to apply in the search mission for multiple stationary underwater targets by a multi-USV system in the environment with obstacles. e proposed algorithm is composed of the environmental sense module and policy module, which are organized by the “divide and conquer” policy-based architecture. e environmental sense module focuses on providing environmental sense values. e policy module focuses on how to learn the optimal policy
Tian et al [43] propose a cooperative search algorithm combining genetic algorithm (GA) and model predictive control (MPC) to solve the search problem in an uncertain environment. is algorithm uses a probability map to describe the uncertainty of the mission area, takes the gains of the information as the optimization target, and uses GA to solve the problem of the optimal control input
Summary
Unmanned surface vehicle (USV) is a robot system with capabilities of autonomous planning, driving, and navigation [1]. Millet et al [13] propose a distributed search algorithm which includes both map update and fusion procedure Based on this algorithm, Hu et al [14] design an algorithm for a cooperative target search mission called coverage control path planning. A novel cooperative search learning algorithm (CSBDRL) based on RL method and probability map method is proposed to apply in the search mission for multiple stationary underwater targets by a multi-USV system in the environment with obstacles. E results prove that, compared with other algorithms, CSBDRL makes the multiUSV system have a higher search efficiency, which can make sure targets are found more quickly and accurately while ensuring the USV avoids obstacles in time during the mission.
Sign-in/Register to view highlights & summary 
Published Version (
Free)
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call