Pursuit-evasion game strategy of USV based on deep reinforcement learning in complex multi-obstacle environment

Abstract

Aiming at the confrontation game problems between pursuit-evasion unmanned surface vehicles under complex multi-obstacle environment, a pursuit-evasion game strategy is proposed. Firstly, the multi-obstacle environment is set up, and the gaming situation can be judged by the perception between pursuit-evasion USVs. For the pursuers, the model training is performed based on multi-agent deep reinforcement learning, so that they can quickly plan a reasonable obstacle avoidance and pursuit route, and form an effective encirclement posture before the evader approaches the target point. Meanwhile, the credit assignment problem among the members of the pursuing group is considered. For the evader, deep reinforcement learning is combined with imitation learning to train the escape model, so that it can reach the preset point in as short a time as possible and avoid the obstacles on the way. Finally, an adversarial-evolutionary game training method under multiple random scenarios is designed and combined with curriculum learning to iteratively update the pursuit and escape models. Through the detailed comparative analysis of the model training process and simulation experiments, it is proved that the proposed two types of models have higher convergence efficiency and stability, and they can have higher intelligence to pursue, escape and avoid obstacles respectively.