Secure and Cooperative Target Tracking via AUV Swarm: A Reinforcement Learning Approach

Abstract

The autonomous underwater vehicle (AUV) has gradually become an important platform for performing various underwater tasks. Due to the shortcomings resulting from a single AUV's poor detection, information processing and moving capabilities, more and more tasks are completed in a cooperative manner by multiple AUVs. However, most of the existing works do not consider security factors in the process of multi-AUV cooperation. In this paper, we propose a novel cooperative tracking scheme towards an underwater moving target, performed by an intelligent AUV swarm. In this scheme, a cooperative multi-agent reinforcement learning (MARL) based tracking algorithm is proposed following a centralized training with distributed execution (CT-DE) manner. After centralized training in the designed secure private network, no information sharing is required during the mission execution. This feature ensures the security of the whole system, especially in a complex confrontation scenario. In addition, we build models of the AUV underwater dynamics and the target sonar detection, which make the algorithm applicable to real target tracking enabled AUV swarms. Then, based on the multi-agent deep deterministic policy gradient (MADDPG) algorithm, we design an end-to-end AUV control algorithm. Simulation results validate that the proposed algorithm can achieve competitive performance in tracking success rate and tracking stability against baselines, while ensuring the security of the entire system.