A transfer learning model for cognitive electronic reconnaissance of unmanned aerial vehicle: Experiments

Abstract

Applying Deep Reinforcement Learning (DRL) technologies to Unmanned Aerial Vehicle (UAV) electronic reconnaissance is one of the current research hotspots. However, simulation and engineering practice show that due to poor generalization of DRL models, the performance of Cognitive Electronic Reconnaissance (CER) policies based on training will significantly decrease when the mission scene undergoes slight changes. To address this issue, we thoughtfully combine the mission area segmentation technique with transfer DRL and propose a difference-adaptive transfer DRL algorithm. This algorithm involves mission subarea segmentation, subarea pre-training, multi-subarea policy transfer, and multi-subarea splicing, providing an efficient solution to the convergence problem of the DRL algorithm caused by mission space expansion and reward sparsity. Additionally, a general CER transfer learning simulator is developed based on the analysis of the capabilities of the maneuvering platform and electronic reconnaissance payload. Multiple sets of CER policy transfer learning experiments are designed for different mission spaces, mission difficulties, and UAV characteristics. Compared with the algorithm baseline, our designed policy model significantly outperforms: the mission completion rate of UAVs in multi-scale mission spaces improves by up to 37.4%, reaching 97.5%, while the training time is reduced by 2.46 h. Further behavior analysis shows that this policy model enables UAVs to exhibit target tracking behaviors such as hovering and sustained approach.