An evolutionary multi-agent reinforcement learning algorithm for multi-UAV air combat

Abstract

In recent years, multi-agent reinforcement learning (MARL) has been widely used as a multi-UAV autonomous countermeasure. However, because training is usually performed in a self-play manner, the existing algorithms may suffer from strategic cycles without progress. Moreover, in complex air combat, the state dimensions change dynamically with the varying number of unmanned aerial vehicles (UAVs), making it difficult for existing MARL algorithms with fixed-input-size network architectures to effectively handle the variations. Therefore, we propose an evolutionary MARL method that can avoid cycles through the evolution of a population of UAV groups. In addition, an attention mechanism with death masking was adopted to handle dynamic changes in the number of UAVs. Experimental results conducted in our constructed multi-UAV air combat environment show that our method outperforms state-of-the-art methods and can help UAVs generate effective maneuver strategies to defeat opponents.