Reinforcement Learning for Multiaircraft Autonomous Air Combat in Multisensor UCAV Platform

Abstract

Autonomous air combat has received significant attention from researchers working on artificial intelligence (AI) applications. Most previous research on autonomous air combat has focused on one-on-one air combat scenarios in which air combat situational information is considered to be precisely observable. However, most modern air combats are conducted in formations, where air combat situational information is obtained from multiple sensors. Therefore, we introduce a novel automated maneuver decision architecture for close-range multi-aircraft air combat scenarios under the multi-sensor UCAV platform that can handle air combat scenarios with variable-sized formations. Then, a multi-agent reinforcement learning algorithm is proposed to obtain the strategy. The training performance of the training algorithm is evaluated and the obtained strategy is analyzed in different air combat scenarios it is found that these formations exhibit effective cooperative behavior in symmetric and asymmetric situations. Finally, we give ideas for the engineering implementation of a maneuver control architecture. This study provides a solution for future multi-aircraft autonomous air combat.