Improving defensive air battle management by solving a stochastic dynamic assignment problem via approximate dynamic programming

Abstract

Military air battle managers face several challenges when directing operations during quickly evolving combat scenarios. These scenarios require rapid assignment decisions to engage moving targets having dynamic flight paths. In defensive operations, the success of a sequence of air battle management decisions is reflected by the friendly force’s ability to maintain air superiority and defend friendly assets. We develop a Markov decision process (MDP) model of a stochastic dynamic assignment problem, named the Air Battle Management Problem (ABMP), wherein a set of unmanned combat aerial vehicles (UCAV) must defend an asset from cruise missiles arriving stochastically over time. Attaining an exact solution using traditional dynamic programming techniques is computationally intractable. Hence, we utilize an approximate dynamic programming (ADP) technique known as approximate policy iteration with least squares temporal differences (API-LSTD) learning to find high-quality solutions to the ABMP. We create a simulation environment in conjunction with a generic yet representative combat scenario to illustrate how the ADP solution compares in quality to a reasonable, closest-intercept benchmark policy. Our API-LSTD policy improves mean success rate by 2.8% compared to the benchmark policy and offers an 81.7% increase in the frequency with which the policy performs perfectly. Moreover, we find the increased success rate of the ADP policy is, on average, equivalent to the success rate attained by the benchmark policy when using a 20% faster UCAV. These results inform military force management and defense acquisition decisions and aid in the development of more effective tactics, techniques, and procedures.