Real-time scheduling for carrier-borne aircraft support operations: a reinforcement learning approach

Abstract

The carrier-borne aircraft dispatch rate is an important indicator to measure the combat performance of an aircraft carrier. The key factor affecting the carrier-borne aircraft dispatch rate is the efficiency of the carrier aircraft support operation scheduling. Shipboard aircraft support operation scheduling refers to a rational arrangement of the order of support operations required by the carrier aircraft as well as to an efficient completion of the support operations for the carrier-borne aircraft under constraints of limited time, space, and resources. The existing solution strategies based on optimization methods (dynamic programming, linear programming, etc.) and heuristic methods (genetic algorithm, particle swarm optimization, etc.) are only suitable for operation scheduling in the case of predictable operations, and it is challenging to meet the real-time support operation scheduling requirements in highly dynamic combat scenarios. For this reason, a new real-time scheduling method for carrier-borne aircraft support operations based on deep Q-networks (DQNs) is proposed. This method consists of modeling the scheduling problem of aircraft support operations as a partially observable Markov decision process (POMDP) problem. The global and long-term benefits are used to optimize the scheduling process, and the decision-making framework of offline learning and online deployment is used to solve the scheduling problem of aircraft support operations. The simulation results demonstrate that this new method can significantly improve the efficiency of shipboard aircraft support operation scheduling, thus enabling to meet the needs of real-time decision-making environments.