Adaptive Dynamic Programming with Reinforcement Learning on Optimization of Flight Departure Scheduling

Abstract

The intricacies of air traffic departure scheduling, especially when numerous flights are delayed, frequently impede the implementation of automated decision-making for scheduling. To surmount this obstacle, a mathematical model is proposed, and a dynamic simulation framework is designed to tackle the scheduling dilemma. An optimization control strategy is based on adaptive dynamic programming (ADP), focusing on minimizing the cumulative delay time for a cohort of delayed aircraft amidst congestion. This technique harnesses an approximation of the dynamic programming value function, augmented by reinforcement learning to enhance the approximation and alleviate the computational complexity as the number of flights increases. Comparative analyses with alternative approaches, including the branch and bound algorithm for static conditions and the first-come, first-served (FCFS) algorithm for routine scenarios, are conducted. Moreover, perturbation simulations of ADP parameters validate the method’s robustness and efficacy. ADP, when integrated with reinforcement learning, demonstrates time efficiency and reliability, positioning it as a viable solution for decision-making in departure management systems.