Aircraft routing using dynamic programming and reinforcement learning: A customer-centric approach

Abstract

Over the past few decades, the airline industry has evolved into a sophisticated business, with conventional objectives such as operational efficiency, effective fleet assignment, flight punctuality, and equitable crew scheduling being the primary focus. Nevertheless, given the growing accessibility to customer feedback (either through public forums or private data), it is crucial to make business decisions that cater to the customers’ requirements in addition to conventional goals. Within aircraft operations, there is a series of decisions to be made, such as flight scheduling, fleet assignment, aircraft routing, and crew scheduling. After designating a fleet type to each flight leg, airlines ascertain the sequence of flight legs, or strings, to be operated by individual aircrafts, while complying with obligatory maintenance checks. This constitutes the aircraft routing problem.We introduce a novel approach to aircraft routing problem, by infusing traditional methodologies with insightful customer feedback to create a more responsive and customer-centric model. Unlike conventional techniques of constructing routes, in our work, we construct maintenance feasible strings. Traditional aircraft routing methodologies typically concentrate on minimizing propagated delay. However, by integrating customer feedback, more effective prioritization to minimize delays for flight legs with higher customer dissatisfaction can be achieved. We propose an optimization model that strives to minimize propagated delay while prioritizing flights based on customer feedback, thus resulting in customer-aware routes. To solve the proposed optimization model, we design an iterative alternating optimization scheme where feasible strings are first constructed from the pool of available flight legs using a string generation procedure, followed by solving an integer linear optimization problem over only the feasible set of strings obtained in the first stage. To construct feasible strings of flights at each iteration, we use two different methods. The first method is based on a dynamic programming approach which is useful for mean delay information. To handle practical cases where delays are uncertain, we propose a reinforcement learning (RL) approach to construct the strings. We provide experiments of our proposed methods on synthetic and real data sets and demonstrate the effectiveness of our RL model in comparison to dynamic programming based model. Our experiments affirm that routing decisions informed by customer feedback prioritize minimizing delays for specific flight legs, resulting in routes that are inherently customer-friendly.