Flight diversions due to adverse weather have a significant negative impact on airspace users, airports, and the environment. Previous work proposed a tree-based model that learned which flights are more likely to be diverted due to adverse weather conditions at the destination airport from historical traffic and weather data. This model was trained using confident learning to exclude from training unpredictable flight diversions caused by reasons other than weather, such as medical emergencies or technical problems. Aggregated performance metrics computed on the test set demonstrated high precision and moderate recall, and inspection of the Shapley values unravelled the most important features. Initial human observations made during the live trial, however, have raised a potential connection between the quality of predictions and the specific weather conditions that lead to flight diversions. This study aims to provide empirical evidence to support this hypothesis. To accomplish this, a data-driven methodology should be used to characterise the main conditions that cause flight diversions. Concurrently, the effectiveness of the model in each of these conditions should be evaluated. The dataset under consideration, however, presents a challenge because it contains a plethora of high-cardinality categorical features (e.g., destination airport and aircraft operator) as well as skewed numerical features with a wide range of scales (e.g., visibility and wind gust). This scenario significantly increases the difficulty of characterising the main conditions that cause flight diversions and conducting subsequent analyses even with established methods like factor analysis of mixed data. To address this challenge, this paper proposes to utilise supervised clustering, a method that combines state-of-the-art feature attribution, dimensionality reduction, and clustering algorithms to identify the most representative decisions of a model. Following that, the distinctive characteristics of the various decisions (e.g., extreme obscuration at the airport), each associated with a cluster, are determined by examining them. Finally, conventional classification and uncertainty metrics are computed for each cluster to assess the performance of the model. This paper shows that supervised clustering is effective in characterising flight diversions due to weather, and provides valuable insights into the performance of the model in different conditions, highlighting situations where predictions require careful consideration.
Read full abstract