Aviation System Performance Metrics Research Articles

Count Models to Represent the Impacts of Weather and Infrastructure on Flight Disruptions

This paper explores the application of count models to represent the relationship between flight disruptions and weather. Throughout the world, flights are regularly disrupted by delays at airports and in the terminal airspace, and less frequently by diversions and cancelations. Many delay studies have been conducted for large American and European airports, in part due to the availability of high-quality data. However, such high-quality data is not as readily available for other airports throughout the world. In this study, excess-zero count models are built using a publicly available dataset for Iqaluit Airport (YFB) in Northern Canada, to determine the influence of different weather components on disruption counts. Visibility and crosswind speeds are shown to have the largest influence on flight disruptions. The models are also applied using Aviation System Performance Metrics (ASPM) flight data for Anchorage Airport (ANC) in Alaska; the data is systematically degraded to match completeness of the Iqaluit data to test the models. The results verify that an excess-zero model using incomplete data yields results similar to that of a count model with complete data, demonstrating that an excess-zero model can overcome data incompleteness to yield acceptable results. Although count models have been applied extensively in the transportation literature, the authors believe this to be the first application to flight disruptions, and the first quantitative model of operations at a northern Canadian airport. This paper demonstrates that challenges in data availability—the case for most airports throughout the world—can be addressed with novel statistical modeling applications, and thus, delay studies can be conducted for almost any airport.

Taxi Event Extraction from Surveillance for Surface Performance Evaluation

Estimated unimpeded taxi times can be used to quantify a flight’s taxiing delay when compared with its actual taxi time. Currently, flight unimpeded taxi times are calculated by the Federal Aviation Administration using a regression method where flights in the Aviation System Performance Metrics data are clustered on the season of operation, airline, airport, and calendar year. The method uses the airline-reported gate-Out, wheels-Off, wheels-On, gate-In (OOOI) times rounded to the nearest minute. For nonreporting airlines, these times are estimated from similar flights. The purpose of this paper is to evaluate the unimpeded time using a surveillance-based approach by identifying the time a flight spent waiting in the system and comparing with the total taxi time. This study focuses on analyzing both arrivals and departures for six U.S. airports (Atlanta, Charlotte, Denver, Houston, New York Kennedy, and Chicago O'Hare) during the month of July 2015. The study results show how the non-surveillance-based and surveillance-based metrics compare, with nonsurveillance methods including the current regression method and the 5th-to-15th clustering method described in current literature. The benefits of using surveillance information for surface performance are explored, as spatial analyses allow for quick identification of taxiway locations that were the most susceptible to delays.

Failures of critical systems at airports: Impact on aircraft operations and safety

Failures of critical systems at airports can significantly impact aircraft operations and cause substantial safety risks. This study develops a generalized method for measuring the impacts of unscheduled communication, navigation and surveillance systems outages on airport operational performance (i.e. throughput) and safety. The proposed method consists of two separate models – the aggregate and the disaggregate model. The aggregate model is a regression model that analyzes flight data, which are averaged per quarter hour and obtained from the Aviation System Performance Metrics database, and investigates the reductions in airport throughput due to an outage. The disaggregate model analyzes individual flights using the Performance Data Analysis and Reporting System database to investigate air traffic controllers’ terminal airspace traffic management procedures during an outage by examining the time-separation (i.e. headways) between consecutive arriving aircraft on each runway. More specifically, the model calculates the changes in the probability of a loss of separation and assesses the necessity of enacting a Traffic Management Initiative. The proposed method is tested on the case of two localizer outages at Los Angeles International Airport. The case-study results reveal that the throughput decreased after air traffic controllers enacted a Ground Delay Program during the first observed outage. The probability of a loss of separation doubled and the smaller aircraft were diverted to other airports. Further results indicate that arriving headways can be better described by a combination of two probability distributions, rather than a single distribution, in order to more accurately capture a complexity of air traffic controllers’ actions. The method can help aviation regulators: (1) make better decisions about the value of equipment and redundancies, (2) analyze operational and safety degradations caused by equipment failures, and (3) develop appropriate traffic management procedures and improve existing ones.

Refinements to a Procedure for Estimating Airfield Capacity

This paper presents a method for obtaining airfield capacity estimates using historical data from FAA's Aviation System Performance Metrics (ASPM) database. The process first involves merging individual flights and quarter-hour airport runway operations data sets from ASPM to create a new data set. Data for Newark Liberty International Airport (EWR) in New Jersey and San Diego International Airport in California from 2006 to 2011 were used. Then, filters for meteorological condition, runway configuration, called rates, and fleet mix were applied to the two airport data sets. The filtered data sets were then used in a censored regression model of capacity that included queue length (number of aircraft waiting to arrive or depart) and arrival–departure throughput count splits as independent variables. These attributes were found to affect airfield capacity at statistically significant levels, and parameters had expected signs and magnitudes. Additionally, capacities under ideal conditions were found to be reasonably close to other sources. The model also confirmed that average capacities at EWR during hours when a ground delay program (GDP) was running were lower than when there was no GDP in effect. The method described in this paper could be used to more precisely quantify airfield capacities in specific conditions of particular interest to air traffic controllers and airport operators to better facilitate decisions that rely heavily on a good understanding of capacity in these conditions. The data exploration and preparation undertaken as part of the study reveal some of the finer points of the ASPM data and how they can be used in a more meaningful way for airfield capacity estimation.

Evaluation of the Performance of Ground Delay Programs

Ground delay programs (GDPs) are frequently used to keep U.S. air transportation safe and efficient. Most research on GDPs has focused on optimal design and implementation, with little attention given to retrospective performance evaluation. This research fills that gap by identifying GDP performance criteria, developing associated performance metrics, and evaluating GDP performance metrics across airports and over time. GDP performance criteria are established, and associated performance metrics are specified for five performance goals: capacity utilization, efficiency, predictability, equity, and flexibility. By defining multiple performance metrics, this research enables FAA traffic managers and flight operators to review GDP performance after the fact in a comprehensive way and to uncover GDP performance trends across airports and over time. Through use of data from the FAA Aggregate Demand List and the FAA Aviation System Performance Metrics, historical GDP performance is assessed for San Francisco International Airport (SFO) in California and Newark Liberty International Airport (EWR) in New Jersey for 2006 and 2011. For both airports, capacity utilization and efficiency scores are high, on average, and reflect the importance that FAA and the flight operator community attach to making effective use of available capacity and keeping air transport efficient and safe. In contrast, predictability performance is weaker and more variable. Lack of consensus on how predictability should be measured or valued could have diminished the importance of this measure in GDP decision making. On average, SFO GDPs have higher capacity utilization and predictability, whereas EWR GDPs are more efficient, equitable, and flexible. A comparison of results for 2006 and 2011 shows that GDPs were more predictable but capacity was used less effectively in 2011.

Deconstructing delay: A non-parametric approach to analyzing delay changes in single server queuing systems

This paper introduces an empirically driven, non-parametric method to isolate and estimate the effects that changes in demand and changes in throughput have on delay – in particular, arrival and departure flight delay at airport runways. Classic queuing concepts were used to develop a method by which an intermediate, or counterfactual, queuing scenario could be constructed, to isolate the delay effects due to shifts in demand and throughput. This method includes the development of a stochastic throughput function that is based entirely on data and has three key features. Firstly, the function relies on non-parametric, empirically-based probability distributions of throughput counts. Secondly, facility capacity needs not be explicitly defined, as it is implicitly included in the probability distributions of throughput. Thirdly, the throughput performance function preserves the effect of factors that cause capacity (and, therefore, throughput) to fluctuate over a given period. Temporal sequences of high, moderate, and low capacity are maintained between the observed and counterfactual scenarios. The method was applied to a case study of the three major New York area airports of LaGuardia (LGA), Newark Liberty (EWR), and John F. Kennedy (JFK), using operational data extracted from the Federal Aviation Administration’s (FAA’s) Aviation System Performance Metrics (ASPM) database. The focus was on the peak summer travel seasons of 2006 and 2007, as these airports experienced record levels of delay in 2007. The results indicate that decreases in both demand and throughput were experienced at LGA and EWR, although the decreases in throughput had more significant effects on operational delays as they increased overall at these airports. At JFK, the increase in departure throughput was not sufficient to offset the increase in departure demands. For arrivals, demand increased and throughput decreased. These trends caused a significant growth in delay at JFK between 2006 and 2007.

Improving quality of prediction in highly dynamic environments using approximate dynamic programming

AbstractIn many applications, decision making under uncertainty often involves two steps—prediction of a certain quality parameter or indicator of the system under study and the subsequent use of the prediction in choosing actions. The prediction process is severely challenged by highly dynamic environments that particularly involve sequential decision making, such as air traffic control at airports in which congestion prediction is critical for smooth departure operations. Taxi‐out time of a flight is an excellent indicator of surface congestion and is a quality parameter used in the assessment of airport delays. The regression, queueing, and moving average models have been shown to perform poorly in predicting taxi‐out times because they are slow in adapting to the changing airport dynamics. This paper presents an approximate dynamic programming approach (reinforcement learning, RL) to taxi‐out time prediction. The taxi‐out prediction performance was tested on flight data obtained from the Federal Aviation Administration's (FAA) Aviation System Performance Metrics (ASPM) database on Detroit International (DTW), Washington Reagan National (DCA), Boston (BOS), New York John F. Kennedy (JFK) and Tampa International (TPA) airports. For example, at the Boston airport (presented in detail) the prediction accuracy by RL model was 14running‐average model. In general, the RL model was 35–50% more accurate than the regression model for all of the above airports. Copyright © 2010 John Wiley & Sons, Ltd.

Airport Taxi-Out Prediction Using Approximate Dynamic Programming

Flight delay is one of the pressing problems that have far-reaching effects on society and the nation's economy. A primary cause of flight delay in the National Airspace System is high taxi-out times (time between gate push-back and wheels-off) at major airports. Accurate prediction of taxi-out time is needed to make downstream schedule adjustments and for better departure planning, which could mitigate delays, emissions, and congestion on the ground. However, accurate prediction of taxi-out time is difficult because of uncertainties associated with the dynamically changing airport operation. A novel stochastic approximation scheme based on reinforcement learning (RL) is presented for predicting taxi-out times in the presence of weather and other departure-related uncertainties. The prediction problem is cast in a probabilistic framework of stochastic dynamic programming and solved by using approximate dynamic programming approaches (particularly RL). The strengths of the method are that it is nonparametric, unlike regression models with fixed parameters, it is highly adaptable to the dynamic airport environment since it is learning based, it is scalable, it is inexpensive since it does not need highly sophisticated surface management systems, and it can effectively handle uncertainties because of the probabilistic framework. Taxi-out prediction performance was tested on data obtained from the FAA Aviation System Performance Metrics database on Detroit International and Washington Reagan National Airports. Results show that the root-mean-square prediction error calculated 15 min before gate departure time is on average 2.9 min for about 80% of the predicted flights.

Procedural and Operational Consequences of Navigational Equipment Outages: Exploration of Airport Performance

This paper explores the impact of unscheduled navigational and surveillance equipment outages on airport performance during visual flight rules and instrument flight rules at San Francisco International Airport and Phoenix Sky Harbor International Airport. The following most common unscheduled short outages, which occurred during January 2000-December 2002, are analyzed in detail: very high frequency omnidirectional range (VOR), approach lighting system type 2 (ALSF-2), the secondary radar Mode S, and air traffic control radio beacon system (ATCRBS). We explore the air traffic control procedures and responses to unscheduled outages, and develop a methodology to examine the effects of unscheduled outages on airport performance. Two Federal Aviation Administration databases, maintenance management system and aviation system performance metrics, are used to construct censored regression models (i.e., Tobit models) to assess the airport throughput performance during such unscheduled outages. It is found that airport arrival and departure throughputs do not deteriorate with VOR and ALSF-2 outages due to availability of air traffic procedures that allow alternative approaches into the airport. However, the unscheduled outages of secondary radar Mode S and ATCRBS do cause airport throughput degradations.

Measuring the Performance of the National Airspace System in the En Route Environment

Three metrics that can be used to evaluate the performance of the National Airspace System (NAS) in the en route environment are defined and explored. The metrics are excess arrival center transit times, excess Air Route Traffic Control Center (ARTCC) transit times, and airborne holding. These metrics can be used to identify en route problem areas on a next day basis. The relationship of airborne holding to the two excess transit time metrics is investigated, as well as the relationship of airborne holding to weather and other delay metrics currently reported by the Federal Aviation Administration (FAA). Since these metrics provide timely and useful information about system performance not available elsewhere, it is recommended that they be added to the FAA’s on-line report, Aviation System Performance Metrics.