Aviation Safety Reporting System Research Articles

A novel hybrid STL-transformer-ARIMA architecture for aviation failure events prediction

Accurate prediction of aviation failure events helps to anticipate future safety situations and protect against further uncontrollable accidents. However, the large sample size, complex temporal characteristics, and significant long-term correlation of aviation failure events increase the operational cost of accurate prediction. To address these challenges, this paper proposes a novel approach involving seasonal-trend decomposition using Loess (STL) and a hybrid prediction model consisting of a transformer and autoregressive integrated moving average (ARIMA). First, STL decomposition is utilized to isolate trend, seasonal, and remainder components, contributing to a comprehensive understanding of the events sample characteristics. The trend component is then trained and predicted using transformer, solving the vanishing gradient problem and improving computational efficiency. ARIMA is employed to train and predict the seasonal and remainder components, maintaining accuracy while reducing complexity. Finally, a comparative evaluation between the proposed and multiple existing approaches is conducted using Aviation Safety Reporting System (ASRS) data. The results demonstrate that the STL-transformer-ARIMA provides more accurate predictions of failure events than single model. It also exhibits significant advantages in robustness and generalization capacity compared to single transformer-based predictors. This revealed that the proposed approach performed better in predicting aviation failure events.

Cognitive Processing Disruptions Affecting Flight Deck Performance: Implications for Cognitive Resilience

The flight deck of a commercial aircraft has become progressively digitized and operates in multiple modes with displays and indicators that require increasing levels of comprehension. Examining several aspects of cognitive processing is important to understand how threats to safety might occur and what actions might be taken to reduce severity or to eliminate the threat altogether. This paper presents the elements of cognition to consider, relevant characteristics of working memory and cognitive processing speed, types of disruptions and how they are addressed, results from overload or confusion, and the need for effective cognitive resilience to recover and repair the threat. Data from Aviation Safety Reporting System (ASRS) databases indicate 30% of cases could represent a distinct threat of cognitive overload. These are evaluated to identify sources and likelihood for surprise disruptions and to assess the potential of cognitive resilience. Adaptation of the CRM-TEM model is considered for potential application in training and investigations.

Examining the Potential of Generative Language Models for Aviation Safety Analysis: Case Study and Insights Using the Aviation Safety Reporting System (ASRS)

This research investigates the potential application of generative language models, especially ChatGPT, in aviation safety analysis as a means to enhance the efficiency of safety analyses and accelerate the time it takes to process incident reports. In particular, ChatGPT was leveraged to generate incident synopses from narratives, which were subsequently compared with ground-truth synopses from the Aviation Safety Reporting System (ASRS) dataset. The comparison was facilitated by using embeddings from Large Language Models (LLMs), with aeroBERT demonstrating the highest similarity due to its aerospace-specific fine-tuning. A positive correlation was observed between the synopsis length and its cosine similarity. In a subsequent phase, human factors issues involved in incidents, as identified by ChatGPT, were compared to human factors issues identified by safety analysts. The precision was found to be 0.61, with ChatGPT demonstrating a cautious approach toward attributing human factors issues. Finally, the model was utilized to execute an evaluation of accountability. As no dedicated ground-truth column existed for this task, a manual evaluation was conducted to compare the quality of outputs provided by ChatGPT to the ground truths provided by safety analysts. This study discusses the advantages and pitfalls of generative language models in the context of aviation safety analysis and proposes a human-in-the-loop system to ensure responsible and effective utilization of such models, leading to continuous improvement and fostering a collaborative approach in the aviation safety domain.

Flying the not-so-friendly skies: airline passenger misconduct, 1999-2020

ABSTRACT Over the past several years, media reports of airline passenger misconduct have climbed sharply. Research has shown that while industry-level factors such as delayed and canceled flights, poor customer service, and overcrowded seating contribute to increased unruly behavior, additional contextual elements related to personal use of alcohol (prohibited and served) have also played a role in this shift. Drawing on data from more than 915 incidents voluntarily reported to the Aviation Safety Reporting System (ASRS) over a 21-year time period, we use a multi-level case classification coding process to examine three broad types of passenger misconduct – physical, verbal, and other general types. Our results highlight the diverse range of behaviors associated with unruly airline passengers and offer insights into how environmental factors, spatial context, weak enforcement of existing federal legislation, as well as industry structures and practices, create pathways for offender behavior.

Qualitative Analysis of General Aviation Pilots’ Aviation Safety Reporting System Incident Narratives Using the Human Factors Analysis and Classification System

ABSTRACT NASA's Aviation Safety Reporting System (ASRS) gathers data from voluntary aviation safety reports for nonfatal accidents and incidents. These reports are a valuable resource in identifying unsafe occurrences and hazardous situations in the aviation industry. A tool known as the Human Factors Analysis and Classification System (HFACS), initially developed for military use, has proven useful for root cause analysis of human causes in aviation accidents and incidents, including in commercial contexts. This research study utilizes HFACS to classify the ASRS safety reports to identify key indicators in General Aviation (GA) non-fatal accidents and incidents. Qualitative data analysis reveals relationships between incidents and causal factors, indicating 53% of incidents to be perceptual errors, 32% decision errors, and 26% skill-based errors. Prevalent preconditions for these errors include crew resource management (53%), physical environment (32%), and technological environment (16%). The study aligns with previous research findings, suggesting that crew resource management is a common factor in both GA incidents and commercial and military aviation. While GA pilots may make different types of errors, such discrepancies are not prevalent in the voluntarily reported data.

Research on Aviation Safety Prediction Based on Variable Selection and LSTM.

Accurate prediction of aviation safety levels is significant for the efficient early warning and prevention of incidents. However, the causal mechanism and temporal character of aviation accidents are complex and not fully understood, which increases the operation cost of accurate aviation safety prediction. This paper adopts an innovative statistical method involving a least absolute shrinkage and selection operator (LASSO) and long short-term memory (LSTM). We compiled and calculated 138 monthly aviation insecure events collected from the Aviation Safety Reporting System (ASRS) and took minor accidents as the predictor. Firstly, this paper introduced the group variables and the weight matrix into LASSO to realize the adaptive variable selection. Furthermore, it took the selected variable into multistep stacked LSTM (MSSLSTM) to predict the monthly accidents in 2020. Finally, the proposed method was compared with multiple existing variable selection and prediction methods. The results demonstrate that the RMSE (root mean square error) of the MSSLSTM is reduced by 41.98%, compared with the original model; on the other hand, the key variable selected by the adaptive spare group lasso (ADSGL) can reduce the elapsed time by 42.67% (13 s). This shows that aviation safety prediction based on ADSGL and MSSLSTM can improve the prediction efficiency of the model while keeping excellent generalization ability and robustness.

The effect of COVID-19 on self-reported safety incidents in aviation: An examination of the heterogeneous effects using causal machine learning

Introduction: Disruptions to aviation operations occur daily on a micro-level with negligible impacts beyond the inconvenience of rebooking and changing aircrew schedules. The unprecedented disruption in global aviation due to COVID-19 highlighted a need to evaluate emergent safety issues rapidly. Method: This paper uses causal machine learning to examine the heterogeneous effects of COVID-19 on reported aircraft incursions/excursions. The analysis utilized self report data from NASA Aviation Safety Reporting System collected from 2018 to 2020. The report attributes include self identified group characteristics and expert categorization of factors and outcomes. The analysis identified attributes and subgroup characteristics that were most sensitive to COVID-19 in inducing incursions/excursions. The method included the generalized random forest and difference-in-difference techniques to explore causal effects. Results: The analysis indicates first officers are more prone to experiencing incursion/excursion events during the pandemic. In addition, events categorized with the human factors confusion, distraction, and the causal factor fatigue increased incursion/excursion events. Practical Applications: Understanding the attributes associated with the likelihood of incursion/excursion events provides policymakers and aviation organizations insights to improve prevention mechanisms for future pandemics or extended periods of reduced aviation operations.

ASRS-CMFS vs. RoBERTa: Comparing Two Pre-Trained Language Models to Predict Anomalies in Aviation Occurrence Reports with a Low Volume of In-Domain Data Available

We consider the problem of solving Natural Language Understanding (NLU) tasks characterized by domain-specific data. An effective approach consists of pre-training Transformer-based language models from scratch using domain-specific data before fine-tuning them on the task at hand. A low domain-specific data volume is problematic in this context, given that the performance of language models relies heavily on the abundance of data during pre-training. To study this problem, we create a benchmark replicating realistic field use of language models to classify aviation occurrences extracted from the Aviation Safety Reporting System (ASRS) corpus. We compare two language models on this new benchmark: ASRS-CMFS, a compact model inspired from RoBERTa, pre-trained from scratch using only little domain-specific data, and the regular RoBERTa model, with no domain-specific pre-training. The RoBERTa model benefits from its size advantage, while the ASRS-CMFS benefits from the pre-training from scratch strategy. We find no compelling statistical evidence that RoBERTa outperforms ASRS-CMFS, but we show that ASRS-CMFS is more compute-efficient than RoBERTa. We suggest that pre-training a compact model from scratch is a good strategy for solving domain-specific NLU tasks using Transformer-based language models in the context of domain-specific data scarcity.

Development and application of a Bayesian network-based model for systematically reducing safety risks in the commercial air transportation system

Aiming to explore the nature of incidents in the commercial air transportation system, multiple causal factor events and result events, and their cause and effect relationships were retrieved from 7,265 incident cases in the Aviation Safety Reporting System (ASRS). The Bayesian network of commercial air transportation system safety risk (CATSSR-BN) was constructed on the basis of the hybrid algorithm of max–min hill-climbing (MMHC). The k-fold cross-validation of structure learning verified the accuracy and effectiveness of the CATSSR-BN. The measure of mutual information between causal factor event and result event was evaluated for determining its importance on safety performance of commercial air transportation system. Ten causal factor events with the highest values of mutual information across different result events were identified. A decision-making model was developed for optimizing resource investment in commercial aviation safety management. As a result, resource investment combination was determined on the combination of five causal factor events, which provided a systematic approach for reducing safety risk in commercial air transportation system and avoiding the occurrence of commercial aviation incidents with a high level of risk.

A model fusion strategy for identifying aircraft risk using CNN and Att-BiLSTM

The Aviation Safety Reporting System (ASRS) data is an important information source for risk identification in civil aviation. However, ASRS data has the characteristics of high dimensionality, unstructured and data imbalance. This brings great challenges for risk identification. In this paper, a model fusion strategy is proposed for identifying the aircraft risk by using convolutional neural network (CNN) and bidirectional long short-term memory neural network with attention mechanism (Att-BiLSTM). Firstly, all obtained ASRS data are reasonably divided into five risk levels. Secondly, the CNN is utilized to deeply mine the relationship between risk levels and structured data. Meanwhile, the Att-BiLSTM is adopted to deeply mine the relationship between risk levels and unstructured event narratives. Finally, a model fusion strategy is proposed to fuse the preliminary identifications of CNN and Att-BiLSTM to obtain the final identifications. The adopted CNN and Att-BiLSTM are compared with two individual models respectively. The experimental results show that the CNN and Att-BiLSTM have superior identification performance. Furthermore, by comparing the performance of the proposed model fusion strategy against the effect of using the individual model, the proposed model fusion strategy is effective in improving identification accuracy and reducing the influence of data imbalance in civil aviation.

Natural Language Processing of Aviation Safety Reports to Identify Inefficient Operational Patterns

With the growth in commercial aviation traffic and the need for improved environmental performance, strategies to lower emissions that can be implemented in the near term are necessary. Since novel technology takes time to enter the market, operational improvements that employ existing aircraft and require no new infrastructure are fit for this goal. While quantified data collected throughout aviation, such as arrival/departure statistics and flight data, have been well-utilized, text data collected through safety reports have not been leveraged to their full extent. In this paper, a methodology is presented that can use aviation text data to identify high-level causes of flight delays and cancellations, using delays as a metric of operational inefficiency. The dataset is extracted from the Aviation Safety Reporting System (ASRS), which includes voluntary safety incident reports in text narrative and metadata formats. The methodology uses natural language processing tools, K Means clustering, and dimensionality reduction by t-Distributed Stochastic Neighbor Embedding (t-SNE) to categorize and visualize narratives. The method identified 7 major clusters and a total of 23 sub-clusters. A comparison between the subclusters’ topics and the causes of flight delays revealed by the quantified data shows that the ASRS database provides a unique safety perspective to delay cause identification, as illustrated by the method’s identification of maintenance as the main cause of delays, rather than weather.

Reacting and responding to rare, uncertain and unprecedented events

This work examines how we may be able to anticipate, respond to, and train for the occurrence of rare, uncertain, and unexpected events in human-machine systems operations. In particular, it uses a foundational matrix which describes the combinations of the state-of-the-world and the state-of-the-respondent, to formulate preferred response strategies, contingent upon what is knowable and actionable in each circumstance. It employs the dichotomy of System I and System II forms of cognitive response and augments these perspectives with a further form of decision-making, namely Systems III. The latter is predicated upon reactions to novel, unprecedented, and even ‘unthinkable’ events. The degree to which any human operator, the associated automation and/or the autonomy of a system, or each of these acting in concert, can best deal with these ‘blue swan’ events is explored. Potential forms of remediation, especially featuring training, are discussed, and evaluated in light of the skills needed to respond to even prohibitive degrees of situational uncertainty. Practitioners summary: Practitioners are liable to witness a growing spectrum of unusual and, on occasion, even unprecedented events in the operation of systems for which they are responsible. They will be required to account for their response to these circumstances to a spectrum of involved constituencies to whom they answer. This work aids them in succeeding to bring clarity to such difficult and challenging processes. Abbreviations: K: Known; Unk: Unknown; AI: Artificial Intelligence; ML: Machine Learning; CHARM: Cockpit Human-Automation Resource Management; SDT: signal detection theory; ASRS: Aviation Safety Reporting System.

Application of structural topic modeling to aviation safety data

Data-driven frameworks for analyzing aviation safety data have recently gained traction. Text-based machine learning techniques often rely purely on word frequency analysis to eliminate the innate subjectivity of human language, but more refined techniques like structural topic modeling (STM) attempt to simulate text generation to identify the thematic undertones of text corpora. This paper presents an application of STM to two text-based sets of aviation safety data, the Aviation Safety Reporting System (ASRS) and accident and incident reports published by the National Transportation Safety Board (NTSB). A framework for cleaning and pre-processing the datasets is discussed, including a brief discussion of bag-of-words and TF–IDF representations of narratives. The methodology behind STM is described, including techniques for selecting the optimal number of topics. The results of the STM analysis on the ASRS and NTSB datasets are presented, with a focus on the clarity and specificity based on most common words associated with topics. A brief exploration of the correlation between pairs of topic labels is also undertaken, including a visualization of narratives in 2-dimensional space. STM is found to show promise in identifying themes within technical datasets, with model performance increasing for more specific corpora that use precise and unique language.

Setting up new standards in aviation industry with the help of artificial intelligent – machine learning application

In these times, dominated by innovation, technology is an important asset in optimizing maintenance, aircraft operations and even the investigation process of aviation accidents, with unprecedented results so far. All the information related to an aircraft, generated over time, can be included in a database, in order to be analyzed for better technical insights. The same approach can be applied in analyzing evidence and reports generated after an aircraft accident occurred. Aircraft investigation involves tones of hard-working hours and evidence analysis. This research aims to identify a new tool to improve the investigation process of aircraft incidents caused by human factors errors. Using artificial intelligence to process the resulted evidence during the aircraft accidents investigation, will shed light on the root cause of the incident much faster. Accelerating an investigation means huge benefits for all parties involved. In this sense, a proof-of-concept software has been developed to emphasize the power of artificial intelligent in the aircraft accidents investigation. Using machine learning algorithms, a program was trained with data, to develop an application capable of automatically determining the root cause of an accident based on witness statements. All data was derived from the Aviation Safety Reporting System database from the U.S, which was queried for incidents, occurred between 2000 and 2020. This application is not only about automating a stage in an aircraft accident investigation, is rather about setting up new standards in aviation industry with the help of artificial intelligence.

Alarms, alerts, and warnings in air traffic control: An analysis of reports from the Aviation Safety Reporting System

Signals (alarms, alerts, and warnings) are essential for alerting air traffic controllers to potential collisions and other adverse events. Excessive or misleading signals can increase response times and decrease their response rates. We used reports from the Aviation Safety Reporting System and structured interviews to understand the complexity of the controller’s tasks in the context of potentially high-consequence situations and events, and to develop design strategies to enhance the effectiveness of signals in the ATC environment. MethodsWe reviewed ASRS reports over a 6-year interval from 2015 to 2020, searching for reports that mentioned alarm, alert, or warning and were submitted by air traffic controllers. We found 370 relevant reports that we analyzed for hits, misses, false alarms, and correct rejections. Structured interviews with former controllers further explored the role of signals in air traffic control. ResultsThe most common signals in reports were MSAW (139), ASDE-X and ASSC (27), CA (195), and AMASS (4). ASDE-X, ASSC, or AMASS were mentioned in 30 reports by ground or local controllers. TRACON controllers reported events involving MSAW 70 times and CA 51 times; these were also implicated by local controllers. CA was most mentioned, cited a total of 195 times in the reports. ConclusionsThis information may help us to develop strategies that can enhance signaling modality (e.g., new auditory, visual, and tactile signals). Trust in automation may be improved by using strategies such as indicating the automation’s level of confidence in situations like impending loss of separation.

Noncompliance and Decision-Making of Airline Pilots: An Analysis of Narratives From the Aviation Safety Reporting System

The purpose of this study was to identify self-reported flight deck noncompliance in aviation safety reports and explore the relationship between adaptive expertise, deliberate vs. non-deliberate actions (errors), and intentional vs. unintentional noncompliance. The heuristics for assessing adaptive thinking and behavior were based on subscales of the Adaptive Expertise Survey (AES; Fisher & Peterson, 2001). We analyzed a random sample of 200 ASRS reports from 2019 and coded them with respect to (a) whether they described intentional or unintentional noncompliance by one or more flightcrew members, (b) whether the decision making was deliberate, and (c) whether the decision-making process involved correlates of adaptive or routine (non-adaptive) expertise. We found that unintentional noncompliance was associated most frequently with non-deliberate actions and non-adaptive behaviors. Adaptive behaviors were strongly associated with deliberate actions and intentional noncompliance. Our on-going research to investigate adaptive expertise and its relationship with predictors of noncompliance is discussed.

Identifying Incident Causal Factors to Improve Aviation Transportation Safety: Proposing a Deep Learning Approach

Aviation is a complicated transportation system, and safety is of paramount importance because aircraft failure often involves casualties. Prevention is clearly the best strategy for aviation transportation safety. Learning from past incident data to prevent potential accidents from happening has proved to be a successful approach. To prevent potential safety hazards and make effective prevention plans, aviation safety experts identify primary and contributing factors from incident reports. However, safety experts’ review processes have become prohibitively expensive nowadays. The number of incident reports is increasing rapidly due to the acceleration of advances in information technologies and the growth of the commercial and private aviation transportation industries. Consequently, advanced text mining algorithms should be applied to help aviation safety experts facilitate the process of incident data extraction. This paper focuses on constructing deep-learning-based models to identify causal factors from incident reports. First, we prepare the data sets used for training, validation, and testing with approximately 200,000 qualified incident reports from the Aviation Safety Reporting System (ASRS). Then, we take an open-source natural language model, which is well trained with a large corpus of Wikipedia texts, as the baseline and fine-tune it with the texts in incident reports to make it more suited to our specific research task. Finally, we build and train an attention-based long short-term memory (LSTM) model to identify primary and contributing factors in each incident report. The solution we propose has multilabel capability and is automated and customizable, and it is more accurate and adaptable than traditional machine learning methods in extant research. This novel application of deep learning algorithms to the incident reporting system can efficiently improve aviation safety.

The dynamics between voluntary safety reporting and commercial aviation accidents

Since 1975, the Aviation Safety Reporting System (ASRS) has collected over 1.6 million reports from aviation community members and contributed to the efficiency and safety improvement of the National Airspace System (NAS) of the United States. Despite a large number of studies using ASRS data, the dynamics between safety reporting and aviation accidents remains unclear. Focusing specifically on the Part 121 air carriers of the U.S., this paper addresses the temporal relationship between voluntary safety reporting and the occurrence of commercial aviation accidents. Due to the uncertain and potentially mixed order of integration of the time-series data, this study uses Autoregressive Distributed Lag (ARDL) bounds testing and a special Vector Autoregressive (VAR) model based on Toda and Yamamoto (1995) for data analysis and cross-validation of the results. The ARDL bounds testing finds a long-run relationship from aviation accidents to safety reporting. This finding is confirmed by the estimation results of the VAR model that aviation accidents Granger cause voluntary safety reporting. Short-run relationships identified in ADRL bounds testing and impulse-response function (IRF) of the VAR model reveal that the response of safety reporting to aviation accidents peaks in the 4th and 5th quarter after the occurrence of accidents. Understanding the inter-temporal relationship between safety reporting and aviation accidents could facilitate the interpretation of reporting data for government agencies or safety departments of airlines overseeing safety reporting systems. The short-run and long-run relationships between voluntary safety reporting and aviation accidents identified in this study for U.S. air carriers could be used as benchmarks for other national aviation safety agencies or airlines to assess their safety reporting cultures.

AIRLINE PILOT SITUATION AWARENESS: PRESENTING A CONCEPTUAL MODEL FOR META-COGNITION, REFLECTION AND EDUCATION

The dissertation research summarized here, utilized the Grounded Theory Method to develop a conceptual model of pilot situation awareness from 223 Aviation Safety Reporting System (ASRS) narratives. The application of Latent Semantic Analysis aided the theoretical sampling of ASRS reports. A multistage model was developed involving attention, perception, interpretation, decision making, and action in support of goal-driven behavior. Narrative report coding identified several categories of situation awareness elements that pilots direct their attention to in building and maintaining situation awareness. Internal to the aircraft, flight crews directed their attention to the aircraft’s flight state and automation state. They also directed their attention to the condition of the aircraft, the functioning of the crew, and the status of the cabin. External to the aircraft, flight crews directed their attention to airport conditions, air traffic control, terrain, traffic, and weather. Pilots were also aware of the passage of time. Twelve characteristics of situation awareness were identified from narrative report coding which were subsequently compared with existing theoretical perspectives of situation awareness.

Impact of COVID -19 on Pilot Proficiency – A Risk Analysis

Pilot proficiency is essential in the aviation industry to maintain a safety record for successfully operating airlines. The current COVID-19 scenario has made it practically difficult for pilots to achieve the federal requirements stated in Title 14 CFR 121.439 (a) and 14 CFR 61.57 (a) (1). It has placed a burden on pilots to maintain their currency and proficiency related to cost and lack of aircraft access due to furloughs and lay-offs. The purpose of the first part of this study was to assess the impact of COVID-19 on global air traffic based on the data collected from the International Civil Aviation Organization (ICAO) and the U.S. Bureau of Transportation Statistics (BTS). The second part of this research study aims to assess the pilot’s proficiency before and after COVID-19 was officially declared as pandemic to identify any threats to aviation safety based on the incident reports submitted in the Aviation Safety Reporting System. The findings show that the highest reduction in flight operations was observed during May 2020 with -70.67 % compared with May 2019 at the global level. Middle East region witnessed a -57.35 % reduction in flight operations. Based on the number of flights operated in the year 2020, the Asia-Pacific region was the most affected region with 4,913,303 fewer flights than in the year 2019. For the U.S. domestic air traffic, a severe impact occurred in May 2020 with a -70.88 % reduction and 532,834 fewer flights than in May 2019 which was similar to the global air traffic trend. The number of incident reports filed by pilots regarding proficiency-related to COVID-19 issues in the ASRS data analysis increased by 1000% during this pandemic period. The results and recommendations of this research study will provide valuable information to the aviation industry that can aid in developing mitigation measures to decrease future aviation accidents related to pilot proficiency.