Boeing 737MAX Through DC6 Fleet Grounding Decisions Revisited with Event Interval Probability Analysis

Abstract

Listen

Translate article

SUMMARY & CONCLUSIONSThe application of statistics and probability to event timing data is a powerful decision-making aid. Cause and effect data that are necessary to identify issues and make corrections are sparse or nonexistent at the time of the event. Cause and effect data can take days or months to acquire and analyze, but event interval timing data are simple system performance data and are available at the instant the event occurs. Event interval probability analysis is independent from cause and effect and organizational and other interfaces, e.g., human factors design and pilot error. It considers bottom-line total system performance.Statistics and probability analysis demand much data; however, for serious critical events failure data must be few. This conflict is resolved by a null hypothesis that the data are generated by a homogeneous Poisson process (HPP). The analysis uses the infinite quantity of perfect data inherent in this null hypothesis. Event data are compared with the null hypothesis and the null is rejected or not with Poisson and/or computer simulation probability values (p-values). The Poisson interval is not limited to time, and for this paper, the number of departures between accidents are used, except where noted.This paper reviews six fleet groundings on five aircraft types with 13 different grounding decisions. Data for the analysis and all analysis results are presented; however, the first opportunity to ground decision is the most important. The first opportunity to ground decision is retrospectively judged to be wrong if future events unfolded that demonstrate earlier grounding would have been appropriate.Decisions for five of the six fleet groundings revisited involved US designed and manufactured aircraft with grounding decisions made by the FAA or its predecessor organizations. On these five critical decisions, the grounding of the DC 10 due to a crash from engine pylon cracking on 5/25/79 was correct. The other four decisions were proven to be incorrect by future events. The FAA made only one correct grounding decision out of five.Event interval probability analysis failed to ground on first opportunity upon a DC 6 crash on 10/24/47, using data that existed contemporaneously with the crash. The other five first opportunity decisions (including the Concorde) had p-values less than 0.025. The method would reject the null hypothesis and ground on these five occasions. So the application of event interval probability analysis would lead to five out of six correct decisions, in the total absence of cause and effect data.There is always a possibility of a false positive with these statistical methods. The total false positive probability for the five correct decisions is 0.0467 for an average false positive per grounding decision of 0.0093. An average one percent chance of unnecessarily grounding a fleet is small relative to the risk of not grounding timely, as the current Boeing 737 MAX situation demonstrates. The first crash of the 737 MAX has a p-value of 0.022. Upon the second crash, the p-value is 0.00099. Grounding by p-values would have led to immediate grounding upon the first crash, and certainly after the second.When the null hypothesis is rejected, the alternative hypothesis that the fleet accident rate is above expectation by a statistically significant amount is accepted. Upon acceptance of the alternative hypothesis, unreliability probability distributions are obtained via computer simulation for the demonstrated low reliability aircraft fleet. From these probability distributions, the risk of continuing to fly the low reliability aircraft is found. For example, the risk of flying three days following the second 737 MAX crash, assuming all the fleet was flying, was 4.75% chance of a third event. After the Concord crash, British Airways flew an additional 21 days following the crash with a significantly low reliability aircraft and incurred an unrecognized 1.12% chance of a second event.The current FAA and predecessor decision record of 20% correct versus 83% correct using event interval probability analysis indicates that the FAA and aircraft manufactures can improve their decision-making by incorporating the method. Note this poor decision record covers 75 years; therefore, cannot be attributed to current organizations and individuals. The following specific steps are recommended:1-The FAA should conduct an event interval probability analysis of the aircraft events in this paper using best available data and publish results.2-The method should be conducted immediately upon future major events, such as crashes, by manufacturers or the FAA. Departure intervals and p-values should be made public.3-Air worthiness certification should specify the p-values at which a serious event will lead to automatic grounding of the fleet, in the absence of immediately available cause and effect evidence indicating to the contrary.4-To help avoid even the first accident, include p-values in the monitoring of accident precursor events that typically precede a more serious event, e.g. maintenance and operational issues. This may require automating the analysis due to volume.