Analysis of an Intervention for Small Unmanned Aerial System (SUAS) Accidents: A Case Study Involving Simpson's Paradox

Abstract

ABSTRACT We investigate a process intervention designed to lower small unmanned aerial system (SUAS) mishap rates. The intervention implemented review boards, required various certifications, and produced apparent decreases in flight failures and flight damage, albeit at increased costs. However, we demonstrate how the results actually represent an undetected example of Simpson's Paradox in which aggregated results yield differing conclusions than disaggregated results. We use the Wilson score method to produce confidence intervals on the difference in mishap rates both before and after the intervention. Using this method, we show that no mishap rate reduction was truly attributable to the process intervention. Our case study reinforces the importance of using deeper statistical knowledge when making policy decisions based in part on the statistical analysis of empirical data.