Assuring Autonomous UAS Traffic Management Systems Using Explainable, Fuzzy Logic, Black Box Monitoring

Abstract

Researchers have shown via simulation and early flight tests the feasibility and safety benefit of adding autonomy to the concept of unmanned aircraft system (UAS) Traffic Management (UTM), which is the Federal Aviation Administration’s (FAA) vision for air traffic management below 400 feet. Such simulations are a complex interaction between UAS, UAS operators, and the UTM system. Autonomy in this system is achieved through the algorithms used for strategic de-confliction (UAS launch scheduling) and flight planning (creates waypoints from delays introduced by scheduling), which have been shown to improve safety in congested UAS airspace. However, autonomous algorithms have been known to make poor decisions without notice, and thus need to be constantly monitored to prevent these decisions from negatively affecting airspace performance and safety. This is the impetus for developing and implementing a fuzzy assurance black box monitor. Using a rule set generated from parameters that have been shown to improve safety in congested airspace, this monitor only considers the inputs and outputs of the autonomous UTM system to estimate the risk of the autonomous algorithms making poor decisions. Fuzzy rules that fire during the operation of the fuzzy assurance monitor help identify offending algorithms and their poor decisions, and thus provide a level of explainable artificial intelligence (AI) capability. The goal is to use fuzzy inference rules to evaluate the performance of strategic de-confliction algorithms in the UTM simulation. We investigate several airspace operational use cases (i.e., normal and rogue behavior in congested airspace) and several different autonomous UTM configurations (No Strategic de-confliction and Strategic de-confliction). The simulation data is analyzed with the help of the fuzzy inference system rules to help identify offending algorithms and poor decisions that lead to unsafe airspace. Our results show that the fuzzy assurance monitor is able to use the inputs and outputs of the autonomous UTM system to assign safety risks appropriately across use cases and autonomous UTM configuration. The fuzzy assurance monitor can also provide insight on the performance trade-offs of black-box algorithms.