Toward Runtime Assurance of Complex Systems with AI Components

Abstract

AI components (e.g., Deep Neural Networks) are increasingly used in safety-relevant aerospace applications. Rigorous Verification and Validation (V&V) is mandatory for such components, yet V&V techniques for DNNs are still in their infancy and can often only provide relatively weak guarantees. In this paper, we will present a runtime-monitoring architecture, which combines the advanced statistical analysis framework SYSAI (System Analysis using Statistical AI) with temporal and probabilistic runtime monitoring carried out by R2U2 (Realizable, Responsive, and Unobtrusive Unit). We will present initial results of our tool set and architecture on a case study, a DNN-based autonomous centerline tracking system (ACT).