Learning-enabled Components Research Articles

Out-of-distribution Detection in Dependent Data for Cyber-physical Systems with Conformal Guarantees

Uncertainty in the predictions of learning-enabled components hinders their deployment in safety-critical cyber-physical systems (CPS). A shift from the training distribution of a learning-enabled component (LEC) is one source of uncertainty in the LEC’s predictions. Detection of this shift or out-of-distribution (OOD) detection on individual datapoints has therefore gained attention recently. But in many applications, inputs to CPS form a temporal sequence. Existing techniques for OOD detection in time-series data for CPS either do not exploit temporal relationships in the sequence or do not provide any guarantees on detection. We propose using deviation from the in-distribution temporal equivariance as the non-conformity measure in conformal anomaly detection framework for OOD detection in time-series data for CPS. Computing independent predictions from multiple conformal detectors based on the proposed measure and combining these predictions by Fisher’s method leads to the proposed detector CODiT with bounded false alarms. CODiT performs OOD detection on fixed-length windows of consecutive time-series datapoints by using Fisher value of the input window. We further propose performing OOD detection on real-time time-series traces of variable lengths with bounded false alarms. This can be done by using CODiT to compute Fisher values of the sliding windows in the input trace and combining these values by a merging function. Merging functions such as Harmonic Mean, Arithmetic Mean, Geometric Mean, Bonferroni Method, and so on, can be used to combine Fisher values of the sliding windows in the input trace, and the combined value can be used for OOD detection on the trace with bounded false alarm rate guarantees. We illustrate the efficacy of CODiT by achieving state-of-the-art results in two case studies for OOD detection on fixed-length windows. The first one is on an autonomous driving system with perception (or vision) LEC. The second case study is on a medical CPS for walking pattern or GAIT analysis where physiological (non-vision) data is collected with force-sensitive resistors attached to the subject’s body. For OOD detection on variable length traces, we consider the same case studies on the autonomous driving system and medical CPS for GAIT analysis. We report our results with four merging functions on the Fisher values computed by CODiT on the sliding windows of the input trace. We also compare the false alarm rate guarantees by these four merging functions in the autonomous driving system case study. Code, data, and trained models are available at https://github.com/kaustubhsridhar/time-series-OOD .

Exploring the role of simulator fidelity in the safety validation of learning‐enabled autonomous systems

AbstractThis article presents key insights from the New Faculty Highlights talk given at AAAI 2023, focusing on the crucial role of fidelity simulators in the safety evaluation of learning‐enabled components (LECs) within safety‐critical systems. With the rising integration of LECs in safety‐critical systems, the imperative for rigorous safety and reliability verification has intensified. Safety assurance goes beyond mere compliance, forming a foundational element in the deployment of LECs to reduce risks and ensure robust operation. In this evolving field, simulations have become an indispensable tool, and fidelity's role as a critical parameter is increasingly recognized. By employing multifidelity simulations that balance the needs for accuracy and computational efficiency, new paths toward comprehensive safety validation are emerging. This article delves into our recent research, emphasizing the role of simulation fidelity in the validation of LECs in safety‐critical systems.

MoDALAS: addressing assurance for learning-enabled autonomous systems in the face of uncertainty.

Increasingly, safety-critical systems include artificial intelligence and machine learning components (i.e., learning-enabled components (LECs)). However, when behavior is learned in a training environment that fails to fully capture real-world phenomena, the response of an LEC to untrained phenomena is uncertain and therefore cannot be assured as safe. Automated methods are needed for self-assessment and adaptation to decide when learned behavior can be trusted. This work introduces a model-driven approach to manage self-adaptation of a learning-enabled system (LES) to account for run-time contexts for which the learned behavior of LECs cannot be trusted. The resulting framework enables an LES to monitor and evaluate goal models at run time to determine whether or not LECs can be expected to meet functional objectives and enables system adaptation accordingly. Using this framework enables stakeholders to have more confidence that LECs are used only in contexts comparable to those validated at design time.

Variational Autoencoder for Classification and Regression for Out-of-Distribution Detection in Learning-Enabled Cyber-Physical Systems

ABSTRACT Learning-Enabled Components (LECs), such as neural networks, are broadly employed in Cyber-Physical Systems (CPSs) to tackle a wide variety of complex tasks in high-uncertainty environments. However, the training dataset is inevitably incomplete, and Out-Of-Distribution (OOD) data not encountered during the LEC training may lead to erroneous predictions, jeopardizing the safety of the system. In this paper, we first analyze the causes of OOD data and define various types of OOD data in learning-enabled CPSs. We propose an approach to effectively detect OOD data for both classification and regression problems. The proposed approach incorporates the variational autoencoder for classification and regression model to the Inductive Conformal Anomaly Detection (ICAD) framework, enabling the detection algorithm to take into consideration not only the LEC inputs but also the LEC outputs. We evaluate the approach using extensive experiments for both classification and regression tasks, and the experimental results validate the effectiveness of the proposed method for detecting different types of OOD data. Furthermore, the execution time of detection is relatively short; therefore, the proposed approach can be used for real-time detection.

Fault-Adaptive Autonomy in Systems with Learning-Enabled Components.

Autonomous Cyber-Physical Systems (CPS) must be robust against potential failure modes, including physical degradations and software issues, and are required to self-manage contingency actions for these failures. Physical degradations often have a significant impact on the vehicle dynamics causing irregular behavior that can jeopardize system safety and mission objectives. The paper presents a novel Behavior Tree-based autonomy architecture that includes a Fault Detection and Isolation Learning-Enabled Component (FDI LEC) with an Assurance Monitor (AM) designed based on Inductive Conformal Prediction (ICP) techniques. The architecture implements real-time contingency-management functions using fault detection, isolation and reconfiguration subsystems. To improve scalability and reduce the false-positive rate of the FDI LEC, the decision-making logic provides adjustable thresholds for the desired fault coverage and acceptable risk. The paper presents the system architecture with the integrated FDI LEC, as well as the data collection and training approach for the LEC and the AM. Lastly, we demonstrate the effectiveness of the proposed architecture using a simulated autonomous underwater vehicle (AUV) based on the BlueROV2 platform.

Runtime Safety Monitoring of Neural-Network-Enabled Dynamical Systems.

Complex dynamical systems rely on the correct deployment and operation of numerous components, with state-of-the-art methods relying on learning-enabled components in various stages of modeling, sensing, and control at both offline and online levels. This article addresses the runtime safety monitoring problem of dynamical systems embedded with neural-network components. A runtime safety state estimator in the form of an interval observer is developed to construct the lower bound and upper bound of system state trajectories in runtime. The developed runtime safety state estimator consists of two auxiliary neural networks derived from the neural network embedded in dynamical systems, and observer gains to ensure the positivity, namely, the ability of the estimator to bound the system state in runtime, and the convergence of the corresponding error dynamics. The design procedure is formulated in terms of a family of linear programming feasibility problems. The developed method is illustrated by a numerical example and is validated with evaluations on an adaptive cruise control system.

Verification Approaches for Learning-Enabled Autonomous Cyber–Physical Systems

<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Editor’s notes:</i> Neural network control systems are often at the heart of autonomous systems. The authors classify existing verification methods for these systems and advocate the necessity of integrating verification techniques in the training process to enhance robustness. — <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Selma Saidi, TU Dortmund</i>

Dynamic-weighted simplex strategy for learning enabled cyber physical systems

Abstract Cyber Physical Systems (CPS) have increasingly started using Learning Enabled Components (LECs) for performing perception-based control tasks. The simple design approach, and their capability to continuously learn has led to their widespread use in different autonomous applications. Despite their simplicity and impressive capabilities, these components are difficult to assure, which makes their use challenging. The problem of assuring CPS with untrusted controllers has been achieved using the Simplex Architecture. This architecture integrates the system to be assured with a safe controller and provides a decision logic to switch between the decisions of these controllers. However, the key challenges in using the Simplex Architecture are: (1) designing an effective decision logic, and (2) sudden transitions between controller decisions lead to inconsistent system performance. To address these research challenges, we make three key contributions: (1) dynamic-weighted simplex strategy – we introduce “weighted simplex strategy” as the weighted ensemble extension of the classical Simplex Architecture. We then provide a reinforcement learning based mechanism to find dynamic ensemble weights, (2) middleware framework – we design a framework that allows the use of the dynamic-weighted simplex strategy, and provides a resource manager to monitor the computational resources, and (3) hardware testbed – we design a remote-controlled car testbed called DeepNNCar to test and demonstrate the aforementioned key concepts. Using the hardware, we show that the dynamic-weighted simplex strategy has 60% fewer out-of-track occurrences (soft constraint violations), while demonstrating higher optimized speed (performance) of 0.4 m/s during indoor driving than the original LEC driven system.