Safety-critical Research Articles

Safe Reinforcement Learning Using Robust Control Barrier Functions

Reinforcement Learning (RL) has been shown to be effective in many scenarios. However, it typically requires the exploration of a sufficiently large number of state-action pairs, some of which may be unsafe. Consequently, its application to safety-critical systems remains a challenge. An increasingly common approach to address safety involves the addition of a safety layer that projects the RL actions onto a safe set of actions. In turn, a difficulty for such frameworks is how to effectively couple RL with the safety layer to improve the learning performance. In this paper, we frame safety as a differentiable robust-control-barrier-function layer in a model-based RL framework. Moreover, we also propose an approach to modularly learn the underlying reward-driven task, independent of safety constraints. We demonstrate that this approach both ensures safety and effectively guides exploration during training in a range of experiments, including zero-shot transfer when the reward is learned in a constraint-agnostic fashion.

Multiple aspects maintenance ontology-based intelligent maintenance optimization framework for safety-critical systems

Abstract Maintenance optimization is a process for improving the efficiency of maintenance strategies and activities, considering various aspects of the target system and components, such as the probabilities of system failures and the cost of repair and replacement of a failed component. The improvement of maintenance optimization algorithms generally requires information from various data sources. For example, it may require the system risk information derived from risk analysis tools or the residual lifetime of a component from fault prognosis tools. The requirements of data acquisition (DAQ) and aggregation pose new challenges for maintenance management systems (MMSs) that implement and use these maintenance optimization algorithms. This paper proposes a multiple aspects maintenance ontology-based framework to facilitate DAQ from MMSs, online monitoring systems, fault detection and discrimination tools, risk assessment tools, decision-making tools, and component identification tools, and accelerate the implementation and verification of contemporary maintenance optimization models and algorithms. The proposed framework consists of a multi-aspect maintenance ontology with critical information for maintenance optimization and application interfaces for collecting information from various data sources, such as fault prognosis tools, online monitoring tools, risk assessment tools, and decision-making algorithms. In addition, this paper proposes a heuristic method for integrating concepts and properties from other existing ontologies into the proposed framework when the existing ontology is not fully compatible with the ontology under construction. Finally, the paper verifies the proposed ontology framework using a feedwater system designed for nuclear power plants with valves and filters as the components under maintenance.

Emotion-performance relationship in safety-critical human-machine systems

Affect-adaptive systems (AAS) monitor the emotional user state, analyze it concerning the current context and apply appropriate adaptation strategies when necessary to support the user. In safety-critical work environments, human-machine systems aim at maintaining high performance and avoid failure. The detection of critical emotional user states requiring adaptation should take individual differences in the emotion-performance relationship into account. Hence, we suggested the Affective Response Categories (ARCs), a classification concept considering these differences. In the present contribution, we investigated if the proposed ARCs are part of a stable trait or states changing over time. Physiological measures, such as pupil diameter and facial expression; as well as performance data from two air space surveillance laboratory experiments differing in context and sample origin (N1 = 50; N2 = 17) were analyzed. Results of two one-sided tests (TOSTs) and intraclass correlation coefficients (ICC) based on data from both experiments provided evidence for stability of the categories over time, indicating that a continuous user state diagnosis based on a single classification of a user's ARC is possible. With these results, the present contribution provides the foundation for developing a diagnostic engine of users' emotional state.

Optimal Mission Abort Decisions for Multi-Component Systems Considering Multiple Abort Criteria

This paper studies the optimal mission abort decisions for safety-critical mission-based systems with multiple components. The considered system operates in a random shock environment and is required to accomplish a mission during a fixed mission period. If the failure risk of the system is very high, the main mission can be aborted to avoid higher failure cost. The main contribution of this study lies in the design and optimization of mission abort policies for multi-component systems with multiple abort criteria. Moreover, multi-level transitions are considered in this study to characterize the different shock-resistance abilities for components in different states. Mission abort decisions are determined based on the number of components in either defective or failed state. The problem is formulated in the framework of the finite Markov chain imbedding method. We use the Monte-Carlo simulation method to derive the mission reliability and system survivability. Numerical studies and sensitivity analysis are presented to validate the obtained result.

Optimal Proof-Testing Strategies for Safety-Related Systems Based on Continuous Time Markov Chain

Maintenance activities for safety-related systems are generally required to ensure that the systems are working as intended. Regarding the maintenance activities, proof-testing is known as scheduled inspections and maintenance activities for detecting dangerous undetected faults which cannot be detected by diagnostic testing systems installed in the safety-related systems. However, the proof-testing needs a lot of cost and provokes decreasing of the availability for the whole system because the whole system is needed to shut down for proofing that the whole system is working as intended. We discuss analytical methodologies for obtaining optimal proof-testing interval with harmful risk and proof-testing cost by describing the behavior of the safety-related system based on a continuous-time Markov chain. Further, an analytical optimal policy for obtaining economic proof-testing interval is proposed in this paper.

Design and Assurance of Safety-Critical Systems with Artificial Intelligence in FPGAs: The Safety ArtISt Method and a Case Study of an FPGA-Based Autonomous Vehicle Braking Control System

With the advancements in utilizing Artificial Intelligence (AI) in embedded safety-critical systems based on Field-Programmable Gate Arrays (FPGAs), assuring that these systems meet their safety requirements is of paramount importance for their revenue service. Based on this context, this paper has two main objectives. The first of them is to present the Safety ArtISt method, developed by the authors to guide the lifecycle of AI-based safety-critical systems, and emphasize its FPGA-oriented tasks and recommended practice towards safety assurance. The second one is to illustrate the application of Safety ArtISt with an FPGA-based braking control system for autonomous vehicles relying on explainable AI generated with High-Level Synthesis. The results indicate that Safety ArtISt played four main roles in the safety lifecycle of AI-based systems for FPGAs. Firstly, it provided guidance in identifying the safety-critical role of activities such as sensitivity analyses for numeric representation and FPGA dimensioning to achieve safety. Furthermore, it allowed building qualitative and quantitative safety arguments from analyses and physical experimentation with actual FPGAs. It also allowed the early detection of safety issues—thus reducing project costs—and, ultimately, it uncovered relevant challenges not discussed in detail when designing safety-critical, explainable AI for FPGAs.

Complexity for complexity-How advanced modeling may limit its applicability for decision-makers.

As today's engineering systems have become increasingly sophisticated, assessing the efficacy of their safety-critical systems has become much more challenging. The more classical methods of "failure" analysis by decomposition into components related by logic trees, such as fault and event trees, root cause analysis, and failure mode and effects analysis lead to models that do not necessarily behave like the real systems they are meant to represent. These models need to display similar emergent and unpredictable behaviors to sociotechnical systems in the real world. The question then arises as to whether a return to a simpler whole system model is necessary to understand better the behavior of real systems and to build confidence in the results. This question is more prescient when one considers that the causal chain in many serious accidents is not as deep-rooted as is sometimes claimed. If these more obvious causes are not taken away, why would the more intricate scenarios that emanate from more sophisticated models be acted upon. The paper highlights the advantages of modeling and analyzing these "normal" deviations from ideality, so called weak signals, versus just system failures and near misses as well as catastrophes. In this paper we explore this question.

Reliability and Performance Measurement of Safety-Critical Systems Based on Petri Nets: A Case Study of Nuclear Power Plant

Safety-critical systems (SCSs) mitigate the risk of catastrophic loss of assets and hence do have high dependability targets. Performance and reliability are the critical dependability attributes, particularly in control and safety systems, and hence essential to measure to ensure the dependability. Traditional methods either are not capable to capture the system dynamics or encounter state explosion problem. Also, the methods are not able to measure all critical performance attributes. This article proposes a novel approach to measure the performance and reliability of SCSs. Such systems contain multiple interconnecting processing nodes, the functional requirements of which are modeled using Petri net (PN). A set of ordinary differential equations (ODEs) is derived from the PN model that represents the state of the system. The ODE solution can be used to measure the critical performance attributes, such as latency time and throughput of the system. The proposed method can avoid the state explosion problem and also introduces new metrics of performance, along with their measurement: deadlock, liveness, stability, boundedness, and steady state. The proposed technique is applied to a case study of nuclear power plant. We obtained 99.887% and 99.939% accuracy of performance and reliability measurement, respectively, which proves the effectiveness of our approach.

Boosting Adversarial Training in Safety-Critical Systems Through Boundary Data Selection

Boosting Adversarial Training in Safety-Critical Systems Through Boundary Data Selection

Teaching Model Based System Engineering – A Banking Benchmark Problem

AbstractMBSE has been widely used for industry processes and safety critical avionic and space systems. The use of MBSE process with STPA for safety analysis, mutation testing for verification, model checking and simulations for formal validation, has shown a better alternative to normal development methods. The growth of online financial transactions due to ease of internal payment methods, has resulted in a number of payment applications. System Engineering is not widely popular in academia and undergraduate levels. Benchmark problems are required for the students to get their hands “dirty” and for the instructors to provide hands on examples, matched to theory in textbooks. This paper looks into performing STPA and Model based analysis of such an application, with the goal to teach MBSE methods through industry example.

Health assessment for low-voltage electromagnetic coils based on fusion of creep degradation mechanism and transformed MD

Low-voltage electromagnetic coils play ever-increasing roles in safety-critical systems, like electromechanical actuators in more electric aircraft and tractors in zero emission vehicles. Identifying insulation degradation prior to the formation of turn-to-turn short can effectively avoid accidental shutdown and enable predictive maintenance for these systems. High-frequency electrical response analysis is currently a promising technique for electromagnetic coil insulation health assessment. However, the correlation between high-frequency electrical response parameters (HFERP) and actual insulation degradation status (AIDS) is still not clear, which brings challenges in insulation health assessment and remaining useful life prediction. Therefore, in this paper, a fusion method based on creep degradation mechanism and transformed mahalanobis distance (MD) for insulation health assessment is proposed. First, correlation analysis between the coil HFERP and the AIDS is performed from coil insulation coating creep degradation point of view. Second, a coil on-line monitoring parameter (OLMP) for monitoring insulation degradation status on-site is constructed based on transformed MD. A correlation model between OLMP and AIDS is consequently built to assist in determining OLMP failure threshold for coil insulation health assessment. Coil accelerated testing shows that the fusion method can be used to assess insulation status prior to the formation of harmful shorts, enabling condition-based maintenance for components or systems that incorporate electromagnetic coils.

Integrated Attack Tree in Residual Risk Management Framework

Safety-critical cyber-physical systems (CPSs), such as high-tech cars having cyber capabilities, are highly interconnected. Automotive manufacturers are concerned about cyber attacks on vehicles that can lead to catastrophic consequences. There is a need for a new risk management approach to address and investigate cybersecurity risks. Risk management in the automotive domain is challenging due to technological improvements and advances every year. The current standard for automotive security is ISO/SAE 21434, which discusses a framework that includes threats, associated risks, and risk treatment options such as risk reduction by applying appropriate defences. This paper presents a residual cybersecurity risk management framework aligned with the framework presented in ISO/SAE 21434. A methodology is proposed to develop an integrated attack tree that considers multiple sub-systems within the CPS. Integrating attack trees in this way will help the analyst to take a broad perspective of system security. Our previous approach utilises a flow graph to calculate the residual risk to a system before and after applying defences. This paper is an extension of our initial work. It defines the steps for applying the proposed framework and using adaptive cruise control (ACC) and adaptive light control (ALC) to illustrate the applicability of our work. This work is evaluated by comparing it with the requirements of the risk management framework discussed in the literature. Currently, our methodology satisfies more than 75% of their requirements.

Event Sequence Based Fault Tree Analysis to Evaluate Minimal Combination of Event Sequences Leading to the Reactor Core Damage

Probabilistic safety assessment has been widely used to evaluate nuclear power plant safety systems. It couples fault tree (FT) analysis and event tree (ET) analysis. FT analysis is to develop failure scenarios, to quantify the failure probability and to identify critical components. ET analysis is to develop event sequences (ESs) leading to reactor core damage and quantify core damage frequency. Currently, there is no approach in probabilistic safety assessment that can be used to identify minimal combinations of safety system failures leading to reactor core damage. This study proposes an event sequence based fault tree analysis, which integrates FT model with ET model. The ET model is to identify the ESs leading to the reactor core damage. Meanwhile, the FT model is to identify minimal combinations of those ESs found in the ET model. The motivation of this study is on how to identify critical safety systems in nuclear power plants to mitigate disturbances caused by a group of postulated initiating events to avoid core damage. To confirm the feasibility and the applicability of the proposed approach, the performance of the AP1000 passive core cooling system is evaluated. It is found that if in-containment refueling water storage tank, automatic depressurization system - full, accumulator, core make-up tank, and passive residual heat removal work properly, the reactor core will intact. These results confirmed that the proposed approach can be applicable to identify minimal combinations of safety systems to keep the reactor core intact.

Systematic review on contract-based safety assurance and guidance for future research

The safety requirements are often described via specifications called contracts. To verify that the system fulfils certain safety requirements, for instance, in the assume-guarantee contract specification, the key safety indicators are organized, so that if certain assumptions hold then the respective behaviour is guaranteed. Safety contracts provide a means of exposing potential incompatibilities early in the development process, selecting components to reuse, certifying systems, and identifying uncertainty sources during the operational phase. There exist several studies on contract-based safety assurance, however, there is not any systematic study in this field. For this, a first Systematic Literature Review (SLR) is carried out to obtain an overview of the various contract-based safety assurance concepts, problems, proposed solutions, and their usefulness. In our study, the identification and selection of the primary studies were based on a well-planned search strategy. The search process identified a total of 2881 studies published between 1969 and 2021, out of which 66 studies were selected through a multi-stage process according to our predefined SLR protocol. This SLR aims to highlight the state-of-the-art of contract-based safety assurance and identify potential gaps for future research. Based on research topics in selected studies, we identified the following main categories: contract type, analysis techniques for system safety, compliance with standards, development stage, domain, level of automation, type of study and evaluation, and tool support. The findings of the systematic review not only highlight that the contracts are even more important for advanced safety-critical systems but also strategies to exploit their full potential should be considered in future studies. The suggestions revealed for future research include the usage of contracts for adapting new behaviour, defining system boundaries, interacting with other systems, managing risk during operation, dynamic/runtime safety assurance, and integration of safety with security.

Impact of ISP Tuning on Object Detection.

In advanced driver assistance systems (ADAS) or autonomous vehicle research, acquiring semantic information about the surrounding environment generally relies heavily on camera-based object detection. Image signal processors (ISPs) in cameras are generally tuned for human perception. In most cases, ISP parameters are selected subjectively and the resulting image differs depending on the individual who tuned it. While the installation of cameras on cars started as a means of providing a view of the vehicle's environment to the driver, cameras are increasingly becoming part of safety-critical object detection systems for ADAS. Deep learning-based object detection has become prominent, but the effect of varying the ISP parameters has an unknown performance impact. In this study, we analyze the performance of 14 popular object detection models in the context of changes in the ISP parameters. We consider eight ISP blocks: demosaicing, gamma, denoising, edge enhancement, local tone mapping, saturation, contrast, and hue angle. We investigate two raw datasets, PASCALRAW and a custom raw dataset collected from an advanced driver assistance system (ADAS) perspective. We found that varying from a default ISP degrades the object detection performance and that the models differ in sensitivity to varying ISP parameters. Finally, we propose a novel methodology that increases object detection model robustness via ISP variation data augmentation.

Sensitivity analysis by differential importance measure for unsupervised fault diagnostics

Fault diagnostic approaches based on supervised classifiers are difficult to apply to safety-critical or new design systems, because they require the availability of labelled data collected when the systems operate abnormally, which is a rare situation. To address this challenge, we develop a novel unsupervised method for fault diagnostics based on a fault detection module and on sensitivity analysis . Specifically, the Differential Importance Measure (DIM) is originally used to quantify how much a signal is, or a set of signals are, responsible for the variation of the system health state. The proposed method is tested on simulated data from a wind turbine and on real data from a gas turbine. The advantages of the proposed fault diagnostic method are: (1) it can be developed using only normal condition data (2) it allows identifying the component responsible for the abnormality by quantifying the contribution of groups of signals to the variation of the system health state; (3) it is capable of distinguishing the abnormalities caused by changes in external conditions from those caused by components malfunctions; (4) it can be used in combination with any fault detection technique.

Supporting model-based safety analysis for safety-critical IoT systems

Dependability is regarded as the ability of the system to provide services that can be trusted within a specific period. As the complexity and heterogeneity of Internet of Things (IoT) systems rise, so does the possibility of errors and failure. Early safety analysis not only reduces the cost of late failure but also makes it easier to trace and determine the source of the failure beforehand in case something goes wrong. In this paper, we present an early safety analysis approach based on Failure-Logic Analysis (FLA) and Fault-Tree Analysis (FTA) for safety-critical IoT systems. The safety analysis infrastructure, supported by the CHESSIoT tool, takes into account the system-level physical architecture model annotated with the component’s failure logic properties to perform different kinds of automated failure analyses. In addition to its ability to generate the system Fault-Trees (FTs), the new FTA analysis approach automatically performs qualitative and quantitative analyses which include the elimination of redundant events, unnecessary failure paths, as well as automatic probabilistic calculation of the undesired events. To assess the effectiveness of the approach, a comparative study between our propose approach with 19 existing approaches in both academia and industry was conducted showcasing its contribution to the state of the art. Finally, a Patient Monitoring System (PMS) use case has been developed to demonstrate the capabilities of the supporting CHESSIoT tool, and the results are thoroughly presented.

A systematic co-engineering of safety and security analysis in requirements engineering process

Co-engineering safety and security is increasingly important in safety-critical systems as more diverse interacting functions are implemented in software. Many studies have tried to perform safety and security analyses in unified or in parallel. While the unified approach requires more complex analysis with new delicate methods, the parallel needs further improvement on additional integration activity for harmonizing safety and security analyses results. This paper tries to improve the harmonization activity seamlessly and systematically in typical requirements engineering process for safety-critical systems. It encompasses both requirements elicitation and analysis as well as safety and security analyses, regardless of which analysis techniques are used. The paper suggests performing an appropriate safety analysis first to derive safety requirements as summary information. It then performs goal-tree analysis to refine the high-level safety requirements into lower-level ones, from which any security analysis can work on to derive security requirements. Another goal-tree analysis then tries to refine the high-level security requirements into specific functional ones too, and it ends the analysis activity in a cycle of requirements engineering process. The sequence of safety analysis, goal-tree refinement, security analysis and another goal-tree refinement is seamlessly iterated in the process of requirements engineering, where any conflict of requirements will have an opportunity to be resolved. Our case study of a simplified UAV example uses STPA and STRIDE techniques for safety and security analysis respectively, and shows that the proposed approach is fully applicable up to industrial cases.

Dual regularized policy updating and shiftpoint detection for automated deployment of reinforcement learning controllers on industrial mechatronic systems

We propose an algorithmic pipeline enabling deep reinforcement learning controllers to detect when a significant change in system characteristics has occurred and update the control policy accordingly to reattain performance. Reinforcement learning algorithms can learn a policy directly from input–output data and thus optimize for system-specific properties. Yet they face difficulties to adapt, after deployment, to varying operating conditions. Real-world industrial mechatronic systems however demand further levels of performance through adaptation while remaining safe. So far, methods that detect changes in environments exist but have never been studied and applied as a means to update control policies for time-varying systems. We benchmark several methods that detect significant changes in these systems, i.e. shiftpoint detection methods, and present a novel algorithm with a dual regularization architecture. This architecture exploits the prior policy while allowing sufficient flexibility to update for the safety-critical and time-varying system. We validate the method’s performance through benchhmarking and study the effect of its different components and targeted ablation studies on mechatronic systems, both in simulations and experimentally. Results show that our algorithmic pipeline allows for rapid shiftpoint detection, followed by a policy update that reaches expert performance after convergence.

Unified Human–Robot–Environment Interaction Control in Contact-Rich Collaborative Manipulation Tasks via Model-Based Reinforcement Learning

Many safety-critical or performance-demanding systems are human-in-the-loop, i.e., the robot interacts with human being and environment, in which human-in-the-loop control becomes a key research topic. In this paper, a unified optimal interaction control in joint space is presented for multi-point human-robot-environment interaction (HREI) problems, which are very common in human-robot collaborative manipulation tasks. Specifically, model-based reinforcement learning method is leveraged for obtaining optimal interaction control. For multi-point human-robot-environment interaction, the interaction forces exerted on each link are isolated and estimated via the backward generalized momentum observer method. In human-robot-environment interaction problems, the environmental as well as the human arm's dynamics parameters are usually unknown, stochastic, and time-varying. To obviate the dependence on these parameters, the GMM/GMR method is employed to learn the unknown external dynamics. It is noteworthy the interaction forces are considered as system states, the time derivatives of which are computed based on the GMM/GMR learning results via chain-rule. Then the iterative linear quadratic Gaussian with learned external dynamics (ILQG-LED) method is utilized to realize optimal multi-point human-robot-environment interaction control. The validity of the proposed method is verified through experimental studies.