Design Of Safety-critical Systems Research Articles

Design and Assurance of Safety-Critical Systems with Artificial Intelligence in FPGAs: The Safety ArtISt Method and a Case Study of an FPGA-Based Autonomous Vehicle Braking Control System

With the advancements in utilizing Artificial Intelligence (AI) in embedded safety-critical systems based on Field-Programmable Gate Arrays (FPGAs), assuring that these systems meet their safety requirements is of paramount importance for their revenue service. Based on this context, this paper has two main objectives. The first of them is to present the Safety ArtISt method, developed by the authors to guide the lifecycle of AI-based safety-critical systems, and emphasize its FPGA-oriented tasks and recommended practice towards safety assurance. The second one is to illustrate the application of Safety ArtISt with an FPGA-based braking control system for autonomous vehicles relying on explainable AI generated with High-Level Synthesis. The results indicate that Safety ArtISt played four main roles in the safety lifecycle of AI-based systems for FPGAs. Firstly, it provided guidance in identifying the safety-critical role of activities such as sensitivity analyses for numeric representation and FPGA dimensioning to achieve safety. Furthermore, it allowed building qualitative and quantitative safety arguments from analyses and physical experimentation with actual FPGAs. It also allowed the early detection of safety issues—thus reducing project costs—and, ultimately, it uncovered relevant challenges not discussed in detail when designing safety-critical, explainable AI for FPGAs.

Unified Graphical Co-modelling, Analysis and Verification of Cyber-physical Systems by Combining AADL and Simulink/Stateflow

The design of safety-critical cyber-physical systems (CPSs) involve several dimensions, including physics, hardware rchitecture and software functionality. It is desirable to design CPSs by taking these issues into account uniformly and yet, few existing design workflows support this aim. For instance, AADL is an architecturecentric modelling formalism for CPSs, which focuses on modelling architecture and prototyping real-time hardware platforms, but it delegates physical and software behavioral models to so-called annexes. By contrast, Simulink/Stateflow (S/S) focuses on modelling interacting physical and software behaviors, but does not render the non-functional characteristics of their hardware platforms. To address this issue, in [1], we proposed the combination of AADL and S/S, called AADL S/S, to comodel CPSs and presented a method to uniformly analyse and verify them. AADL S/S provides a unified graphical co-modelling environment for CPS design and supports simulation through C code generation. Also, [1] presented a formal semantics of AADL S/S by translation to Hybrid Communicating Sequential Processes (HCSP), yielding a deductive verification framework of the combined models using Hybrid Hoare Logic (HHL). Additionally, [1] proved the correctness of the translation of AADL S/S to HCSP.

Multi-objective ω-Regular Reinforcement Learning

The expanding role of reinforcement learning (RL) in safety-critical system design has promoted ω-automata as a way to express learning requirements—often non-Markovian—with greater ease of expression and interpretation than scalar reward signals. However, real-world sequential decision making situations often involve multiple, potentially conflicting, objectives. Two dominant approaches to express relative preferences over multiple objectives are: (1) weighted preference , where the decision maker provides scalar weights for various objectives, and (2) lexicographic preference , where the decision maker provides an order over the objectives such that any amount of satisfaction of a higher-ordered objective is preferable to any amount of a lower-ordered one. In this article, we study and develop RL algorithms to compute optimal strategies in Markov decision processes against multiple ω-regular objectives under weighted and lexicographic preferences. We provide a translation from multiple ω-regular objectives to a scalar reward signal that is both faithful (maximising reward means maximising probability of achieving the objectives under the corresponding preference) and effective (RL quickly converges to optimal strategies). We have implemented the translations in a formal reinforcement learning tool, Mungojerrie , and we present an experimental evaluation of our technique on benchmark learning problems.

COMPUTER-AIDED DESIGN OF FAULT-TOLERANT HARDWARE ARCHITECTURES FOR AUTONOMOUS DRIVING SYSTEMS

AbstractFault-tolerant hardware architectures for autonomous vehicles can be implemented through redundancy, diversity, separation, self-diagnosis, and reconfiguration. These approaches can be coupled with majority redundancy through M-out-of-N independent system architectures. The development of fault- tolerant systems is of central importance in the launch of autonomous driving systems from level 4. The increasing complexity of electrical and electronic systems is challenging for the design of safety-critical systems. This work aims to develop a method to manage this complexity in product development and to use it to compare different types of architectures. The basis is a system consisting of sensors and microcontrollers. The reliability of all possible MooN configurations of the system is calculated automatically by numerically solving the master equation of the corresponding Markov chain. Subsequently, a software-based fault tree analysis enables more detailed modeling of the component structure. The results show that four-line architectures can provide suitable results and that the development effort for 2-ECU systems is higher than for 1-ECU systems with respect to the ISO 26262 target values.

Risk-informed collision avoidance system design for maritime autonomous surface ships

The maritime industry is paving the way towards developing Maritime Autonomous Surface Ships (MASSs) through the adoption of key enabling technologies for safety-critical operations, which are associated with new challenges, especially at their early design phase. This study aims to develop a methodology to conduct the risk-informed design for the Collision Avoidance System (CAS) of MASSs. Pertinent regulatory instruments are reviewed to identify functional and system requirements and develop a baseline CAS configuration at the component level. Quantitative Fault Tree Analysis is performed to derive risk metrics, such as probability of failure, Importance measures, and Minimal Cut Sets, whereas criticality analysis is conducted to recommend risk-reducing measures. A Short Sea Shipping case study is investigated considering four operating modes based on various weather and illumination conditions. Results demonstrate that the developed Fault Tree diagram provides a robust representation of the CAS failure. The most critical components are found to be related to the Intention Communication and Situation Awareness Systems, the redundancy of which leads to 91% reduction of the CAS probability of failure. This study contributes towards the risk-informed design of safety-critical systems required for the development of MASSs.

Individuals Utilization Behavior While Using Critical Healthcare Systems (Medical Ventilator)

Ergonomics and human factors play an important role in interaction design. Poor usabil-ity design of critical safety systems can easily lead to human error. While inspecting the critical systems, there are vital factors that have to be considered, such as demographic, organizational, interface design, and task factors. This study has inspected the influence of different demographic factors (age, gender, and education) on the medical ventilator system's usability. The primary purpose is to seek the difference in task completion time, reflection time, and the number of touches during the interaction, considering different demographic factors. As well as to seek the difference in perceived task workload in-dex. To test the study hypothesis, the Hamilton-c6 ventilator prototype was developed. The experiments were conducted on 54 participants, including physicians and nurses, in a controlled environment from different public-private sector hospitals in Pakistan. De-scriptive statistics and ANOVA were used to assess the study objectives. The results suggest significant differences between male vs. female and physician's vs. nurses groups while interacting with safety-critical systems. However, the difference between young and older adults was not significant. These findings have substantial implications for the effective user-centric design of safety-critical systems.

Automating Safety and Security Co-design through Semantically Rich Architecture Patterns

During the design of safety-critical systems, safety and security engineers make use of architecture patterns, such as Watchdog and Firewall, to address identified failures and threats. Often, however, the deployment of safety architecture patterns has consequences on security; e.g., the deployment of a safety architecture pattern may lead to new threats. The other way around may also be possible; i.e., the deployment of a security architecture pattern may lead to new failures. Safety and security co-design is, therefore, required to understand such consequences and tradeoffs in order to reach appropriate system designs. Currently, architecture pattern descriptions, including their consequences, are described using natural language. Therefore, their deployment in system design is carried out manually by experts and thus is time-consuming and prone to human error, especially given the high system complexity. We propose the use of semantically rich architecture patterns to enable automated support for safety and security co-design by using Knowledge Representation and Reasoning (KRR) methods. Based on our domain-specific language, we specify reasoning principles as logic specifications written as answer-set programs. KRR engines enable the automation of safety and security co-engineering activities, including the automated recommendation of which architecture patterns can address failures or threats, and consequences of deploying such patterns. We demonstrate our approach on an example taken from the ISO 21434 standard.

Bridging the Pragmatic Gaps for Mixed-Criticality Systems in the Automotive Industry

An increasingly important trend in the design of safety-critical systems is the integration of components with different levels of criticality onto a common hardware platform. Mixed-criticality systems (MCSs) have been well researched in academia, but can be difficult to implement in industrial scenarios as the theoretical models underpinning the research do not sufficiently consider industrial safety practice and safety standards. In this article, we make the first attempt toward the implementation of the MCS theoretical model in industrial settings. To this end, we identify the pragmatic gaps between theory and practice, and then propose a generic industrial MCS architecture, termed <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">P-MCS</i> ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Practical-MCS</i> ). <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">P-MCS</i> is built upon the conventional theoretical MCS model with additional considerations of industrial safety requirements: 1) runtime safety analysis, determining preserved applications in each system mode and 2) correct partitioning and isolation of different critical elements. We introduce three implementing methods for <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">P-MCS</i> . Corresponding to the new system architecture, we present a theoretical model and schedulability analysis (with consideration of shared resources) to ensure system predictability. Finally, we evaluate and demonstrate <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">P-MCS</i> in terms of system schedulability, overheads, throughput, and predictability, along with a real-world case study. As shown in the evaluation, the considerations of industrial requirements lead to extra overheads and performance reduction in <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">P-MCS</i> . Such weaknesses can be considerably mitigated by hardware assistance and acceleration.

Unified graphical co-modeling, analysis and verification of cyber-physical systems by combining AADL and Simulink/Stateflow

The efficient design of safety-critical cyber-physical systems (CPS) requires the co-modeling and -verification of its physics, architecture and functionalities. Existing co-modeling formalisms do not account for these three design aspects into account uniformly. AADL is a precise formalism for modeling architecture and prototyping real-time hardware platforms, but it delegates co-modeling physical and software behaviors to so-called annexes. By contrast, Simulink/Stateflow (S/S) is strong for modeling interacting physical and software behaviors, but weak for modeling architecture and hardware platforms. To address this issue, this paper considers the combination of AADL and S/S to co-model CPSs and presents a method to uniformly analyze and verify this combination. AADL⊕S/S provides a unified graphical co-modeling environment for CPS design and supports simulation through C code generation. We present a formal semantics of AADL⊕S/S by translating it to Hybrid Communicating Sequential Processes (HCSP), which yields a deductive verification framework for AADL⊕S/S models based on Hybrid Hoare Logic (HHL). We also prove the correctness of the translation of AADL⊕S/S to HCSP. The effectiveness of our approach is illustrated by the realistically-scaled case study of an automatic cruise control system.

Incremental design-space model checking via reusable reachable state approximations

The design of safety-critical systems often requires design space exploration: comparing several system models that differ in terms of design choices, capabilities, and implementations. Model checking can compare different models in such a set, however, it is continuously challenged by the state space explosion problem. Therefore, learning and reusing information from solving related models becomes very important for future checking efforts. For example, reusing variable ordering in BDD-based model checking leads to substantial performance improvement. In this paper, we present a SAT-based algorithm for checking a set of models. Our algorithm, FuseIC3, extends IC3 to minimize time spent in exploring the common state space between related models. Specifically, FuseIC3 accumulates artifacts from the sequence of over-approximated reachable states, called frames, from earlier runs when checking new models, albeit, after careful repair. It uses bidirectional reachability; forward reachability to repair frames, and IC3-type backward reachability to block predecessors to bad states. We extensively evaluate FuseIC3 over a large collection of challenging benchmarks. FuseIC3 is on-average up to 5.48 $$\times $$ (median 1.75 $$\times $$ ) faster than checking each model individually, and up to 3.67 $$\times $$ (median 1.72 $$\times $$ ) faster than the state-of-the-art incremental IC3 algorithm. Moreover, we evaluate the performance improvement of FuseIC3 by smarter ordering of models and property grouping using a linear-time hashing approach.

Optimization Techniques and Formal Verification for the Software Design of Boolean Algebra Based Safety-Critical Systems

Artificial intelligence, and the ability to learn optimized solutions that comply with a set of safety rules, could facilitate the human-based design process of safety-critical systems. However, the reconciliation of state-of-the-art artificial intelligence technology with current safety standards and safety engineering processes is a challenge to be addressed. In this article, this publication describes a method based on optimization and on formal verification for the design of safety-critical systems that are defined by Boolean algebra. Several diverse optimization techniques and a hybrid of these approaches are used to find an optimized design that considers performance requirements, availability rules, and complies with all defined safety rules. Subsequently, this solution is translated into an alternative knowledge representation that can be formally verified and developed in compliance with currently considered safety standards. This method is evaluated with a simplified safety-critical case study.

Multiclock Constraint System Modelling and Verification for Ensuring Cooperative Autonomous Driving Safety

CADS (cooperative autonomous driving systems) are software-intensive and safety-critical reactive systems and give great promise to our daily life, but system errors may not be identified in the design stage until the implement stage, and the cost to correct them will be more expensive later than the early stage. For designing trustworthy autonomous software systems, we have to deal with multiclock constraint models. SysML (System Modeling Language) meets increasing adoption in order to carry out system-level modelling and verification against abstract representations, but it suffers from semantic ambiguities in the design of safety-critical autonomous systems. The main objective is to investigate methods for coping with the design and analysis models simultaneously and to achieve semantic consistency based on mathematical foundations and formal model transformation. In this paper, we propose a method to combine the requirement modelling process with analysis process together for CADS safety and reliability guarantee. Firstly, we extend SysML metamodels and construct SysML profile for the CADS domain that could improve modelling correctness and enhance reusability. An instantiated CADS model has been designed by means of adopting a profile containing different key functional and nonfunctional attributes and behaviors. Secondly, we define formal syntax and semantic notations for modelling elements in the SysML state machine diagram and show transformation rules between the state machine diagram and the CCSL (Clock Constraint Specification Language) model. Semantic preservation is also proved using the bisimulation relation between them for rigorous mapping correctness. Thirdly, a cooperative autonomous overtaking driving case study on the highway scenario is used for illustration, and we use the tool TimeSquare to simulate CCSL specification execution traces at the system design stage.

State of the Journal

During 2011, IEEE Transactions on Computers (TC) has maintained its position as a leading and prestigious publication in the field of computing. The number of manuscript submissions (inclusive of Special Sections) was 872 papers (at the time of writing this editorial in November 2011). The number of papers published was 139 papers plus approximately 140 additional papers were posted online as preprints (with an acceptance rate around 23.6%). Going by the number of papers received to date, I can guarantee you that 2012 will be another successful year for TC. The page budget stood at 1872 pages during 2011, which will maintained at the same level in 2012. As a measure of overall timely review, over the last 12 months the delay encountered from submission to fi rst notifi cation has been under three months. We are striving to improve this turnaround period and I am positive that this will be achieved due to the effort of the great team that we have. The Editor-in-Chie extends his gratitude to the wonderful team of Associate Editors, Guest Editors, reviewers, and IEEE Computer Society staff. Of course, we cannot forget our loyal authors and readers. A number of special sections have been published in 2011. TC does not publish special issues and the special sections are the mechanism used to enable the organization of mini special issues to publish on topical themes that are of importance to our readership. The themes covered in 2011 were Dependable Computer Architecture, Computer Arithmetic, Chips and Architectures for Emerging Technologies and Applications, Science of Design for Safety Critical Systems, and Concurrent Online Testing and Error/Fault Resilience of Digital Systems. The organization of such special sections will continue in 2012. Information on the scheduled special sections appears on the journal's homepage

Human Cognitive Bias Identification for Generating Safety Requirements in Safety Critical Systems

Safety critical systems are systems whose failure could result loss of life, economic damage, incidents, accidents or undesirable outcome, but it is not doubt that critical system safety has improved greatly under the development of the technology as the number of hardware and software induced accidents has been definitely reduced, but number of human deviations in their decision making found in each accident range remains more. We deeply reviewed traditional human error approaches and their limitations and propose approach of Human Cognitive Bias Identification Technique (H-CoBIT) that identifies, mitigates human potential cognitive biases and generates safety requirements during the initial phase of system Design. This proposed method, analyses the design of safety critical systems from a human factors perspective. It contributes system analyst, designers, and software engineers to identify potential cognitive biases (metal deviations in operator’s decision-making process) during the system use. To ensure the validity of the proposed method, we conducted an empirical experiment to validate the method for accuracy and reliability comparing different experimental outcomes using signal detection theorem.

Checsdm: A Method for Ensuring Consistency in Heterogeneous Safety-Critical System Design

Safety-critical systems are highly heterogeneous, combining different characteristics. Effectively designing such systems requires a complex modelling approach that deals with diverse components (e.g., mechanical, electronic, software)—each having its own underlying domain theories and vocabularies—as well as with various aspects of the same component (e.g., function, structure, behaviour). Furthermore, the regulated nature of such systems prescribes the objectives for their design verification and validation. This paper proposes <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">checsdm</i> , a systematic approach, based on Model-Driven Engineering (MDE), for assisting engineering teams in ensuring consistency of heterogeneous design of safety-critical systems. The approach is developed as a generic <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">methodology</i> and a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">tool framework</i> , that can be applied to various design scenarios involving different modelling languages and different design guidelines. The methodology comprises an iterative three-phased process. The first phase, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">elicitation</i> , aims at specifying requirements of the heterogeneous design scenario. Using the proposed tool framework, the second phase, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">codification</i> , consists in building a particular tool set that supports the heterogeneous design scenario and helps engineers in flagging consistency errors for review and eventual correction. The third phase, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">operation</i> , applies the tool set to actual system designs. Empirical evaluation of the work is presented through two executions of the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">checsdm</i> approach for the specific cases of a design scenario involving a mix of UML, Simulink and Stateflow, and a design scenario involving a mix of AADL, Simulink and Stateflow. The <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">operation</i> phase of the first case was performed over three avionics systems and the identified inconsistencies in the design models of these systems were compared to the results of a fully manual verification carried out by professional engineers. The evaluation also includes an assessment workshop with industrial practitioners to examine their perceptions about the approach. The empirical validation indicates the feasibility and “cost-effectiveness” of the approach. Inconsistencies were identified in the three avionics systems with a greater recall rate over the manual verification. The assessment workshop shows the practitioners found the approach easy to understand and gave an overall likelihood of adoption within the context of their work.

An Engineering Approach to Storage and Access in High Priority Scenarios

Over the past few years, an extensive amount of research has been done in the field of Human Factors. Applications range from the design of day-to-day products like cell phones to the design and development of safety-critical systems like flight displays. The highly critical aviation industry has shown time and again the importance of human-centered approach in developing systems for the safety of those operating it and the passengers. Similarly, other safety-critical industries like law enforcement have been seen to incorporate human factors in the design of weapons and exoskeletons aimed at adapting to humans and making their unit stronger. Many manufacturing firms have begun to see the importance of proper work postures for their employees to avoid musculoskeletal disorders and the financial and regulatory implications of not following proper work ethics that take care of employees’ health. Further, many organizations have started to consider team dynamics in their operations understanding the importance of healthy interaction among the employees and between employees and the management. However, there are a very few references to any studies or organizational practices that draw a connection between human performance and human-centric re-design of work places, with most designs being limited to work desks and activity-based working (ABW) work spaces. This paper focuses on the organizational engineering of storage spaces to enable easy location and retrieval of equipment, thus supporting the time-critical nature of operations at a miscellaneous storage room at the Story County Sheriff’s Office. Experiments were carried out using two familiar scenarios both before and after the redesign of the storage room. A significant improvement in the performance of the operator was observed after the redesign, as could be seen by the reduction in time taken to identify and retrieve equipment and the qualitative survey that was obtained at the end of the experiment. The wasted time was translated to a cost and the newly designed storage design saved a significant amount of money spent on actions that precluded efficient accomplishment of tasks, something that could have been used by the Sheriff’s office to purchase equipment for normal operation of the office. The results suggested such interventions in different sectors that have similar high-priority operations. The results of the study indicates that there is a need for the industry to extend research towards this field that we name “organization engineering”.

The new era of software reuse

The new era of software reuse

A cognitive architecture safety design for safety critical systems

This research is presented as a safety analysis of a cognitive architecture with an intelligent decision support model (IDSM) that is embedded into an autonomous non-deterministic safety-critical system.Cognitive technology is currently simulated within safety-critical systems in order to highlight variables of interest, interface with intelligent technologies, and provide an environment that improves the system's cognitive performance. In this research, the safety of the architecture was analyzed on an actual safety-critical system, an unmanned surface vehicle (USV). The safety analysis was conducted in both a simulated and a real world nautical based environment. The objective was to define the safety design of a cognitive architecture. The input to the safety design was provided through an approach that identified and mitigated hazards associated with a USV controlled by a cognitive architecture. This analysis provided a structured, task-oriented approach for the dissemination of information concerning safety requirements. This approach was necessary to achieve a safe execution of the USV's capabilities through a design that reduces the potential for injury to personnel and damage to equipment.Other real time applications that would benefit from advancing the safety of cognitive technologies are unmanned platforms, transportation technologies, and service robotics. The results will provide cognitive science researchers with a reference for safety engineering of artificially intelligent safety-critical systems.

A Formal Approach for Scalable Simulation of Gastric ICC Electrophysiology.

Efficient and accurate organ models are crucial for closed-loop validation of implantable medical devices. This paper investigates bio-electric slow wave modeling of the stomach, so that gastric electrical stimulator (GES) can be validated and verified prior to implantation. In particular, we consider high-fidelity, scalable, and efficient modeling of the pacemaker, Interstitial cells of Cajal (ICC), based on the formal hybrid input output automata (HIOA) framework. Our work is founded in formal methods, a collection of mathematically sound techniques originating in computer science for the design and validation of safety-critical systems. We modeled each ICC cell using an HIOA. We also introduce an HIOA path model to capture the electrical propagation delay between cells in a network. The resultant network of ICC cells can simulate normal and diseased action potential propagation patterns, making it useful for device validation. The simulated slow wave of a single ICC cell had high correlation ( ≈ 0.9) with the corresponding biophysical models. The proposed model is able to simulate the slow wave activity of a network of ICC cells with high-fidelity for device validation. The proposed HIOA model is significantly more efficient than the corresponding biophysical models, scales to larger networks of ICC cells, and is capable of simulating varying propagation patterns. This has the potential to enable verification and validation of implantable GESs in closed-loop with gastrointestinal models in the future.

Qualitative analysis for state/event fault trees using formal model checking

A state/event fault tree (SEFT) is a modeling technique for describing the causal chains of events leading to failure in software-controlled complex systems. Such systems are ubiquitous in all areas of everyday life, and safety and reliability analyses are increasingly required for these systems. SEFTs combine elements from the traditional fault tree with elements from state-based techniques. In the context of the real-time safety-critical systems, SEFTs do not describe the time properties and important time-dependent system behaviors that can lead to system failures. Further, SEFTs lack the precise semantics required for formally modeling time behaviors. In this paper, we present a qualitative analysis method for SEFTs based on transformation from SEFT to timed automata (TA), and use the model checker UPPAAL to verify system requirements' properties. The combination of SEFT and TA is an important step towards an integrated design and verification process for real-time safety-critical systems. Finally, we present a case study of a powerboat autopilot system to confirm our method is viable and valid after achieving the verification goal step by step.