Safety-critical Cyber-physical Systems Research Articles

Compositional Thinking in Cyberphysical Systems Theory

Engineering safe and secure cyber-physical systems requires system engineers to develop and maintain a number of model views, both dynamic and static, which can be seen as algebras. We posit that verifying the composition of requirement, behavioral, and architectural models using category theory gives rise to a strictly compositional interpretation of cyber-physical systems theory, which can assist in the modeling and analysis of safety-critical cyber-physical systems.

Unified graphical co-modeling, analysis and verification of cyber-physical systems by combining AADL and Simulink/Stateflow

The efficient design of safety-critical cyber-physical systems (CPS) requires the co-modeling and -verification of its physics, architecture and functionalities. Existing co-modeling formalisms do not account for these three design aspects into account uniformly. AADL is a precise formalism for modeling architecture and prototyping real-time hardware platforms, but it delegates co-modeling physical and software behaviors to so-called annexes. By contrast, Simulink/Stateflow (S/S) is strong for modeling interacting physical and software behaviors, but weak for modeling architecture and hardware platforms. To address this issue, this paper considers the combination of AADL and S/S to co-model CPSs and presents a method to uniformly analyze and verify this combination. AADL⊕S/S provides a unified graphical co-modeling environment for CPS design and supports simulation through C code generation. We present a formal semantics of AADL⊕S/S by translating it to Hybrid Communicating Sequential Processes (HCSP), which yields a deductive verification framework for AADL⊕S/S models based on Hybrid Hoare Logic (HHL). We also prove the correctness of the translation of AADL⊕S/S to HCSP. The effectiveness of our approach is illustrated by the realistically-scaled case study of an automatic cruise control system.

Adversarial attack and defense methods for neural network based state estimation in smart grid

Deep learning has been recently used in safety-critical cyber-physical systems (CPS) such as the smart grid. The security assessment of such learning-based methods within CPS algorithms, however, is still an open problem. Despite existing research on adversarial attacks against deep learning models, only few works are concerned about safety-critical energy CPS, especially the state estimation routine. This paper investigates security issues of neural network based state estimation in the smart grid. Specifically, the problem of adversarial attacks against neural network based state estimation is analysed and an efficient adversarial attack method is proposed. To thwart this attack, two defense methods based on protection and adversarial training, respectively, are proposed further. The experiments demonstrate that the proposed attack method poses a major threat to neural network based state estimation models. In addition, our results present that defense methods can improve the ability of neural network models to defend against such adversarial attacks.

Improving Variational Autoencoder based Out-of-Distribution Detection for Embedded Real-time Applications

Uncertainties in machine learning are a significant roadblock for its application in safety-critical cyber-physical systems (CPS). One source of uncertainty arises from distribution shifts in the input data between training and test scenarios. Detecting such distribution shifts in real-time is an emerging approach to address the challenge. The high dimensional input space in CPS applications involving imaging adds extra difficulty to the task. Generative learning models are widely adopted for the task, namely out-of-distribution (OoD) detection. To improve the state-of-the-art, we studied existing proposals from both machine learning and CPS fields. In the latter, safety monitoring in real-time for autonomous driving agents has been a focus. Exploiting the spatiotemporal correlation of motion in videos, we can robustly detect hazardous motion around autonomous driving agents. Inspired by the latest advances in the Variational Autoencoder (VAE) theory and practice, we tapped into the prior knowledge in data to further boost OoD detection’s robustness. Comparison studies over nuScenes and Synthia data sets show our methods significantly improve detection capabilities of OoD factors unique to driving scenarios, 42% better than state-of-the-art approaches. Our model also generalized near-perfectly, 97% better than the state-of-the-art across the real-world and simulation driving data sets experimented. Finally, we customized one proposed method into a twin-encoder model that can be deployed to resource limited embedded devices for real-time OoD detection. Its execution time was reduced over four times in low-precision 8-bit integer inference, while detection capability is comparable to its corresponding floating-point model.

Towards a Verification-Driven Iterative Development of Software for Safety-Critical Cyber-Physical Systems

Software systems are complicated, and the scientific and engineering methodologies for software development are relatively young. Cyber-physical systems are now in every corner of our lives, and we need robust methods for handling the ever-increasing complexity of their software systems. Model-Driven Development is a promising approach to tackle the complexity of systems through the concept of abstraction, enabling analysis at earlier phases of development. In this paper, we propose a model-driven approach with a focus on guaranteeing safety using formal verification. Cyber-physical systems are distributed, concurrent, asynchronous and event-based reactive systems with timing constraints. The actor-based textual modeling language, Rebeca, with model checking support is used for formal verification. Starting from structured requirements and system architecture design the behavioral models, including Rebeca models, are built. Properties of interest are also derived from the structured requirements, and then model checking is used to formally verify the properties. This process can be performed in iterations until satisfaction of desired properties are ensured, and possible ambiguities and inconsistencies in requirements are resolved. The formally verified models can then be used to develop the executable code. The Rebeca models include the details of the signals and messages that are passed at the network level including the timing, and this facilitates the generation of executable code. The natural mappings among the models for requirements, the formal models, and the executable code improve the effectiveness and efficiency of the approach.

Towards a Statistical Model Checking Method for Safety-Critical Cyber-Physical System Verification

Safety-Critical Cyber-Physical System (SCCPS) refers to the system that if the system fails or its key functions fail, it will cause casualties, property damage, environmental damage, and other catastrophic consequences. Therefore, it is vital to verify the safety of safety critical systems. In the community, the SCCPS safety verification mainly relies on the statistical model checking methodology, but for SCCPS with extremely high safety requirements, the statistical model checking method is difficult/infeasible to sample the extremely small probability event since the probability of the system violating the safety is very low (rare property). In response to this problem, we propose a new method of statistical model checking for high-safety SCCPS. Firstly, with the CTMC-approximated SCCPS path probability space model, it leverages the maximum likelihood estimation method to learn the parameters of CTMC. Then, the embedded DTMC can be derived from CTMC, and a cross-entropy optimization model based on DTMC can be constructed. Finally, we propose an algorithm of iteratively learning the optimal importance sampling distribution on the discrete path space and an algorithm to check the statistical model of verifying the rare attribute. Eventually, experimental results show that the method proposed in this paper can effectively verify the rare attributes of SCCPS. Under the same sample size, comparing with the heuristic importance sampling methods, the estimated value of this method can be better distributed around the mean value, and the related standard deviation and relative error are reduced by more than an order of magnitude.

Model based security verification of Cyber-Physical System based on Petrinet: A case study of Nuclear power plant

A variety of modern Cyber-Physical Systems (CPSs) are distributed and asynchronous systems that are becoming the backbone for smart infrastructures and systems such as smart grids, power plants, medical appliances, social robots etc. The weaving of cyber components with the physical system improves resource utilization and system reliability. However, it makes the CPSs significantly vulnerable to several cyber threats by increasing the attack surfaces. In safety-critical cyber-physical systems, the primary focus is on safety assurance. Any mistake or ignorance in security analysis may greatly amplify the losses in the presence of cyber threats. Such security failure may result in severe damages that may range from affecting critical infrastructures up to even loss of human lives. Thus, an early-staged security modeling of these systems is a prominent issue and requires a systematic approach for modeling a secure design as well as performance evaluation of alternative designs for the same. An in-depth security analysis involves the identification of a standard set of evaluation metrics. The proposed work provides the design-time methodology to map and analyze system security qualitatively and quantitatively using Stochastic Petri nets and their fundamental properties. The effectiveness of the proposed methodology is evaluated using a Nuclear power plant (NPP) case study.

Towards a framework of enforcing resilient operation of cyber‐physical systems with unknown dynamics

Ensuring that safety-critical cyber-physical systems (CPSs) continue to satisfy correctness and safety specifications even under faults or adversarial attacks is very challenging, especially in the presence of legacy components for which accurate models are unknown to the designer. Current techniques for secure-by-design systems engineering do not provide an end-to-end methodology for a designer to provide real-time assurance for safety-critical CPSs by identifying system dynamics and updating control strategies in response to newly discovered faults, attacks or other changes such as system upgrades. We propose a new methodology, along with an integrated framework implemented in MATLAB to guarantee the resilient operation of safety-critical CPSs with unknown dynamics. The proposed framework consists of three main components. The runtime monitor evaluates the system behaviour on-the-fly against its correctness specifications expressed as signal temporal logic formulas. The model synthesiser incorporates a sparse identification approach that is used to continually update the plant model and control policies to adapt to any changes in the system or the environment. The decision and control module designs a controller to ensure that the correctness specifications are satisfied at runtime. For evaluation, we apply our proposed framework to ensure the resilient operations of two CPS case studies.

Switching controller synthesis for time-delayed hybrid systems

How to design safe and reliable safety-critical cyber-physical systems (CPSs) so thatwe can bet our daily lives on them is a grand challenge to computer science and control theory. Delays in feedback control are ubiquitous, i.e., the behavior evolution of a system depends not only on its current state, but also on its execution history. Obviously, delays may invalidate the stability/safetycertificates obtained by abstracting them away as in the design of modern CPSs normally. In this paper, we study the switching controller synthesis problem of time-delayed hybrid systems, and propose an invariant-based approach by extending the corresponding approach to the design of CPSs without delays. To this end, based on spectral analysis and linearization, we first show that the differential invariant generation problem of delayed dynamical systems can be reduced to computing reachable sets over bounded time horizons; we then propose an abstract-based algorithm to over-approximate the reachable set over a given time bound; finally, we implement a prototypical tool of our approach and illustrate it with examples.

Challenges of Machine Learning Applied to Safety-Critical Cyber-Physical Systems

Machine Learning (ML) is increasingly applied for the control of safety-critical Cyber-Physical Systems (CPS) in application areas that cannot easily be mastered with traditional control approaches, such as autonomous driving. As a consequence, the safety of machine learning became a focus area for research in recent years. Despite very considerable advances in selected areas related to machine learning safety, shortcomings were identified on holistic approaches that take an end-to-end view on the risks associated to the engineering of ML-based control systems and their certification. Applying a classic technique of safety engineering, our paper provides a comprehensive and methodological analysis of the safety hazards that could be introduced along the ML lifecycle, and could compromise the safe operation of ML-based CPS. Identified hazards are illustrated and explained using a real-world application scenario—an autonomous shop-floor transportation vehicle. The comprehensive analysis presented in this paper is intended as a basis for future holistic approaches for safety engineering of ML-based CPS in safety-critical applications, and aims to support the focus on research onto safety hazards that are not yet adequately addressed.

A sensorless state estimation for a safety-oriented cyber-physical system in urban driving: Deep learning approach

In today's modern electric vehicles, enhancing the safety-critical cyber-physical system (CPS)'s performance is necessary for the safe maneuverability of the vehicle. As a typical CPS, the braking system is crucial for the vehicle design and safe control. However, precise state estimation of the brake pressure is desired to perform safe driving with a high degree of autonomy. In this paper, a sensorless state estimation technique of the vehicle's brake pressure is developed using a deep-learning approach. A deep neural network (DNN) is structured and trained using deep-learning training techniques, such as, dropout and rectified units. These techniques are utilized to obtain more accurate model for brake pressure state estimation applications. The proposed model is trained using real experimental training data which were collected via conducting real vehicle testing. The vehicle was attached to a chassis dynamometer while the brake pressure data were collected under random driving cycles. Based on these experimental data, the DNN is trained and the performance of the proposed state estimation approach is validated accordingly. The results demonstrate high-accuracy brake pressure state estimation with RMSE of 0.048 MPa.

Designing Neural Networks for Real-Time Systems

Artificial Neural Networks (ANNs) are increasingly being used within safety-critical Cyber-Physical Systems (CPSs). They are often co-located with traditional embedded software, and may perform advisory or control-based roles. It is important to validate both the timing and functional correctness of these systems. However, most approaches in the literature consider guaranteeing only the functionality of ANN based controllers. This issue stems largely from the implementation strategies used within common neural network frameworks -- their underlying source code is often simply unsuitable for formal techniques such as static timing analysis. As a result, developers of safety-critical CPS must rely on informal techniques such as measurement based approaches to prove correctness, techniques that provide weak guarantees at best. In this work we address this challenge. We propose a design pipeline whereby neural networks trained using the popular deep learning framework Keras are compiled to functionally equivalent C code. This C code is restricted to simple constructs that may be analysed by existing static timing analysis tools. As a result, if compiled to a suitable time-predictable platform all execution bounds may be statically derived. To demonstrate the benefits of our approach we execute an ANN trained to drive an autonomous vehicle around a race track. We compile the ANN to the Patmos time-predictable controller, and show that we can derive worst case execution timings.

The System That Cried Wolf

Fire alarm and signaling systems are a networked system of fire detectors, fire control units, automated fire extinguishers, and fire notification appliances. Malfunction of these safety-critical cyber-physical systems may lead to chaotic evacuations, property damage, and even loss of human life. Therefore, reliability is one of the most crucial factors for fire detectors. Indeed, even a single report of a fire cannot be ignored, considering the importance of early fire detection and suppression. In this article, we show that wide-area smoke detectors, which are globally installed in critical infrastructures such as airports, sports facilities, and auditoriums, have significant vulnerabilities in terms of reliability; one can remotely and stealthily induce false fire alarms and suppress real fire alarms with a minimal attacker capability using simple equipment. The practicality and generalizability of these vulnerabilities has been assessed based on the demonstration of two types of sensor attacks on two commercial off-the-shelf optical beam smoke detectors from different manufacturers. Further, the practical considerations of building stealthy attack equipment has been analyzed, and an extensive survey of almost all optical beam smoke detectors on the market has been conducted. In addition, we show that the current standards of the fire alarm network connecting the detector and a control unit exacerbate the problem, making it impossible or very difficult to mitigate the threats we found. Finally, we discuss hardware- and software-based possible countermeasures for both wide-area smoke detectors and the fire alarm network; the effectiveness of one of the countermeasures is experimentally evaluated.

Timing Predictability and Security in Safety-Critical Industrial Cyber-Physical Systems: A Position Paper

Cyber Physical Systems (CPSs) are systems that are developed by seamlessly integrating computational algorithms and physical components, and they are a result of the technological advancement in the embedded systems and distributed systems domains, as well as the availability of sophisticated networking technology. Many industrial CPSs are subject to timing predictability, security and functional safety requirements, due to which the developers of these systems are required to verify these requirements during the their development. This position paper starts by exploring the state of the art with respect to developing timing predictable and secure embedded systems. Thereafter, the paper extends the discussion to time-critical and secure CPSs and highlights the key issues that are faced when verifying the timing predictability requirements during the development of these systems. In this context, the paper takes the position to advocate paramount importance of security as a prerequisite for timing predictability, as well as both security and timing predictability as prerequisites for functional safety. Moreover, the paper identifies the gaps in the existing frameworks and techniques for the development of time- and safety-critical CPSs and describes our viewpoint on ensuring timing predictability and security in these systems. Finally, the paper emphasises the opportunities that artificial intelligence can provide in the development of these systems.

Testing and verification of neural-network-based safety-critical control software: A systematic literature review

Context: Neural Network (NN) algorithms have been successfully adopted in a number of Safety-Critical Cyber-Physical Systems (SCCPSs). Testing and Verification (T&V) of NN-based control software in safety-critical domains are gaining interest and attention from both software engineering and safety engineering researchers and practitioners. Objective: With the increase in studies on the T&V of NN-based control software in safety-critical domains, it is important to systematically review the state-of-the-art T&V methodologies, to classify approaches and tools that are invented, and to identify challenges and gaps for future studies. Method: We retrieved 950 papers on the T&V of NN-based Safety-Critical Control Software (SCCS). To reach our result, we filtered 83 primary papers published between 2001 and 2018, applied the thematic analysis approach for analyzing the data extracted from the selected papers, presented the classification of approaches, and identified challenges. Conclusion: The approaches were categorized into five high-order themes: assuring robustness of NNs, assuring safety properties of NN-based control software, improving the failure resilience of NNs, measuring and ensuring test completeness, and improving the interpretability of NNs. From the industry perspective, improving the interpretability of NNs is a crucial need in safety-critical applications. We also investigated nine safety integrity properties within four major safety lifecycle phases to investigate the achievement level of T&V goals in IEC 61508-3. Results show that correctness, completeness, freedom from intrinsic faults, and fault tolerance have drawn most attention from the research community. However, little effort has been invested in achieving repeatability; no reviewed study focused on precisely defined testing configuration or on defense against common cause failure.

Adaptive Time-Triggered Multi-Core Architecture

The static resource allocation in time-triggered systems offers significant benefits for the safety arguments of dependable systems. However, adaptation is a key factor for energy efficiency and fault recovery in Cyber-Physical System (CPS). This paper introduces the Adaptive Time-Triggered Multi-Core Architecture (ATMA), which supports adaptation using multi-schedule graphs while preserving the key properties of time-triggered systems including implicit synchronization, temporal predictability and avoidance of resource conflicts. ATMA is an overall architecture for safety-critical CPS based on a network-on-a-chip with building blocks for context agreement and adaptation. Context information is established in a globally consistent manner, providing the foundation for the temporally aligned switching of schedules in the network interfaces. A meta-scheduling algorithm computes schedule graphs and avoids state explosion with reconvergence horizons for events. For each tile, the relevant part of the schedule graph is efficiently stored using difference encodings and interpreted by the adaptation logic. The architecture was evaluated using an FPGA-based implementation and example scenarios employing adaptation for improved energy efficiency. The evaluation demonstrated the benefits of adaptation while showing the overhead and the trade-off between the degree of adaptation and the memory consumption for multi-schedule graphs.

Survey on security and privacy issues in cyber physical systems

The notion of Cyber-Physical Systems (CPS) is proposed by the National ScientificFoundation to describe a type of systems which combine hardware and software components andbeing the next step in development of embedded systems. CPS includes a wide range of researchtopics ranging from signal processing to data analysis. This paper contains a brief review of the basicinfrastructure for CPS including smart objects and network aspects in relation to TCP/IP stack. AsCPS reflect the processes of the physical environment onto the cyber space, virtualisation as animportant tool for abstraction plays crucial role in CPS. In this context paper presents the challengesassociated with mobility and vritualisation; accordingly three main types of virtualisation, namelynetwork, devices and applications virtualisation are presented in the paper. These aspects are tightlycoupled with security and safety issues. Therefore, different threats, attack types with correspondingsubtypes and possible consequences are discussed as well as analysis of various approaches to copewith existing threats is introduced. In addition threat modelling approaches were also in scope of thiswork. Furthermore, needs and requirements for safety-critical CPS are reviewed. Thus the mainefforts of this paper are directed on introducing various aspects of the CPS with regard to securityand safety issues.

High-Precision Modulation of a Safety-Critical Cyber-Physical System: Control Synthesis and Experimental Validation

As a typical safety-critical cyber-physical system (CPS), the brake-by-wire (BBW) system, which provides improved control and enhanced performance, shows promise as an automotive braking scheme. In addition, the BBW system can be incorporated into all types of vehicles, including conventional vehicle with an internal combustion engine, electrified vehicles, and intelligent vehicles. However, the high cost and complexity of existing BBW systems limit their usage. In this paper, we consider a novel BBW system that we have developed, which has the advantages of having a simple structure and low cost. To enhance control accuracy of hydraulic pressure modulation and reduce noise, two pressure modulation methods, namely, the hydraulic pump−based pressure modulation (HPBPM) and the closed-loop pressure-difference-limiting (CLPDL) modulation, were proposed. So as to demonstrate the performance of the proposed approach, hardware-in-the-loop (HiL) experiments were conducted. The experimental results indicated that the proposed BBW system, which incorporates the CLPDL control method, appears to be a promising system in terms of the overall performance, including control performance, noise reduction, and system simplification.

TORUS

Cyber-Physical Systems (CPS) contain intertwined and distributed software, hardware, and physical components to control complex physical processes. They find wide application in industrial systems, such as smart grid protection systems, which face increasingly complex communication and computation needs. Due to the scale and complexity of the interactions that occur within CPS, tracing requirements through to the system components and software code that implement them is often hard. Existing requirements management systems do not scale well, and traceability is difficult to implement and maintain in highly heterogeneous systems. However, the information trace that links provide is crucial for supporting testing and certification activities in safety-critical environments such as smart grids. The well-formed models of power systems provided by the IEC 61850 standard and the software design structure provided by the IEC 61499 Function Blocks standard can be leveraged to automate many traceability operations. We present Traceability of Requirements Using Splices (TORUS), a novel traceability framework for the development of large-scale safety-critical CPS. TORUS introduces splices , autonomous graph-based data structures that automatically create and manage trace links between requirements and components through the inevitable changes that occur during system development. The formal, graph-based structure of TORUS lends itself well to the development of sophisticated algorithms to automate the extraction of useful traceability information such as historical records and metrics for requirements coverage and component coupling. By capturing not only the current state of the system but also historical information, TORUS allows project teams to see a much richer view of the system and its artifacts. We apply TORUS to the development of a protection system for smart grid substations. In addition, through a number of experiments in splice creation, modification, and application of automated algorithms, we show that TORUS scales easily to large systems containing hundreds of thousands of requirements and system components and millions of possible trace links.

Levenberg–Marquardt Backpropagation Training of Multilayer Neural Networks for State Estimation of a Safety-Critical Cyber-Physical System

As an important safety-critical cyber-physical system (CPS), the braking system is essential to the safe operation of the electric vehicle. Accurate estimation of the brake pressure is of great importance for automotive CPS design and control. In this paper, a novel probabilistic estimation method of brake pressure is developed for electrified vehicles based on multilayer artificial neural networks (ANNs) with Levenberg–Marquardt backpropagation (LMBP) training algorithm. First, the high-level architecture of the proposed multilayer ANN for brake pressure estimation is illustrated. Then, the standard backpropagation (BP) algorithm used for training of the feed-forward neural network (FFNN) is introduced. Based on the basic concept of BP, a more efficient training algorithm of LMBP method is proposed. Next, real vehicle testing is carried out on a chassis dynamometer under standard driving cycles. Experimental data of the vehicle and the powertrain systems are collected, and feature vectors for FFNN training collection are selected. Finally, the developed multilayer ANN is trained using the measured vehicle data, and the performance of the brake pressure estimation is evaluated and compared with other available learning methods. Experimental results validate the feasibility and accuracy of the proposed ANN-based method for braking pressure estimation under real deceleration scenarios.