Real-world Cyber-physical Systems Research Articles

Privacy-Preserving Deep Learning Framework Based on Restricted Boltzmann Machines and Instance Reduction Algorithms

The combination of collaborative deep learning and Cyber-Physical Systems (CPSs) has the potential to improve decision-making, adaptability, and efficiency in dynamic and distributed environments. However, it brings privacy, communication, and resource restrictions concerns that must be properly addressed for successful implementation in real-world CPS systems. Various privacy-preserving techniques have been proposed, but they often add complexity and decrease accuracy and utility. In this paper, we propose a privacy-preserving deep learning framework that combines Instance Reduction Techniques (IR) and the Restricted Boltzmann Machine (RBM) to preserve privacy while overcoming the limitations of other frameworks. The RBM encodes training data to retain relevant features, and IR selects the relevant encoded instances to send to the server for training. Privacy is preserved because only a small subset of the training data is sent to the server. Moreover, it is sent after encoding it using RBM. Experiments show that our framework preserves privacy with little loss of accuracy and a substantial reduction in training time. For example, using our framework, a CNN model for the MNIST dataset achieves 96% accuracy compared to 99% in a standard collaborative framework (with no privacy measures taken), with training time reduced from 133.259 s to 99.391 s. Our MLP model for MNIST achieves 97% accuracy compared to 98% in the standard collaborative framework, with training time reduced from 118.146 s to 87.873 s. Compared to other studies, our method is a simple approach that protects privacy, maintains the utility of deep learning models, and reduces training time and communication costs.

Diagnosis driven Anomaly Detection for Cyber-Physical Systems

In Cyber-Physical Systems (CPS) research, the detection of anomalies—identifying abnormal behaviors—and the diagnosis—pinpointing the underlying root causes—are frequently considered separate, isolated tasks. However, diagnostic algorithms necessitate symptoms, i.e., temporally and spatially isolated anomalies, as inputs. Therefore, integrating anomaly detection and diagnosis is essential for developing a comprehensive diagnostic solution for CPS. This paper introduces a method leveraging deep learning for anomaly detection to effectively identify and localize symptoms within CPS. Our approach is validated on both simulated and real-world CPS datasets, demonstrating robust performance in symptom detection and localization when compared to other state-of-the-art models.

Conceptualization, Realization, and Validation of Distributed Cyber-Physical Systems for Sensing and Automation

Abstract This paper investigates the feasibility of the pragmatic implementation of distributed algorithms in real-world cyber-physical systems (CPSs). We detail how distributed (and parallel) computing systems can be realized using existing system-design software for CPS development. A series of experiments is devised and used to verify the information-exchange capacity of the computing nodes and the synchronous and real-time operational effectiveness of the overall distributed computing system development framework. Finally, an actual distributed computing system is realized and validated by executing a fundamental distributed consensus protocol.

Digital Twin-based Anomaly Detection with Curriculum Learning in Cyber-physical Systems

Anomaly detection is critical to ensure the security of cyber-physical systems (CPS) . However, due to the increasing complexity of attacks and CPS themselves, anomaly detection in CPS is becoming more and more challenging. In our previous work, we proposed a digital twin-based anomaly detection method, called ATTAIN, which takes advantage of both historical and real-time data of CPS. However, such data vary significantly in terms of difficulty. Therefore, similar to human learning processes, deep learning models (e.g., ATTAIN) can benefit from an easy-to-difficult curriculum. To this end, in this paper, we present a novel approach, named digitaL twin-based Anomaly deTecTion wIth Curriculum lEarning ( LATTICE ) , which extends ATTAIN by introducing curriculum learning to optimize its learning paradigm. LATTICE attributes each sample with a difficulty score, before being fed into a training scheduler. The training scheduler samples batches of training data based on these difficulty scores such that learning from easy to difficult data can be performed. To evaluate LATTICE, we use five publicly available datasets collected from five real-world CPS testbeds. We compare LATTICE with ATTAIN and two other state-of-the-art anomaly detectors. Evaluation results show that LATTICE outperforms the three baselines and ATTAIN by 0.906%-2.367% in terms of the F1 score. LATTICE also, on average, reduces the training time of ATTAIN by 4.2% on the five datasets and is on par with the baselines in terms of detection delay time.

Uncertainty-aware Prediction Validator in Deep Learning Models for Cyber-physical System Data

The use of Deep learning in Cyber-Physical Systems (CPSs) is gaining popularity due to its ability to bring intelligence to CPS behaviors. However, both CPSs and deep learning have inherent uncertainty. Such uncertainty, if not handled adequately, can lead to unsafe CPS behavior. The first step toward addressing such uncertainty in deep learning is to quantify uncertainty. Hence, we propose a novel method called NIRVANA (uNcertaInty pRediction ValidAtor iN Ai) for prediction validation based on uncertainty metrics. To this end, we first employ prediction-time Dropout-based Neural Networks to quantify uncertainty in deep learning models applied to CPS data. Second, such quantified uncertainty is taken as the input to predict wrong labels using a support vector machine, with the aim of building a highly discriminating prediction validator model with uncertainty values. In addition, we investigated the relationship between uncertainty quantification and prediction performance and conducted experiments to obtain optimal dropout ratios. We conducted all the experiments with four real-world CPS datasets. Results show that uncertainty quantification is negatively correlated to prediction performance of a deep learning model of CPS data. Also, our dropout ratio adjustment approach is effective in reducing uncertainty of correct predictions while increasing uncertainty of wrong predictions.

Predictive Monitoring with Logic-Calibrated Uncertainty for Cyber-Physical Systems

Predictive monitoring—making predictions about future states and monitoring if the predicted states satisfy requirements—offers a promising paradigm in supporting the decision making of Cyber-Physical Systems (CPS). Existing works of predictive monitoring mostly focus on monitoring individual predictions rather than sequential predictions. We develop a novel approach for monitoring sequential predictions generated from Bayesian Recurrent Neural Networks (RNNs) that can capture the inherent uncertainty in CPS, drawing on insights from our study of real-world CPS datasets. We propose a new logic named Signal Temporal Logic with Uncertainty (STL-U) to monitor a flowpipe containing an infinite set of uncertain sequences predicted by Bayesian RNNs. We define STL-U strong and weak satisfaction semantics based on whether all or some sequences contained in a flowpipe satisfy the requirement. We also develop methods to compute the range of confidence levels under which a flowpipe is guaranteed to strongly (weakly) satisfy an STL-U formula. Furthermore, we develop novel criteria that leverage STL-U monitoring results to calibrate the uncertainty estimation in Bayesian RNNs. Finally, we evaluate the proposed approach via experiments with real-world CPS datasets and a simulated smart city case study, which show very encouraging results of STL-U based predictive monitoring approach outperforming baselines.

Engineering Swarms of Cyber-Physical Systems with the CPSwarm Workbench

Engineering swarms of cyber-physical systems (CPSs) is a complex process. We present the CPSwarm workbench that creates an automated design workflow to ease this process. This formalized workflow guides the user from modeling, to code generation, to deployment, both in simulation and on CPS hardware platforms. The workbench combines existing and emerging tools to solve real-world CPS swarm problems. As a proof-of-concept, we use the workbench to design a swarm of unmanned aerial vehicles (UAVs) and unmanned ground vehicles (UGVs) for a search and rescue (SAR) use case. We evaluate the resulting swarm behaviors on three levels. First, abstract simulations for rapid prototyping. Second, detailed simulation to test the correctness of the results. Third, deployment on hardware to demonstrate the applicability. We measure the swarm performance in terms of area covered and victims rescued. The results show that the performance of the swarm is proportional to its size. Despite some manual steps, the proposed workbench shows to be well suited to ease the complicated task of deploying a swarm of CPSs.

Resource Allocation and Service Provisioning in Multi-Agent Cloud Robotics: A Comprehensive Survey

Robotic applications nowadays are widely adopted to enhance operational automation and performance of real-world Cyber-Physical Systems (CPSs) including Industry 4.0, agriculture, healthcare, and disaster management. These applications are composed of latency-sensitive, data-heavy, and compute-intensive tasks. The robots, however, are constrained in the computational power and storage capacity. The concept of multi-agent cloud robotics enables robot-to-robot cooperation and creates a complementary environment for the robots in executing large-scale applications with the capability to utilize the edge and cloud resources. However, in such a collaborative environment, the optimal resource allocation for robotic tasks is challenging to achieve. Heterogeneous energy consumption rates and application of execution costs associated with the robots and computing instances make it even more complex. In addition, the data transmission delay between local robots, edge nodes, and cloud data centres adversely affects the real-time interactions and impedes service performance guarantee. Taking all these issues into account, this paper comprehensively surveys the state-of-the-art on resource allocation and service provisioning in multi-agent cloud robotics. The paper presents the application domains of multi-agent cloud robotics through explicit comparison with the contemporary computing paradigms and identifies the specific research challenges. A complete taxonomy on resource allocation is presented for the first time, together with the discussion of resource pooling, computation offloading, and task scheduling for efficient service provisioning. Furthermore, we highlight the research gaps from the learned lessons, and present future directions deemed beneficial to further advance this emerging field.

On Physical-Social-Aware Localness Inference by Exploring Big Data from Location-Based Services

A user's localness (i.e., whether a user is a local resident in a city or not) and a venue's local attractiveness (i.e., the likelihood of a venue to attract local people) are important information for many location-based applications related with Cyber-Physical Systems (CPS), such as participatory sensing, urban planning, traffic control and localized travel recommendations. Previous effort has been devoted to geo-locating users in a city using supervised learning approaches, which depend on the availability of high quality training datasets. However, it is difficult to obtain such training datasets in the real-world CPS applications due to the issue of privacy. In this work, we develop an unsupervised approach, called a Physical-Social-Aware Inference (PSAI) scheme, to jointly infer a user's localness and a venue's local attractiveness by exploring both the physical and social information embedded in the location-based social networks (LBSN). We further implement a parallel PSAI framework on the platform of a Graphic Processing Unit (GPU) to enhance its ability to process large-scale data. Our extensive experiments on the real-world LBSN datasets demonstrate the effectiveness and efficiency of the PSAI scheme compared to the state-of-the-art baselines.

RTCE: Real-Time Co-Emulation Framework for EMT-Based Power System and Communication Network on FPGA-MPSoC Hardware Architecture

With the expansion of smart grid infrastructure world-wide, modeling the interaction between power systems and communication networks becomes paramount and has created a new challenge of co-simulating the two domains before commissioning. Existing co-simulation methods mostly concentrate on the off-line software-level interface design to synchronize messages between the simulators of both domains. Instead of simulating in software with a large latency, this article proposes a novel real-time co-emulation (RTCE) framework on FPGA-MPSoC based hardware architecture for a more practical emulation of real-world cyber-physical systems. The discrete-time based power system electromagnetic transient (EMT) emulation is executed in programmable hardware units so that the transient-level behaviour can be captured in real-time, while the discrete-event based communication network emulation is modeled in abstraction-level or directly executed on the hardware PHY and network ports of the FPGA-MPSoC platform, which can perform the communication networking in real-time. The data exchange between two domains is handled within each platform with an extremely low latency, which is sufficiently fast for real-time interaction; and the multi-board scheme is deployed to practically emulate the communication between different power system areas. The hardware resource cost and emulation latency for the test system and case studies are evaluated to demonstrate the validity and effectiveness of the proposed RTCE framework.

Improving Verification Accuracy of CPS by Modeling and Calibrating Interaction Uncertainty

Cyber-Physical Systems (CPS) intrinsically combine hardware and physical systems with software and network, which are together creating complex and correlated interactions. CPS applications often experience uncertainty in interacting with environment through unreliable sensors. They can be faulty and exhibit runtime errors if developers have not considered environmental interaction uncertainty adequately. Existing work in verifying CPS applications ignores interaction uncertainty and thus may overlook uncertainty-related faults. To improve verification accuracy, in this article we propose a novel approach to verifying CPS applications with explicit modeling of uncertainty arisen in the interaction between them and the environment. Our approach builds an Interactive State Machine network for a CPS application and models interaction uncertainty by error ranges and distributions. Then it encodes both the application and uncertainty models to Satisfiability Modulo Theories (SMT) formula to leverage SMT solvers searching for counterexamples that represent application failures. The precision of uncertainty model can affect the verification results. However, it may be difficult to model interaction uncertainty precisely enough at the beginning, because of the uncontrollable noise of sensors and insufficient data sample size. To further improve the accuracy of the verification results, we propose an approach to identifying and calibrating imprecise uncertainty models. We exploit the inconsistency between the counterexamples’ estimate and actual occurrence probabilities to identify possible imprecision in uncertainty models, and the calibration of imprecise models is to minimize the inconsistency, which is reduced to a Search-Based Software Engineering problem. We experimentally evaluated our verification and calibration approaches with real-world CPS applications, and the experimental results confirmed their effectiveness and efficiency.

Parameter-Invariant Monitor Design for Cyber–Physical Systems

The tight interaction between information technology and the physical world inherent in cyber–physical systems (CPS) can challenge traditional approaches for monitoring safety and security. Data collected for robust CPS monitoring is often sparse and may lack rich training data describing critical events/attacks. Moreover, CPS often operate in diverse environments that can have significant inter/intra-system variability. Furthermore, CPS monitors that are not robust to data sparsity and inter/intra-system variability may result in inconsistent performance and may not be trusted for monitoring safety and security. Towards overcoming these challenges, this paper presents recent work on the design of parameter-invariant (PAIN) monitors for CPS. PAIN monitors are designed such that unknown events and system variability minimally affect the monitor performance. This work describes how PAIN designs can achieve a constant false alarm rate (CFAR) in the presence of data sparsity and intra/inter system variance in real-world CPS. To demonstrate the design of PAIN monitors for safety monitoring in CPS with different types of dynamics, we consider systems with networked dynamics, linear-time invariant dynamics, and hybrid dynamics that are discussed through case studies for building actuator fault detection, meal detection in type I diabetes, and detecting hypoxia caused by pulmonary shunts in infants. In all applications, the PAIN monitor is shown to have (significantly) less variance in monitoring performance and (often) outperforms other competing approaches in the literature. Finally, an initial application of PAIN monitoring for CPS security is presented along with challenges and research directions for future security monitoring deployments.

AdaFT

Cyber-physical systems (CPS) frequently have to use massive redundancy to meet application requirements for high reliability. While such redundancy is required, it can be activated adaptively, based on the current state of the controlled plant. Most of the time, the plant is in a state that allows for a lower level of fault tolerance. Avoiding the continuous deployment of massive fault tolerance will greatly reduce the workload of the CPS, and lower the operating temperature of the cyber sub-system, thus increasing its reliability. In this article, we extend our prior research by demonstrating a software simulation framework Adaptive Fault Tolerance (AdaFT) that can automatically generate the sub-spaces within which our adaptive fault tolerance can be applied. We also show the theoretical benefits of AdaFT and its actual implementation in several real-world CPSs.

Machine Learning-based CPS for Clustering High throughput Machining Cycle Conditions

Cyber-physical systems (CPS) have opened up a wide range of opportunities in terms of performance analysis that can be applied directly to the machine tool industry and are useful for maintenance systems and machine designers. High-speed communication capabilities enable the data to be gathered, pre-processed and processed for the purpose of machine diagnosis. This paper describes a complete real-world CPS implementation cycle, ranging from machine data acquisition to processing and interpretation. In fact, the aim of this paper is to propose a CPS for machine component knowledge discovery based on clustering algorithms using real data from a machining process. Therefore, it compares three clustering algorithms –k-means, hierarchical agglomerative and Gaussian mixture models– in terms of their contribution to spindle performance knowledge during high throughput machining operation.

Collaborative Verification-Driven Engineering of Hybrid Systems

Hybrid systems with both discrete and continuous dynamics are an important model for real-world cyber-physical systems. The key challenge is to ensure their correct functioning w.r.t. safety requirements. Promising techniques to ensure safety seem to be model-driven engineering to develop hybrid systems in a well-defined and traceable manner, and formal verification to prove their correctness. Their combination forms the vision of verification-driven engineering. Often, hybrid systems are rather complex in that they require expertise from many domains (e.g., robotics, control systems, computer science, software engineering, and mechanical engineering). Moreover, despite the remarkable progress in automating formal verification of hybrid systems, the construction of proofs of complex systems often requires nontrivial human guidance, since hybrid systems verification tools solve undecidable problems. It is, thus, not uncommon for development and verification teams to consist of many players with diverse expertise. This paper introduces a verification-driven engineering toolset that extends our previous work on hybrid and arithmetic verification with tools for (i) graphical (UML) and textual modeling of hybrid systems, (ii) exchanging and comparing models and proofs, and (iii) managing verification tasks. This toolset makes it easier to tackle large-scale verification tasks.