Data In Cyber-physical Systems Research Articles

Out-of-Distribution Detection in Dependent Data for Cyber-Physical Systems with Conformal Guarantees

Uncertainty in the predictions of learning-enabled components hinders their deployment in safety-critical cyber-physical systems (CPS). A shift from the training distribution of a learning-enabled component (LEC) is one source of uncertainty in the LEC’s predictions. Detection of this shift or out-of-distribution (OOD) detection on individual datapoints has therefore gained attention recently. But in many applications, inputs to CPS form a temporal sequence. Existing techniques for OOD detection in time-series data for CPS either do not exploit temporal relationships in the sequence or do not provide any guarantees on detection. We propose using deviation from the in-distribution temporal equivariance as the non-conformity measure in conformal anomaly detection framework for OOD detection in time-series data for CPS. Computing independent predictions from multiple conformal detectors based on the proposed measure and combining these predictions by Fisher’s method leads to the proposed detector CODiT with bounded false alarms. CODiT performs OOD detection on fixed-length windows of consecutive time-series datapoints by using Fisher value of the input window. We further propose performing OOD detection on real-time time-series traces of variable lengths with bounded false alarms. This can be done by using CODiT to compute Fisher values of the sliding windows in the input trace and combining these values by a merging function. Merging functions such as Harmonic Mean, Arithmetic Mean, Geometric Mean, and Bonferroni Method, etc. can be used to combine Fisher values of the sliding windows in the input trace, and the combined value can be used for OOD detection on the trace with bounded false alarm rate guarantees. We illustrate the efficacy of CODiT by achieving state-of-the-art results in two case studies for OOD detection on fixed-length windows. The first one is on an autonomous driving system with perception (or vision) LEC. The second case study is on a medical CPS for walking pattern or GAIT analysis where physiological (non-vision) data is collected with force-sensitive resistors attached to the subject’s body. For OOD detection on variable length traces, we consider the same case studies on the autonomous driving system and medical CPS for GAIT analysis. We report our results with four merging functions on the Fisher values computed by CODiT on the sliding windows of the input trace. We also compare the false alarm rate guarantees by these four merging functions in the autonomous driving system case study. Code, data, and trained models are available at https://github.com/kaustubhsridhar/time-series-OOD.

A novel ensemble learning approach for fault detection of sensor data in cyber-physical system

Cyber-physical systems (CPS) play a pivotal role in various critical applications, ranging from industrial automation to healthcare monitoring. Ensuring the reliability and accuracy of sensor data within these systems is of paramount importance. This research paper presents a novel approach for enhancing fault detection in sensor data within a cyber-physical system through the integration of machine learning algorithms. Specifically, a hybrid ensemble methodology is proposed, combining the strengths of AdaBoost and Random Forest with Rocchio’s algorithm, to achieve robust and accurate fault detection. The proposed approach operates in two phases. In the first phase, AdaBoost and Random Forest classifiers are trained on a diverse dataset containing normal and faulty sensor data to develop individual base models. AdaBoost emphasizes misclassified instances, while Random Forest focuses on capturing complex interactions within the data. In the second phase, the outputs of these base models are fused using Rocchio’s algorithm, which exploits the similarities between faulty instances to improve fault detection accuracy. Comparative analyses are conducted against individual classifiers and other ensemble methods to validate the effectiveness of the hybrid approach. The results demonstrate that the proposed approach achieves superior fault detection rates. Additionally, the integration of Rocchio’s algorithm significantly contributes to the refinement of the fault detection process, effectively leveraging the strengths of AdaBoost and Random Forest. In conclusion, this research offers a comprehensive solution to enhance fault detection capabilities in cyber-physical systems by introducing a novel ensemble framework. By synergistically combining AdaBoost, Random Forest, and Rocchio’s algorithm, the proposed methodology provides a robust mechanism for accurately identifying sensor data anomalies, thus bolstering the reliability and performance of cyber-physical systems across a multitude of critical applications.

Towards DevOps for Cyber-Physical Systems (CPSs): Resilient Self-Adaptive Software for Sustainable Human-Centric Smart CPS Facilitated by Digital Twins

The Industrial Revolution drives the digitization of society and industry, entailing Cyber-Physical Systems (CPSs) that form ecosystems where system owners and third parties share responsibilities within and across industry domains. Such ecosystems demand smart CPSs that continuously align their architecture and governance to the concerns of various stakeholders, including developers, operators, and users. In order to satisfy short- and long-term stakeholder concerns in a continuously evolving operational context, this work proposes self-adaptive software models that promote DevOps for smart CPS. Our architectural approach extends to the embedded system layer and utilizes embedded and interconnected Digital Twins to manage change effectively. Experiments conducted on industrial embedded control units demonstrate the approach’s effectiveness in achieving sub-millisecond real-time closed-loop control of CPS assets and the simultaneous high-fidelity twinning (i.e., monitoring) of asset states. In addition, the experiments show practical support for the adaptation and evolution of CPS through the dynamic reconfiguring and updating of real-time control services and communication links without downtime. The evaluation results conclude that, in particular, the embedded Digital Twins can enhance CPS smartness by providing service-oriented access to CPS data, monitoring, adaptation, and control capabilities. Furthermore, the embedded Digital Twins can facilitate the seamless integration of these capabilities into current and future industrial service ecosystems. At the same time, these capabilities contribute to implementing emerging industrial services such as remote asset monitoring, commissioning, and maintenance.

Deep Neural Networks for Spatial-Temporal Cyber-Physical Systems: A Survey

Cyber-physical systems (CPS) refer to systems that integrate communication, control, and computational elements into physical processes to facilitate the control of physical systems and effective monitoring. The systems are designed to interact with the physical world, monitor and control the physical processes while in operation, and generate data. Deep Neural Networks (DNN) comprise multiple layers of interconnected neurons that process input data to produce predictions. Spatial-temporal data represents the physical world and its evolution over time and space. The generated spatial-temporal data is used to make decisions and control the behavior of CPS. This paper systematically reviews the applications of DNNs, namely convolutional, recurrent, and graphs, in handling spatial-temporal data in CPS. An extensive literature survey is conducted to determine the areas in which DNNs have successfully captured spatial-temporal data in CPS and the emerging areas that require attention. The research proposes a three-dimensional framework that considers: CPS (transportation, manufacturing, and others), Target (spatial-temporal data processing, anomaly detection, predictive maintenance, resource allocation, real-time decisions, and multi-modal data fusion), and DNN schemes (CNNs, RNNs, and GNNs). Finally, research areas that need further investigation are identified, such as performance and security. Addressing data quality, strict performance assurance, reliability, safety, and security resilience challenges are the areas that are required for further research.

Reinforced practical Byzantine fault tolerance consensus protocol for cyber physical systems

Cyber–physical systems (CPS) are becoming an essential component of modern life. CPS services enable information to be exchanged between physical devices and virtual systems. The increasing malicious activities causing confidential data leakage and incorrect performance of devices are posing challenges for protection against cyber threats. Therefore, development of effective solutions that can protect both CPS data and data exchange networks are extremely urgent. This study makes some improvements on Practical Byzantine Fault Tolerance (PBFT), and provides a critical analysis of the feasibility of using blockchain technology to protect constrained CPS data. It also justifies the choice of the improved PBFT for implementation on such devices and simulates the main distributed ledger scenarios. The improved PBFT works as follows: first, the client broadcasts the signed transaction data to the entire network, rather than just the master node, followed by a hash value comparison verification process; second, select the master node based on the longest chain principle, and punish the malicious node via the “blacklist” mechanism; third, add the data synchronization and verification process, as well as dynamically entering and leaving nodes via state transitions. Ultimately, the PBFT commit phase is eliminated, resulting in a two-stage process. In comparison to the original PBFT protocol, the Reinforced Practical Byzantine Fault Tolerance (RPBFT) protocol may substantially enhance system throughput and minimize consensus communication time, thereby improving overall system efficiency while guaranteeing security.

Robust Dynamic Latent Variable Model for Dynamic Process Monitoring With Missing Data in Cyber-Physical Systems

Robust Dynamic Latent Variable Model for Dynamic Process Monitoring With Missing Data in Cyber-Physical Systems

Challenges of Digital Twin Technologies Integration in Modular Construction: A Case from a Manufacturer’s Perspective

Automated models of physical objects are known as Digital twins; letting one to outline, test, array, and monitor and manage robotics in the real world. .CPS (Cyber-physical system) data have to be assembled through real life procedures to form a real-time monitoring cyber model in order to produce Digital Twin. Modification in the cyber model will be shown in real life system to guess or manage. As a result of digital and the progression in ICT, manufacturing and aviation or aerospace industries are now utilizing digital twin. Nonetheless, the uses of DT's in several production firms have not been researched massively. Those studies were sparse in construction, where structures are built without Superstructures, which thus sparked global concern. Herein, DT applications in building/manufacturing sector and in various firms were reviewed first and thereafter aimed on publications concerning DT applications in industry area by organizing a systematic search via Scopus. Notably, the publications were singled out immediately after the assessment of the publications and the study continues to investigate and evaluate the Potentials of digital twins in MIC, Restriction of digital twins in MIC, Impact of digital twins on industry, and Cost of time which should be appropriate for model development. The analysis report however demonstrated that DT is dedicated and thoughtful of in midst of inclusion along other digital technologies. More so, theoretical structure is formed in order to apply DT in module installation in MiC around the circumstances of Hong Kong which happens to be usual city case of high-density. Interestingly, the implementation of Digital twin in Modular Integrated Construction is expected to provide promising potential with significant benefits, such as improved logistics and manufacturing management by employing Digital Twins to track on-site progress during module installation.

Data-Correlation-Aware Unsupervised Deep-Learning Model for Anomaly Detection in Cyber–Physical Systems

A cyber–physical system (CPS) is a multidimensional complex system integrating computing, communication, and control technologies. Because of their key functionality within the system, CPS requires large robustness and security to ensure its reliable operation. Due to its importance in supporting overall system security, anomaly detection (AD) is likely to continue to play an important role in the CPS security. Moreover, unsupervised AD models based on deep learning have shown better performances in rule training, adaptive update, detection efficiency, and accuracy. Due to the nature of the CPS systems, CPS data is more likely to exhibit implicit correlative relationship among data, which would be vital to exploit for CPS security provisions in more complex data environments. In view of this observation, we propose the data-correlation-aware unsupervised deep learning model for AD in CPS, which uses an undigraph structure to store samples and implicit correlation among samples. We design a dual-autoencoder to train both original features and implicit correlation features among data, and we construct an estimation network using Gaussian mixture model (GMM) to evaluate the probability distribution of samples to complete the anomaly analysis. Experimental results compared the performance with relevant AD models based on deep learning which did not use data-correlation analysis. The results showed that, under some representative application scenarios of CPS considered, data-correlation-aware unsupervised deep-learning model achieved superior results in parameter sensitivity, ablation, relationship between correlation degree and detection performance, visualization, and detection effects.

Computational Intelligence in Security of Digital Twins Big Graphic Data in Cyber-physical Systems of Smart Cities

This investigation focuses on the application of computational intelligence to the security of Digital Twins (DTs) graphic data of the Cyber-physical System (CPS). The intricate and diverse physical space of CPS in the smart city is mapped in virtual space to construct the DTs CPS in the smart city. Besides, Differential Privacy Frequent Subgraph-Big Multigraph (DPFS-BM) is employed to ensure data privacy security. Moreover, the analysis and prediction model for the DTs big graphic data (BGD) in the CPS is built based on Differential Privacy-AlexNet (DP-AlexNet). Alexnet successfully solves the gradient dispersion problem of the Sigmoid function of deep network structures. Finally, the comparative analysis approach is utilized to verify the performance of the model reported here by comparing it with Long Short-Term Memory, Convolutional Neural Network, Recurrent Neural Network, original AlexNet, and Multi-Layer Perceptron in a simulation experiment. Through the comparison in the root mean square error, the mean absolute error, the mean absolute percentage error, training time, and test time, the model proposed here outperforms other models regarding errors, time delay, and time consumption. In the same environment, the system performs better with multi-hop paths, extra relays, and a high fading index; in that case, the outage probability is minimal. Therefore, the DP-AlexNet model is suitable for processing BGD. Moreover, its speed acceleration is more apparent than that of other models, with a higher SpeedUp indicator. The research effectively combines data mining and data security, which is of significant value for optimizing the privacy protection technology of frequent subgraph mining on a single multi-graph. Besides, the constructed DTs of CPS can provide excellent accuracy and a prominent acceleration effect on the premise of low errors. In addition, the model reported here can provide reference for the intelligent and digital development of smart cities.

Uncertainty-aware Prediction Validator in Deep Learning Models for Cyber-physical System Data

The use of Deep learning in Cyber-Physical Systems (CPSs) is gaining popularity due to its ability to bring intelligence to CPS behaviors. However, both CPSs and deep learning have inherent uncertainty. Such uncertainty, if not handled adequately, can lead to unsafe CPS behavior. The first step toward addressing such uncertainty in deep learning is to quantify uncertainty. Hence, we propose a novel method called NIRVANA (uNcertaInty pRediction ValidAtor iN Ai) for prediction validation based on uncertainty metrics. To this end, we first employ prediction-time Dropout-based Neural Networks to quantify uncertainty in deep learning models applied to CPS data. Second, such quantified uncertainty is taken as the input to predict wrong labels using a support vector machine, with the aim of building a highly discriminating prediction validator model with uncertainty values. In addition, we investigated the relationship between uncertainty quantification and prediction performance and conducted experiments to obtain optimal dropout ratios. We conducted all the experiments with four real-world CPS datasets. Results show that uncertainty quantification is negatively correlated to prediction performance of a deep learning model of CPS data. Also, our dropout ratio adjustment approach is effective in reducing uncertainty of correct predictions while increasing uncertainty of wrong predictions.

A Symmetric Scheme for Securing Data in Cyber-Physical Systems / IoT Sensor-Based Systems based on AES and SHA256

Cyber-Physical technologies and Internet have modernized and enhanced the lifestyle of people in this present era. Internet of Things (IoT) is also being seen as the new technology that is changing human discernment about daily life. Many cyber physical devices generate a lot of sensitive data, which needs to be secured. All the applications of CPS / IoT fundamentally, need symmetric keys for the encryption / decryption of the sensitive data generated. This paper seeks to propose a symmetric scheme for securing data in a Cyber-Physical system using IoT sensor-based technology, which will be based on AES and SHA256 encryption / decryption method.

Software Engineering and AI for Data Quality in Cyber- Physical Systems - SEA4DQ'21 Workshop Report

Cyber-physical systems (CPS) have been developed in many industrial sectors and application domains in which the quality requirements of data acquired are a common factor. Data quality in CPS can deteriorate because of several factors such as sensor faults and failures due to operating in harsh and uncertain environments. How can software engineering and artificial intelligence (AI) help manage and tame data quality issues in CPS? This is the question we aimed to investigate in the SEA4DQ workshop. Emerging trends in software engineering need to take data quality management seriously as CPS are increasingly datacentric in their approach to acquiring and processing data along the edge-fog-cloud continuum. This workshop provided researchers and practitioners a forum for exchanging ideas, experiences, understanding of the problems, visions for the future, and promising solutions to the problems in data quality in CPS. Examples of topics include software/hardware architectures and frameworks for data quality management in CPS; software engineering and AI to detect anomalies in CPS data or to repair erroneous CPS data. SEA4DQ 2021, which took place on August 24th, 2021 was a satellite event of the ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC / FSE) 2021. The workshop attracted 35 international participants and was exciting with a great keynote, six excellent presentations, and concluded on a high note with a panel discussion. SEA4DQ was motivated by the common research interests from the EU projects for Zero-Defects Manufacturing such as InterQ and Dat4.Zero.

A novel affine power flow method for improving accuracy of interval power flow data in cyber physical systems of active distribution network

A novel affine power flow method for improving accuracy of interval power flow data in cyber physical systems of active distribution network

METHOD OF DATA TRANSMISSION PROTECTION IN CYBERPHYSICAL SYSTEMS

Wireless networks are used to transmit data in cyberphysical systems. This creates an opportunity for attackers to intercept information and then decrypt it and use it for malicious purposes. Consequently, the need to increase the protection of transmitted data in cyberphysical systems increases. This is made possible by new approaches to ensuring the cryptographic stability of the wireless security system. The purpose of ensuring the protection of data transmission in a cyberphysical system is to create a method of encryption that even when intercepting data by decryption was not possible. After sending the test sequence, the attacker hopes to obtain the key sequence that was used during the encryption as a result of decrypting this sequence (reverse operation the sum of module 2 to the encrypted sequence). Having a code sequence and intercepting the following blocks of information, their decryption will not be difficult. Using the coding method for cyberphysical system data before encryption allows you to change the statistical characteristics of the input sequence of elements (message length, data volume, entropy, probability of occurrence of characters). These are the statistical characteristics that an attacker needs to decrypt data. Using encryption before encryption does not allow you to select the input test sequence and obtain the key. The statistical characteristics of the input data and the data obtained as a result of encoding (compression) before encryption were determined; it is established that the use of the proposed method allows to obtain data that are statistically independent of each other in the input and coded sequence; it is shown that the method of protection of data transmission in the cyberphysical system is working and allows to protect data of different volumes.

Guaranteeing Timely Response to Changes of Monitored Objects by Assigning Deadlines and Periods to Tasks

Timely response to changes of monitored objects is the key to ensuring the safety and reliability of cyber-physical systems (CPSs). There are two kinds of tasks in CPSs: update tasks and control tasks. Update tasks are responsible for updating the data in the system based on the state of the objects they monitor. Control tasks are responsible for making decisions based on the data in the system. The response time of the system to the change of a monitored object consists of two parts: the time taken by update tasks to reflect the change to the system, and the time taken by control tasks to make decisions according to the data in the system. Deadlines and periods of update tasks and control tasks directly affect the response time. Reasonable deadline and period assignment is the key to ensuring timely response to the changes of monitored objects. In this paper, we study the deadline and period assignment in CPSs. To the best of our knowledge, all existing work only focuses on the deadline and period assignment for update tasks with the goal of ensuring the freshness of the data in CPSs, and this is the first study focusing on the deadline and period assignment for both update tasks and control tasks with the goal of ensuring timely response to the changes of monitored objects. A new problem about response time control and system workload control is defined in this paper. Two deadline and period assignment methods are proposed to solve the defined problem. All the proposed methods can be used in the CPSs adopting the earliest deadline first (EDF) scheduling method. Experiments with randomly generated tasks are conducted to evaluate the performance of the proposed methods in terms of acceptance ratio and execution efficiency.

Continuous detection of concept drift in industrial cyber-physical systems using closed loop incremental machine learning

The embedded, computational and cloud elements of industrial cyber physical systems (CPS) generate large volumes of data at high velocity to support the operations and functions of corresponding time-critical and mission-critical physical entities. Given the non-deterministic nature of these entities, the generated data streams are susceptible to dynamic and abrupt changes. Such changes, which are formally defined as concept drifts, leads to a decline in the accuracy and robustness of predicted CPS behaviors. Most existing work in concept drift detection are classifier dependent and require labeled data. However, CPS data streams are unlabeled, unstructured and change over time. In this paper, we propose an unsupervised machine learning algorithm for continuous concept drift detection in industrial CPS. This algorithm demonstrates three types of unsupervised learning, online, incremental and decremental. Furthermore, it distinguishes between abrupt and reoccurring drifts. We conducted experiments on SEA, a widely cited synthetic dataset of concept drift detection, and two industrial applications of CPS, task tracking in factory settings and smart energy consumption. The results of these experiments successfully validate the key features of the proposed algorithm and its utility of detecting change in non-deterministic CPS environments.

A Federated Learning Approach to Frequent Itemset Mining in Cyber-Physical Systems

Effective vector representation has been proven useful for transaction classification and clustering tasks in Cyber-Physical Systems. Traditional methods use heuristic-based approaches and different pruning strategies to discover the required patterns efficiently. With the extensive and high dimensional availability of transactional data in cyber-physical systems, traditional methods that used frequent itemsets (FIs) as features suffer from dimensionality, sparsity, and privacy issues. In this paper, we first propose a federated learning-based embedding model for the transaction classification task. The model takes transaction data as a set of frequent item-sets. Afterward, the model can learn low dimensional continuous vectors by preserving the frequent item-sets contextual relationship. We perform an in-depth experimental analysis on the number of high dimensional transactional data to verify the developed models with attention-based mechanism and federated learning. From the results, it can be seen that the designed model can help and improve the decision boundary by reducing the global loss function while maintaining both security and privacy.

Optimal Stealthy Innovation-Based Attacks With Historical Data in Cyber-Physical Systems

The problem of stealthy innovation-based attacks in cyber-physical systems is studied in this paper. Different from the existing results which only utilize the current data, a more general attack strategy is designed by combining the historical and the current innovations to deteriorate the estimation performance and keep stealthy to the detector simultaneously. Under the framework of the attacks, the remote state estimation error is analyzed, and the optimal attack policy is derived by solving a convex optimization problem to achieve the maximal estimation error. Moreover, it is proved that the optimal attack strategy is piecewise constant, such that the attack is designed with low calculation cost. Finally, simulation examples are provided to demonstrate the theoretical results.

Generating Artificial Sensor Data for the Comparison of Unsupervised Machine Learning Methods

In the field of Cyber-Physical Systems (CPS), there is a large number of machine learning methods, and their intrinsic hyper-parameters are hugely varied. Since no agreed-on datasets for CPS exist, developers of new algorithms are forced to define their own benchmarks. This leads to a large number of algorithms each claiming benefits over other approaches but lacking a fair comparison. To tackle this problem, this paper defines a novel model for a generation process of data, similar to that found in CPS. The model is based on well-understood system theory and allows many datasets with different characteristics in terms of complexity to be generated. The data will pave the way for a comparison of selected machine learning methods in the exemplary field of unsupervised learning. Based on the synthetic CPS data, the data generation process is evaluated by analyzing the performance of the methods of the Self-Organizing Map, One-Class Support Vector Machine and Long Short-Term Memory Neural Net in anomaly detection.

Scalable Inverse Reinforcement Learning Through Multifidelity Bayesian Optimization.

Data in many practical problems are acquired according to decisions or actions made by users or experts to achieve specific goals. For instance, policies in the mind of biologists during the intervention process in genomics and metagenomics are often reflected in available data in these domains, or data in cyber-physical systems are often acquired according to actions/decisions made by experts/engineers for purposes, such as control or stabilization. Quantification of experts' policies through available data, which is also known as reward function learning, has been discussed extensively in the literature in the context of inverse reinforcement learning (IRL). However, most of the available techniques come short to deal with practical problems due to the following main reasons: 1) lack of scalability: arising from incapability or poor performance of existing techniques in dealing with large systems and 2) lack of reliability: coming from the incapability of the existing techniques to properly learn the optimal reward function during the learning process. Toward this, in this brief, we propose a multifidelity Bayesian optimization (MFBO) framework that significantly scales the learning process of a wide range of existing IRL techniques. The proposed framework enables the incorporation of multiple approximators and efficiently takes their uncertainty and computational costs into account to balance exploration and exploitation during the learning process. The proposed framework's high performance is demonstrated through genomics, metagenomics, and sets of random simulated problems.