Human Cyber-physical Systems Research Articles

Graph neural network-based anomaly detection for human cyber physical systems

ABSTRACT Human Cyber Physical Systems (HCPS) encompass humans, networks, and physical devices, characterized by intricate interactions and interdependencies. The complexity of HCPS networks makes them vulnerable to network issues and cyberattacks, potentially resulting in network paralysis and significantly impacting the reliable operation of HCPS. Therefore, precise anomaly detection is imperative for HCPS. To tackle this issue, we integrate HCPS with the Software Defined Networking (SDN) infrastructure layer and propose an anomaly detection strategy based on Graph Neural Networks (GNN). Our approach utilizes Graph Sample and Aggregates (GraphSAGE) to capture adjacent feature information of nodes, revealing hidden relationships within the graph. These embeddings are then combined with Graph Attention Networks (GAT), which calculate attention coefficients between a node and its neighboring nodes, representing the importance of neighboring nodes to the central node. Finally, a fully connected layer is employed to obtain anomaly node labels. To assess the performance of our method, we conduct experiments using real datasets and compare them with other advanced anomaly detection methods in terms of accuracy, precision, recall, F1 score, and area under the ROC curve (AUC). Our method accurately detects anomalous nodes, offering a viable solution for practical applications.

The rCOS framework for multi-dimensional separation of concerns in model-driven engineering

The software industry increasingly turns to Model-Driven Engineering (MDE) to mitigate complexity by automating model creation and transformation. Many organisations are pursuing Integrated Development Platforms (IDPs) to enhance automation in their software development processes within MDE. However, the adoption of MDE and engagement with IDPs remain limited due to concerns over their efficacy. We address these challenges in this review paper by introducing a framework for the formal refinement of component and object systems (rCOS). It provides: (1) a formal theory that consists of a modelling language (named OPL) with a calculus of refinement for object-oriented models and component models; (2) a suite of analysis and design techniques that facilitate abstractions and decompositions, leading to a multidimensional separation of concerns; and (3) an IDP (named rCOS Modeller) that supports modelling, design and verification from requirements elicitation through to coding. By advocating for an rCOS-enabled multidimensional approach to separating concerns, this paper offers a comprehensive solution to the challenges facing MDE and IDPs, paving the way for their successful implementation in practice. By delineating the emerging challenges and prospects associated with integrating formal methods for modelling and designing human-cyber–physical systems (HCPS), we show the potential of extending rCOS for MDE in HCPS.

A multisensory Interaction Framework for Human-Cyber–Physical System based on Graph Convolutional Networks

Human-Cyber-Physical Systems (HCPS), as an emerging paradigm centered around humans, provide a promising direction for the advancement of various domains, such as intelligent manufacturing and aerospace. In contrast to Cyber-Physical Systems (CPS), the development of HCPS emphasizes the expansion of human capabilities. Humans no longer solely function as operators or agents working in collaboration with computers and machines but extend their roles to include system design and innovation management. This paper proposes a Multisensory Interaction Framework for HCPS (MS-HCPS) that leverages human senses to facilitate system creation and management. Additionally, the introduced Multisensory Graph Convolutional Network (MS-GCN) model calculates recommendation values for multiple senses, elucidating their relevance to system development. Furthermore, the effectiveness of the proposed framework and model is validated through three practical engineering scenarios. This study explores the research on multisensory interaction in HCPS from a human sensory perspective, aiming to facilitate the progress and development of HCPS across various domains.

Enhancing operator health and safety in manufacturing: an intelligent digital humanization approach

The manufacturing industry relies heavily on human labour, making the health and safety of operators paramount. Emerging technologies such as Human-Cyber-Physical Systems (HCPS) and Machine Learning (ML) have the potential to transform the approach to these critical issues. In fact, such technologies enable the creation of virtual replicas of tangible systems, providing innovative solutions for assessing operator safety. In this context, this work presents a digital humanization methodology designed to comprehensively evaluate the health and safety risks associated with operator involvement in production processes. Using cutting-edge sensors, advanced machine learning algorithms, and scenario simulations, such methodology generates real-time virtual representations of the physical condition and behaviour of the operator. These representations allow the risk characterization by estimating magnitude, priority, and occurrence, thereby facilitating early detection and preventive measures against potential hazards. A case study is considered to demonstrate the practical application of the proposed framework.

Autoencoder-Embedded Iterated Local Search for Energy-Minimized Task Schedules of Human–Cyber–Physical Systems

This work considers a task scheduling problem with deadline constraints in human-cyber-physical systems. To find its energy-efficient schedules in a short time, an autoencoder-embedded iterated local search algorithm is proposed to solve it. Iterated local search is selected as a main scheduler. In order to handle real-time requirements and high computational load involved in the problem solution, a Long Short-Term Memory-based AutoEncoder model (LSTM-AE) is constructed to capture the relevant and complementary features of the considered problem. The model is used to find low-order and high-fit sub-solutions via unsupervised end-to-end learning, and generates promising solutions in an informative low-dimensional solution space. To further reduce computational burden, a two-stage optimization framework is constructed, which includes an off-line training phase and an online optimization one. The former trains LSTM-AE by using expert knowledge and historical data. The latter designs optimal resource allocation strategies to build a high-quality initial solution. Then, LSTM-AE-assisted local search operators are proposed and used to reform the initial solution and generate better ones. Various numerical experiments are performed to compare the proposed method with several classic heuristics and some recently-developed methods. The results show its superiority over them. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —In human-cyber-physical systems, a task scheduling problem is usually solved by using heuristics due to the limited computational resources. Nevertheless, fast dispatching rules tend to perform poorly. Meta-heuristics can find a relatively high-quality schedule but are time-consuming, especially for a population-based algorithm that requires to evaluate a fitness function for many candidate solutions at each iteration. To balance computational burden and solution quality, our idea is to combine machine-learning methods with meta-heuristics. Specially, we integrate a long short-term memory-based autoencoder model into iterated local search to improve the latter’s optimization ability. The combination of meta-heuristics and machine learning techniques makes it possible to obtain a high-quality schedule for the concerned problems in a short time. Theoretic analysis and experimental results show that the proposed method well outperforms its competitive peers.

A REFERENCE ARCHITECTURE OF HUMAN CYBER-PHYSICAL SYSTEMS – PART III: SEMANTIC FOUNDATIONS

The design and analysis of multi-agent human cyber-physical systems in safety-critical or industry-critical domains calls for an adequate semantic foundation capable of exhaustively and rigorously describing all emergent effects in the joint dynamic behavior of the agents that are relevant to their safety and well-behavior. We present such a semantic foundation. This framework extends beyond previous approaches by extending the agent-local dynamic state beyond state components under direct control of the agent and belief about other agents (as previously suggested for understanding cooperative as well as rational behavior) to agent-local evidence and belief about the overall cooperative, competitive, or coopetitive game structure. We argue that this extension is necessary for rigorously analyzing systems of human cyber-physical systems because humans are known to employ cognitive replacement models of system dynamics that are both non-stationary and potentially incongruent. These replacement models induce visible and potentially harmful effects on their joint emergent behavior and the interaction with cyber-physical system components.

A References Architecture for Human Cyber Physical Systems - PART II: Fundamental Design Principles for Human-CPS Interaction

As automation increases qualitatively and quantitatively in safety-critical human cyber-physical systems, it is becoming more and more challenging to increase the probability or ensure that human operators still perceive key artefacts and comprehend their roles in the system. In the companion paper, we proposed an abstract reference architecture capable of expressing all classes of system-level interactions in human cyber-physical systems. Here we demonstrate how this reference architecture supports the analysis of levels of communication between agents and helps to identify the potential for misunderstandings and misconceptions. We then develop a metamodel for safe human machine interaction. Therefore, we ask what type of information exchange must be supported on what level so that humans and systems can cooperate as a team, what is the criticality of exchanged information, what are timing requirements for such interactions, and how can we communicate highly critical information in a limited time frame in spite of the many sources of a distorted perception. We highlight shared stumbling blocks and illustrate shared design principles, which rest on established ontologies specific to particular application classes. In order to overcome the partial opacity of internal states of agents, we anticipate a key role of virtual twins of both human and technical cooperation partners for designing a suitable communication.

A Reference Architecture of Human Cyber-Physical Systems – PART I: Fundamental Concepts

We propose a reference architecture of safety-critical or industry-critical human cyber-physical systems (CPSs) capable of expressing essential classes of system-level interactions between CPS and humans relevant for the societal acceptance of such systems. To reach this quality gate, the expressivity of the model must go beyond classical viewpoints such as operational, functional, and architectural views and views used for safety and security analysis. The model does so by incorporating elements of such systems for mutual introspections in situational awareness, capabilities, and intentions in order to enable a synergetic, trusted relation in the interaction of humans and CPSs, which we see as a prerequisite for their societal acceptance. The reference architecture is represented as a metamodel incorporating conceptual and behavioral semantic aspects. We illustrate the key concepts of the metamodel with examples from cooperative autonomous driving, the operating room of the future, cockpit-tower interaction, and crisis management.

Towards a model of human-cyber–physical automata and a synthesis framework for control policies

Advances in research and increasing applications of Cyber–Physical Systems (CPSs) show the need to consider factors of humans in the loop. This has led to the growing research focus on Human-Cyber–Physical Systems (HCPSs). In general, humans in an HCPS interact with both the cyber and physical systems, as well as among the humans themselves. For a better understanding, correct design, development, operation, and maintenance of HCPSs, a computational theory based on a computational model is required. This paper presents our initial work towards a model of human-cyber–physical automata (HCPA). We consider an HCPS as a combination of a human-physical system (HPS) and a CPS in which the control switches between the humans and the machines. We define an HCPA by connecting the automaton of the HPS and the automaton of the CPS through a switch control automaton. The switch control automaton makes switching decision in some critical states shared by the HPS and the CPS. Our theorem shows that the control switching between the HPS and the CPS increases the probability of satisfying a given property. We model the behaviour of a human in specified applications or even in carry out specific tasks, instead of general human intelligence. Therefore, a human can make mistakes to decision making and thus it is a probabilistic automaton with learning ability. The switching between the human and the machine is modelled by an oracle. The oracle learns about the human behaviour, the machine behaviour, as well as the environment to make the control decisions. To generate the control policies of the human and the oracle, we propose a synthesis framework to maximize the probability of the satisfaction of a property specified in Linear Temporal Logic (LTL) by the HCPA. We present a prototype implementation of the framework by extending the model-free reinforcement learning (RL) algorithm and model-free deep-RL algorithm, and our experiment shows that our synthesis framework is effective in obtaining switch policies.

A review of design frameworks for human‐cyber‐physical systems moving from industry 4 to 5

AbstractWithin the Industry 4.0 landscape, humans collaborate with cyber and physical elements to form human‐cyber‐physical systems (HCPS). These environments are increasingly complex and challenging workspaces due to increasing levels of automation and data availability. An effective system design requires suitable frameworks that consider human activities and needs whilst supporting overall system efficacy. Although several reviews of frameworks for technology were identified, none of these focused on the human in the system (moving towards Industry 5). The critical literature review presented provides a summary of HCPS frameworks, maps the considerations for a human in HCPS, and provides insight for future framework and system development. The challenges, recommendations, and areas for further research are discussed.

Data Augmentation-Based Manufacturing Quality Prediction Approach in Human Cyber-Physical Systems

Abstract The vigorous development of the human cyber-physical system (HCPS) and the next generation of artificial intelligence provide new ideas for smart manufacturing, where manufacturing quality prediction is an important issue in the manufacturing system. However, the small-scale data from humans in emerging HCPS-enabled manufacturing restrict the development of traditional quality prediction methods. To address this question, a data augmentation-based manufacturing quality prediction approach in human cyber-physical systems is proposed in this paper. Specifically, a Data Augmentation-Gradient Boosting Decision Tree (DA-GBDT) model is developed for quality prediction under the HCPS context. In addition, an adaptive selection algorithm of data augmentation rate is designed to balance the trade-off between the training time of the prediction model and the prediction accuracy. Finally, the experimental results of automobile covering products demonstrate that the proposed method can improve the average prediction error of the model compared with the prevailing quality prediction methods. Moreover, the predicted quality information can provide guidance for product optimization decisions in smart manufacturing systems.

Cyber Warfare and Cyber Terrorism Threats Targeting Critical Infrastructure: A HCPS-based Threat Modelling Intelligence Framework

Acts of cyber warfare and cyber terrorism (CWCT) that target a nation's critical infrastructure (CI) are quickly becoming a larger threat to national security than conventional kinetic warfare strategies. Adversaries or potential adversaries can target a nation's electrical grids, telecommunications, financial services, transportation, healthcare systems, and other forms of CI. These acts pose a major threat to a nation's CI and consequently exposes citizens to public health, safety, security, and economic development risks. Identifying cyber vulnerabilities and threats can help nations to improve their CI defence strategies. There is a crucial need for research that can aid in understanding the major types of CI threats and by what method they might occur. This paper conducts a systematic literature review to develop an initial threat intelligence framework of CWCT attacks on CI. Drawing from a Human–Cyber–Physical Systems (HCPS) lens, the threat intelligence framework classifies CWCT attacks according to the methods, weapons, vulnerabilities, targets and impact of the CWCT attack. The cyber warfare community can extend the proposed HCPS-based threat intelligence framework to develop more advanced cyber security mitigation strategies, training scenarios and simulations. Large-scale monitoring of CI threats requires in-depth threat intelligence analysis and a collaborative defence strategy. This calls for a higher degree of coordination and orchestration between the military, intelligence agencies, government departments, multinational allies, regulators, and commercial entities. Future research can customize the proposed HCPS-based threat intelligence framework to cater for the unique threats facing specific CI domains.

A human cyber-physical system for human-centered computing in seafaring

Human cyber-physical systems present an appealing platform for facilitating human-centered computing, particularly in environments where humans are required to interact with systems comprised of cyber and physical layers. Universally accepted elements of a human cyber-physical system, and interactions among these elements, do not appear to emerge in literature. To address this, a human cyber-physical system architecture is offered, led by the seafaring focus of this study. A human cyber-physical system implementation, called Mariner 4.0, for monitoring motion sickness of seafarers is presented to demonstrate the operation of a human cyber-physical system as a platform for human-centered computing. The human cyber-physical system implementation enabled the digital representation of the state of motion sickness and location of a seafarer, and conveyance of related information through a state monitor display. Using data captured on board South Africa’s polar supply and research vessel, the SA Agulhas II, results showed that the human cyber-physical system incorporated location-specific variation in motion exposure experienced between two seafarers in motion sickness computations through facilitating individualized data acquisition and integration.

Intelligent Management of Enterprise Business Processes

The article develops the conceptual foundations of natural and artificial intellectualization of the enterprise, as well as the combination of artificial and natural intelligence in managing business processes in the context of modern challenges of the business environment. Based on the methods of structural design, a system model of the enterprise is developed as the basis for intelligent management. Quantitative and qualitative effects of human–cyber–physical systems, which are the result of management intellectualization, are highlighted. The possibilities of using deviation and perturbation management methods in managing the state of enterprise development with the support of decision-making and implementation of an intelligent information system are considered. The features of making managerial decisions during intelligent enterprise management are considered. The place of the human factor in such intellectual management is highlighted, in particular, in terms of improving the intelligence of employees and the natural intellectualization of the enterprise. The problem of assessing and forecasting the state of enterprise development in the context of intellectual management is highlighted. In this context, the expediency of using mathematical methods of Markov process theory, using systems of Kolmogorov differential equations and their solutions, using numerical methods and applied software products is justified. This made it possible to study the dynamics of probabilities of states and stability of development of enterprises and their employees; the dynamics of probabilities of states of innovative and technological processes; scientific and technological, environmental, social and economic efficiency of a business. To test the proposed mathematical models for assessing and predicting the state of development of enterprises and their employees, appropriate studies were conducted on the sanatorium–resort complex in Truskavets (Ukraine).

Human-machine collaborative additive manufacturing

Recent advances in additive manufacturing have transformed the technology from a rapid prototyping tool into a viable production option. Within such transition, the relationships between human-machine in both cyber and physical spaces of additive manufacturing become diverse and complex. It is of great significance to gain insight and build a clear understanding of these relationships in the context of human-cyber-physical systems (HCPS) to facilitate the performance of the human-machine collaboration in various application scenarios of additive manufacturing. The paper utilizes the existing HCPS reference models to examines the emerging research field of human-machine collaborative additive manufacturing, focusing on typical scenarios, definitions, and classifications. The collaboration activities are divided into active-supportive and active-active types. For the active-supportive type, intelligent additive manufacturing and human augmentation are detailed. Also, human-machine co-creation, as an active-active type, is covered. The paper concludes with a discussion of theoretical and practical implications and opportunities for future research.

Towards intelligent welding systems from a HCPS perspective: A technology framework and implementation roadmap

A framework and implementation roadmap for an intelligent welding system (IWS) is proposed from the human-cyber-physical systems (HCPS) perspective of integrating cyber systems with humans and physical systems. Key technologies and system requirements for the framework are comprehensively analyzed. A technology network is proposed to explore the link between complex IWS technologies and application domain requirements. To investigate the practical extent of the state-of-the-art, an example enterprise illustrates the technologies necessary to implement an IWS and the representative technological gaps. Technologies required for IWS were evaluated for their maturity, benefit, and market availability using the Gartner hype-cycle and priority matrix methodology. Finally, an implementation roadmap with research suggestions towards IWS is recommended. This work can support enterprises or factories that want to implement intelligent welding into their operations by clarifying the development state of intelligent technologies and the priority by which further development should be pursued.

A Verifiable Privacy-Preserving Machine Learning Prediction Scheme for Edge-Enhanced HCPSs

As a highly integrated industrial system, human cyber-physical systems (HCPSs) provide accurate and high-quality services for Industry 5.0. In HCPSs, machine learning (ML) prediction provides reliable prediction results for users based on matured models, while security and privacy protection are considerable issues. In this article, based on the modified Okamoto–Uchiyama homomorphic encryption, we propose a verifiable privacy-preserving machine learning prediction scheme for the edge-enhanced HCPSs, which outputs the verifiable prediction results for users without privacy leakage. Specifically, a batch of prediction results can be verified at one time, which improves the efficiency of verification. Security analysis shows that our scheme protects the privacy of inputs, ML model, and prediction results. The experiment results demonstrate that the edge computing architecture remarkably alleviates the computational burden of the cloud server. Furthermore, compared with other related schemes, our scheme shows the best execution efficiency, and batch verification optimizes the performance by about 15% compared with single verification on the same scale.

Visible Light Communication and localization: A study on tracking solutions for Industry 4.0 and the Operator 4.0

The article presents a study on tracking solutions based on localization and telecommunication technologies in Industry 4.0 and Industrial Internet of Things (IIoT) environments oriented on Operator 4.0. Nowadays, the smart industry concept is mainly a technology-driven solution, which is slowly shifting towards a value-driven approach. This value is based on human-centricity, sustainability, and resilience. In this article, we take a look at human-centricity. Important breakthroughs in communication and localization technologies enabled the deployment of multiple digital, networking or smart technologies in an industrial environment. Smart manufacturing systems are typically based on network-enabled smart sensors/actuators, data acquisition and analytics, and intelligent computational algorithms via human-cyber–physical systems that rely on the Industrial Internet of Things (IIoT). Moreover, interactions between humans and intelligent machines in smart factories or enterprises are also an important part of next-gen manufacturing processes. Wireless telecommunication and localization technologies are still expanding, but their deployment can have significant issues in industrial complexes due to electromagnetic interference. This interference can directly affect production or communication devices. This disadvantage can be solved by the Visible Light Communication (VLC) technology described in this article. Indoor localization and tracking are very important for human-centric technologies like augmented reality, virtual reality, cooperative robotics, and others. These technologies, which help operators with human–machine interaction and human-in-the-loop control, need precise and fast localization. Visible Light Positioning (VLP) could provide the answer.

Evaluation of 5G-capable framework for highly mobile, scalable human-machine interfaces in cyber-physical production systems

Human cyber-physical production systems (HCPS) - as an extension of cyber-physical production systems (CPPS) - focus on the human being in the system and the development of socio-technical systems. Humans and therefore anthropogenic behavior have to be integrated into the manufacturing system and its processes. Due to the shift towards customized products, CPPS require high reconfigurability, flexibility and individual manufacturing processes. As a result, the role of and requirements for humans are fundamentally changing, creating the necessity for new interfaces to interact with machines within reconfigurable manufacturing process. Scalable human machine interfaces (HMI) are needed that incorporate emerging technologies as well as allowing mobility for the operator while interacting with different machines. Therefore, new approaches for HMI in the context of CPPS and HCPS are needed, which are simultaneously sufficiently mobile, scalable, and modular as well as human centered. High mobility of HMIs can be ensured by using the 5G communication standard that enables wireless migration of computational resources to an edge server with high reliability, low latency, and high data rates. This paper develops, implements, and evaluates a 5G-based, framework for highly mobile, scalable HMI in CPPS by utilizing the new 5G communication technology. The capabilities of the framework and of the new communication standard are demonstrated and evaluated for a use case, where a brain-computer interface (BCI) is used to control a robot arm. For better accuracy, the BCI is supported by an eye tracker and visual feedback is received via an augmented reality environment, and all devices are embedded via 5G communication. In particular, the influence of 5G communication on the system performance is examined, evaluated, and discussed. For this purpose, experiments are designed and conducted with different network configurations.

Toward human-centric smart manufacturing: A human-cyber-physical systems (HCPS) perspective

Advances in human-centric smart manufacturing (HSM) reflect a trend towards the integration of human-in-the-loop with technologies, to address challenges of human-machine relationships. In this context, the human-cyber-physical systems (HCPS), as an emerging human-centric system paradigm, can bring insights to the development and implementation of HSM. This study presents a systematic review of HCPS theories and technologies on HSM with a focus on the human-aspect is conducted. First, the concepts, key components, and taxonomy of HCPS are discussed. HCPS system framework and subsystems are analyzed. Enabling technologies (e.g., domain technologies, unit-level technologies, and system-level technologies) and core features (e.g., connectivity, integration, intelligence, adaptation, and socialization) of HCPS are presented. Applications of HCPS in smart manufacturing are illustrated with the human in the design, production, and service perspectives. This research offers key knowledge and a reference model for the human-centric design, evaluation, and implementation of HCPS-based HSM.