Framework for a closed-loop cooperative human Cyber-Physical System for the mining industry driven by VR and AR: MHCPS

Abstract

Underground mining systems face numerous challenges, such as limited intelligence equipment, lack of accurate maps, difficult maintenance, and poor integration between humans and equipment. Here, based on the basic architecture of CPS (Cyber-Physical Systems), a closed-loop collaborative human CPS operational system for the mining industry is proposed (MHCPS). The MHCPS uses an information system based on VR (Virtual Reality) and AR (Augmented Reality) which serve to integrate humans with intelligent physical systems, the overall mining process, and closed-loop circulation of computing, communication, and control (3C). The 3C operation allowed centralized control center operators and inspection operators to interact with physical equipment and jointly complete tasks. Communication in the system is made dependable by a two-way information flow network. To ease computing load, a distributed computing mode integrating human and edge computing is proposed. In the system, fused VR simulation data, AR point cloud information, and sensor information are accurately returned to the human–machine interface in real-time. Based on complex computations, an operation method that integrates traditional operations with a VR/AR human–machine interface resolved the conflicts involved when fusing multiple interfaces. Finally, the prototype system was developed and assessed in the laboratory. The experimental results showed that the two-way closed-loop communications transmission between human, VR system, AR system, and physical system were stable. The computations based on imaging, sensing, and simulation information accurately reconstructed a precise virtual image of the physical equipment. The VR/AR system allowed for reliable control while also facilitating remote interactions between human operators and the other three systems, effectively optimizing the physical system. This system demonstrated the feasibility of a CPS system under complex working conditions dominated by humans. It also improved on synchronizing manual controls and machine operations, which maintained efficient cooperation.