Mixed Reality Devices Research Articles

Preoperative mixed reality training improves trainee performance of glenoid guidewire positioning in shoulder arthroplasty in Walch B2 glenoid model

BackgroundVirtual reality training has demonstrated the capacity to improve resident performance with certain surgical skills. Mixed reality (MR) devices such as the Hololens 2 could theoretically be used in a similar way for medical education and skill acquisition. Our hypothesis is that a patient specific preprocedure MR training program for 10-15 minutes followed by glenoid guidewire placement with MR holographic assistance can improve guide pin placement enough to be significantly better than freehand (FH) guidewire placement. MethodsThirty glenoid models with a Walch B2 glenoid defect were 3D printed using ABS filament. Resident trainees were selected and randomized to FH guidewire placement or MR training cohorts that were matched by experience. An app was created and installed on the Hololens 2 to allow practice of glenoid guidewire placement in a pseudo-immersive environment and comparison to the desired plan. This was used for 10-15 minutes prior to definitive placement of the glenoid guidewire in the B2 models by the resident in the MR cohort and a 3D hologram was available during guidewire placement. The volunteers each placed 10 guidewires in B2 glenoid models. The models were 3D scanned back to 3D digital files. The digital models were aligned using the automated iterative closest point method. Measurements were taken in each axis to determine the deviation from the surgical plan in version and inclination in degrees and start point in mm. ResultsAverage starting point difference from the plan for the FH group was 2.52 ± 0.82 mm, and 1.49 ± 0 .71 mm for the MR group, P = .02. The absolute value average version difference from the plan for the FH group was 11.69 ± 8.02 degrees, and 5.3 ± 3.57 degrees for the MR group, P ≤ .0001. The absolute value average inclination difference from the plan for the FH group was 8.4 ± 6.68 degrees, and 7.11 ± 4.96 degrees for the MR group, P = .27. ConclusionPreprocedure MR training improves execution of the surgical plan for guidewire version and start point in a B2 glenoid model compared to FH.

Wear-Resistant Smart Textiles Using Nylon-11 Triboelectric Yarns.

The ever-increasing demand for self-powered systems such as glucose biosensors and mixed reality devices has sparked significant interest in triboelectric generators, which hold large potential as renewable energy solutions. Our study explores new methods for integrating energy-harvesting capabilities into smart textiles by developing strong and efficient yarns that can convert mechanical energy into electrical energy through a triboelectric effect. Specifically, we focused on Nylon-11 (PA11), a material known for its crystalline structure well-suited for generating a powerful triboelectric response. To achieve this, we created triboelectric yarns by electrospinning PA11 fibers onto conductive carbon yarns, enabling energy-harvesting applications. Extensive testing demonstrated that these yarns possess exceptional durability, surpassing real-life usage requirements while experiencing minimal degradation. Additionally, we developed a prototype haptic device by interweaving tribopositive PA11 and tribonegative poly(vinylidene fluoride) (PVDF) triboelectric yarns. Our research has successfully yielded durable and efficient yarns with strong energy-harvesting capabilities, opening up possibilities for integrating smart textiles into practical scenarios. These technologies are promising steps to achieve greener and more reliable self-powered systems.

Building Your Future Holographic Mentor: Can We Use Mixed Reality Holograms for Visual Spatial Motor Skills Acquisition in Surgical Education?

Learning surgical skills require critical visual-spatial motor skills. Current learning methods employ costly and limited in-person teaching in addition to supplementation by videos, textbooks, and cadaveric labs. Increasingly limited healthcare resources and in-person training has led to growing concerns for skills acquisition of trainees. Recent Mixed Reality (MR) devices offer an attractive solution to these resource barriers by providing three-dimensional holographic representations of reality that mimic in-person experiences in a portable, individualized, and cost-effective form. We developed and evaluated two holographic MR models to explore the feasibility of visual-spatial motor skill acquisition from a technical development, learning, and usability perspective. In our first, a pair of holographic hands were created and projected in front of the trainee, and participants were evaluated on their ability to learn complex hand motions in comparison to traditional methods of video and apprenticeship-based learning. The second model displayed a 3D holographic model of the middle and inner ear with labeled anatomical structures which users could explore and user experience feedback was obtained. Our studies demonstrated that scores between MR and apprenticeship learning were comparable. All felt MR was an effective learning tool and most noted that the MR models were better than existing didactic methods of learning. Identified advantages of MR included the ability to provide true 3D spatial representation, improved visualization of smaller structures in detail by upscaling the models, and improved interactivity. Our results demonstrate that holographic learning is able to mimic in-person learning for visual-spatial motor skills and could be a new effective form of self-directed apprenticeship learning.

Human Operation Augmentation through Wearable Robotic Limb Integrated with Mixed Reality Device.

Mixed reality technology can give humans an intuitive visual experience, and combined with the multi-source information of the human body, it can provide a comfortable human-robot interaction experience. This paper applies a mixed reality device (Hololens2) to provide interactive communication between the wearer and the wearable robotic limb (supernumerary robotic limb, SRL). Hololens2 can obtain human body information, including eye gaze, hand gestures, voice input, etc. It can also provide feedback information to the wearer through augmented reality and audio output, which is the communication bridge needed in human-robot interaction. Implementing a wearable robotic arm integrated with HoloLens2 is proposed to augment the wearer's capabilities. Taking two typical practical tasks of cable installation and electrical connector soldering in aircraft manufacturing as examples, the task models and interaction scheme are designed. Finally, human augmentation is evaluated in terms of task completion time statistics.

Mixed reality drills of indoor earthquake safety considering seismic damage of nonstructural components

The damaged indoor nonstructural components in the earthquake often cause casualties. To improve the indoor earthquake safety capacity of occupants, a mixed reality (MR) drill method for indoor earthquake safety considering seismic damage of nonstructural components is proposed. First, an MR device, HoloLens, is used to capture indoor point clouds, and the indoor three-dimensional scene is reconstructed using point clouds. Subsequently, the seismic motion models of indoor components are established, so that the indoor nonstructural seismic damage scene is constructed using the physics engine and displayed using HoloLens. Finally, a guidance algorithm for a safe zone was designed for the drills. Taking a typical office as an example, an indoor earthquake safety drill was performed. The drill results show that the proposed MR method can increase the average efficiency of moving to a safe zone by 43.1%. Therefore, the outcome of this study can effectively improve the earthquake safety ability of occupants, thereby reducing casualties.

Application effect of head-mounted mixed reality device combined with 3D printing model in neurosurgery ventricular and hematoma puncture training

BackgroundThe purpose of this study was to explore the applicability of application effect of head-mounted mixed reality (MR) equipment combined with a three-dimensional (3D) printed model in neurosurgical ventricular and haematoma puncture training.MethodsDigital Imaging and Communications in Medicine (DICOM) format image data of two patients with common neurosurgical diseases (hydrocephalus and basal ganglia haemorrhage) were imported into 3D Slicer software for 3D reconstruction, saved, and printed using 3D printing to produce a 1:1-sized head model with real person characteristics. The required model (brain ventricle, haematoma, puncture path, etc.) was constructed and imported into the head-mounted MR device, HoloLens, and a risk-free, visual, and repeatable system was designed for the training of junior physicians. A total of 16 junior physicians who studied under this specialty from September 2020 to March 2022 were selected as the research participants, and the applicability of the equipment and model during training was evaluated with assessment score sheets and questionnaires after training.ResultsAccording to results of the assessment and questionnaire, the doctors trained by this system are more familiar with the localization of the lateral anterior ventricle horn puncture and the common endoscopic surgery for basal ganglia haemorrhage, as well as more confident in the mastery of these two operations than the traditional training methods.ConclusionsThe use of head-mounted MR equipment combined with 3D printing models can provide an ideal platform for the operation training of young doctors. Through holographic images created from the combination of virtual and real images, operators can be better immersed in the operation process and deepen their understanding of the operation and related anatomical structures. The 3D printed model can be repeatedly reproduced so that doctors can master the technology, learn from mistakes, better achieve the purpose of teaching and training, and improve the effect of training.

Integrated system architecture with mixed-reality user interface for virtual-physical hybrid swarm simulations

This paper introduces a hybrid robotic swarm system architecture that combines virtual and physical components and enables human–swarm interaction through mixed reality (MR) devices. The system comprises three main modules: (1) the virtual module, which simulates robotic agents, (2) the physical module, consisting of real robotic agents, and (3) the user interface (UI) module. To facilitate communication between the modules, the UI module connects with the virtual module using Photon Network and with the physical module through the Robot Operating System (ROS) bridge. Additionally, the virtual and physical modules communicate via the ROS bridge. The virtual and physical agents form a hybrid swarm by integrating these three modules. The human–swarm interface based on MR technology enables one or multiple human users to interact with the swarm in various ways. Users can create and assign tasks, monitor real-time swarm status and activities, or control and interact with specific robotic agents. To validate the system-level integration and embedded swarm functions, two experimental demonstrations were conducted: (a) two users playing planner and observer roles, assigning five tasks for the swarm to allocate the tasks autonomously and execute them, and (b) a single user interacting with the hybrid swarm consisting of two physical agents and 170 virtual agents by creating and assigning a task list and then controlling one of the physical robots to complete a target identification mission.

Effect of Mixed Reality on Delivery of Emergency Medical Care in a Simulated Environment

Mixed-reality (MR) technology has the potential to enhance care delivery, but there remains a paucity of evidence for its efficacy and feasibility. To assess the efficacy and feasibility of MR technology to enhance emergency care delivery in a simulated environment. This pilot randomized crossover trial was conducted from September to November 2021 at a single center in a high-fidelity simulated environment with participants block randomized to standard care (SC) or MR-supported care (MR-SC) groups. Participants were 22 resident-grade physicians working in acute medical and surgical specialties prospectively recruited from a single UK Academic Health Sciences Centre. Data were analyzed from September to December 2022. Participants resuscitated a simulated patient who was acutely unwell, including undertaking invasive procedures. Participants completed 2 scenarios and were randomly assigned to SC or MR-SC for the first scenario prior to crossover. The HoloLens 2 MR device provided interactive holographic content and bidirectional audiovisual communication with senior physicians in the MR-SC group. The primary outcome was error rate assessed via the Imperial College Error Capture (ICECAP) multidimensional error-capture tool. Secondary outcomes included teamwork (Observational Teamwork Assessment for Surgery [OTAS]; range, 0-6 and Teamwork Skills Assessment for Ward Care [T-SAW-C]; range, 1-5), scenario completion, stress and cognitive load (NASA Task Load Index [NASA-TLX; range 0-100]), and MR device user acceptability. A total of 22 physicians (15 males [68.2%]; median [range] age, 28 [25-34] years) were recruited. MR technology significantly reduced the mean (SD) number of errors per scenario compared with SC (5.16 [3.34] vs 8.30 [3.09] errors; P = .003), with substantial reductions in procedural (0.79 [0.75] vs 1.52 [1.20] errors; P = .02), technical (1.95 [1.40] vs 3.65 [2.03] errors; P = .01), and safety (0.37 [0.96] vs 0.96 [0.85] errors; P = .04) domains. MR resulted in significantly greater scenario completion rates vs SC (22 scenarios [100%] vs 14 scenarios [63.6%]; P = .003). It also led to significant improvements in the overall quality of teamwork and interactions vs SC as measured by mean (SD) OTAS (25.41 [6.30] vs 16.33 [5.49]; P < .001) and T-SAW-C (27.35 [6.89] vs 18.37 [6.09]; P < .001) scores. As reported via mean (range) NASA-TLX score, there were significant reductions for MR-SC vs SC in participant temporal demands (38 [20-50] vs 46 [30-70]; P = .03) and significant improvements in self-reported task performance (50 [30-60] vs 39 [10-70]; P = .01). Overall, 19 participants (86.4%) reported that they were more confident in making clinical decisions and undertaking clinical procedures with MR support. This study found that the use of MR technology reduced error, improved teamwork, and enhanced practitioner confidence when used to support the delivery of simulated emergency medical care. ClinicalTrials.gov Identifier: NCT05870137.

Embodied mixed reality with passive haptics in STEM education: randomized control study with chemistry titration

Researchers, educators, and multimedia designers need to better understand how mixing physical tangible objects with virtual experiences affects learning and science identity. In this novel study, a 3D-printed tangible that is an accurate facsimile of the sort of expensive glassware that chemists use in real laboratories is tethered to a laptop with a digitized lesson. Interactive educational content is increasingly being placed online, it is important to understand the educational boundary conditions associated with passive haptics and 3D-printed manipulables. Cost-effective printed objects would be particularly welcome in rural and low Socio-Economic (SES) classrooms. A Mixed Reality (MR) experience was created that used a physical 3D-printed haptic burette to control a computer-based chemistry titration experiment. This randomized control trial study with 136 college students had two conditions: 1) low-embodied control (using keyboard arrows), and 2) high-embodied experimental (physically turning a valve/stopcock on the 3D-printed burette). Although both groups displayed similar significant gains on the declarative knowledge test, deeper analyses revealed nuanced Aptitude by Treatment Interactions (ATIs). These interactionsfavored the high-embodied experimental group that used the MR devicefor both titration-specific posttest knowledge questions and for science efficacy and science identity. Those students with higher prior science knowledge displayed higher titration knowledge scores after using the experimental 3D-printed haptic device. A multi-modal linguistic and gesture analysis revealed that during recall the experimental participants used the stopcock-turning gesture significantly more often, and their recalls created a significantly different Epistemic Network Analysis (ENA). ENA is a type of 2D projection of the recall data, stronger connections were seen in the high embodied group mainly centering on the key hand-turning gesture. Instructors and designers should consider the multi-modal and multi-dimensional nature of the user interface, and how the addition of another sensory-based learning signal (haptics) might differentially affect lower prior knowledge students. One hypothesis is that haptically manipulating novel devices during learning may create more cognitive load. For low prior knowledge students, it may be advantageous for them to begin learning content on a more ubiquitous interface (e.g., keyboard) before moving them to more novel, multi-modal MR devices/interfaces.

Mitigating Technological Anxiety through the Application of Natural Interaction in Mixed Reality Systems

Technology anxiety contributes to an increased cognitive load and reduces user adoption of novel technologies. An illustrative example of this phenomenon is observed in the field of smart homes.To address this issue, we have identified an interaction paradigm known as natural interaction, which enables humans to engage with technology in a way that closely resembles natural human behavior and communication. This approach offers more user-friendly interactions that users are already familiar with, potentially reducing their cognitive load. In this paper, we present a case study of a smart lock system where we implement natural interaction and deploy it on a mixed reality (MR) device. By leveraging advanced features offered by MR head-mounted displays, we recreate the experience of people opening locks in everyday life. We conduct a user study comparing this interaction with traditional WIMP interaction in a mixed-reality environment. Through the analysis of collected data and user feedback, we examine the advantages and limitations of our proposed system.

Usage of Object Matching Algorithms Combined with Mixed Reality for Enhanced Decision Making in Orbital Reconstruction-A Technical Note.

This technical note describes the usage of object matching to virtually compare different modes of reconstruction in orbital trauma and display the results to the surgeon and the patient pre-operatively via mixed reality devices for enhanced surgical decision making and immersive patient education. A case of an orbital floor fracture is presented for which surface and volume matching were implemented to compare orbital reconstruction utilizing pre-fabricated titanium meshes versus patient-specific implants. The results could be visualized by mixed reality devices to further enhance surgical decision-making. The data sets were demonstrated to the patient in mixed reality for immersive patient education and enhanced shared decision making. The advantages of the new technologies are discussed in view of the new possibilities of improved patient education and informed consent processes, as well as new ways of teaching medical trainees.

Use of Mixed Reality in Neurosurgery Training: A Single Centre Experience

Mixed reality (MR) technology has opened new avenues for planning, visualization, and education in surgery. Neurosurgical pathologies require a very clear understanding of the relationships between pathology and critical neurovascular structures. The decline in cadaveric dissections and resource constraints has pushed the educators to find newer ways of rendering the same knowledge. The aim of this study was to determine the feasibility of employing a MR device in a high-volume center for neurosurgical teaching. The study also evaluated the results of the trainee experience in using the MR platform. Three neurosurgical consultants who are part of the teaching faculty were asked to facilitate the session. No prior training on utilizing the MR device was given to the trainees. HoloLens 2 was used as the MR device. Two questionnaires were used to understand the experience of the trainees. Eight active neurosurgical trainees who are currently training at our institution were recruited for the purposes of this study. Despite having no prior training on a MR platform, the learning curve was short for most of the trainees. Whether MR replace current traditional methods of teaching neuroanatomy, the response was divided across the trainees. The results of the User Experience Questionnaire were positive with the trainees finding the device as attractive, dependable, novel, and user-friendly. This study demonstrates the feasibility of using MR platform in neurosurgery training without significant preparation requirements. These data are required to justify the future investment in this technology for training institutions.

Preview the Products on Display at Display Week 2023

Preview the Products on Display at Display Week 2023

3D Indoor Mapping and BIM Reconstruction Editorial

This Special Issue gathers papers reporting research on various aspects of the use of low-cost photogrammetric and lidar sensors for indoor building reconstruction. It includes contributions presenting improvements in the alignment of mobile mapping systems with and without a prior 3D BIM model, the interpretation of both imagery and lidar data of indoor scenery and finally the reconstruction and enrichment of existing 3D point clouds and meshes with BIM information. Concretely, the publications showcase methods and experiments for the Reconstruction of Indoor Navigation Elements for Point Cloud of Buildings with Occlusions and Openings by Wall Segment Restoration from Indoor Context Labeling, Two-Step Alignment of Mixed Reality Devices to Existing Building Data, Pose Normalization of Indoor Mapping Datasets Partially Compliant with the Manhattan World Assumption, A Robust Rigid Registration Framework of 3D Indoor Scene Point Clouds Based on RGB-D Information, 3D Point Cloud Semantic Augmentation for Instance Segmentation of 360° Panoramas by Deep Learning Techniques and the Symmetry-Based Coarse Registration of Smartphone’s Colorful Point Clouds with CAD Drawings (RegARD) for Low-Cost Digital Twin Buildings.

Makeup Application Process Using a Mixed Reality (MR) Training System

Purpose: This study investigated the makeup application process using a mixed reality (MR) education system for beauty majors and compared the effects of virtual reality (VR) and augmented reality (AR) in the makeup education field. This study also investigated the utilization and development potential of makeup education using the recently developed AR-based MR technique.Methods: A survey was conducted targeting beauty majors. A total of 230 questionnaires were distributed, and 200 effective questionnaires were used in the final data analysis. The collected data were analyzed with frequency analysis, factor analysis, descriptive statistical analysis, and repeated measures analysis of variance using the SPSS 21.0 program.Results: An analysis of the questionnaires applied to makeup majors found that the expectation of usability in makeup education according to technological development increased equally in the order of VR, AR, and MR. The realistic expectations of the makeup application education from VR to MR increased in three groups, and expectations were particularly high in the case of AR and the utilization of MR in makeup education based on it. We presented a makeup education method from the first-person point of view using the MR device, HoloLens, and we suggested the direction of customized education for auditors based on the popularization of MR devices.Conclusion: Mixed reality-based makeup education presents a new direction in the field of makeup education by providing various types of makeup information consistent with the 4th industrial revolution and the revitalization of alternatives to face-to-face education in the post-COVID-19 era.

The Application of Mixed Reality in Root Canal Treatment

The priority of modern dentistry is to keep patients’ teeth for as long as possible. Tooth extraction is a procedure performed as a last resort when conservative methods and endodontic surgery procedures have not brought the expected results. As a consequence, the number of patients in dental offices, who require first and repeated endodontic treatment, is increasing. Thus, the development of new technologies in the medical industry, including microscopy, computer tomography (CT), as well as diode and neodymium-YAG-erbium lasers, enables dentists to increase the percentage of successful treatments. Moreover, mixed reality (MR) is a very new technology, in which the 3D view can help plan or simulate various types of tasks before they will be carried out in real life. In dentistry, 3D holography can be applied to display CT data to plan endodontic treatment. The most important element in effective root canal treatment is the precise imaging of the root canal. The CT scans allow dentists to view the anatomy of the patient’s tooth with much higher precision and understanding than using 2D radiography (RTG-radiographic photo) pictures. Recently, the development of new 3D technologies allows dentists to obtain even more data from existing CT scans. In this paper, the CT scan data were applied to generate patient teeth in 3D and simulate the view of the root canal’s anatomy in MR devices, i.e., Microsoft HoloLens 2. Using DICOM RAW data from the CT exam, we generated a 3D model of the jaw with a tooth. In the next step, the crown of the tooth was removed in a similar way to how a dentist would do this using a dental handpiece. Furthermore, all root canals were cleaned of everything inside. This way we achieved empty tunnels, namely root canals. Finally, we added appropriate lighting, similar to the type of lighting that dentists use. The proposed approach enables to display of the root canals in the same way as during the endodontic procedure using a microscope. It allows for the visualization of the root canal and changing its direction, in which dimensional accuracy is crucial. It turns out that mixed reality can be considered a complementary method to the traditional approach, which reduces the amount of time for the root canal treatment procedure by up to 72.25%, depending on the complexity of the case, and increases its effectiveness. Thus, the mixed reality-based system can be considered an effective tool for planning dental treatment.

Comparison of Point Cloud Registration Algorithms for Mixed-Reality Cross-Device Global Localization

State-of-the-art approaches for localization and mapping are based on local features in images. Along with these features, modern augmented and mixed-reality devices enable building a mesh of the surrounding space. Using this mesh map, we can solve the problem of cross-device localization. This approach is independent of the type of feature descriptors and SLAM used onboard the AR/MR device. The mesh could be reduced to the point cloud that only takes vertices. We analyzed and compared different point cloud registration methods applicable to the problem. In addition, we proposed a new pipeline Feature Inliers Graph Registration Approach (FIGRA) for the co-localization of AR/MR devices using point clouds. The comparative analysis of Go-ICP, Bayesian-ICP, FGR, Teaser++, and FIGRA shows that feature-based methods are more robust and faster than ICP-based methods. Through an in-depth comparison of the feature-based methods with the usual fast point feature histogram and the new weighted height image descriptor, we found that FIGRA has a better performance due to its effective graph-theoretic base. The proposed pipeline allows one to match point clouds in complex real scenarios with low overlap and sparse point density.

Applications and implications for extended reality to improve binocular vision and stereopsis.

Extended reality (XR) devices, including virtual reality (VR), augmented reality (AR), and mixed reality (MR) devices, are immersive technologies that can swap or merge the natural environment with virtual content (e.g., videogames, movies, or other content). Although these devices are widely used for playing videogames and other applications, they have one distinct feature that makes them potentially very useful for the measurement and treatment of binocular vision anomalies-they can deliver different content to the two eyes simultaneously. Indeed, horizontally shifting the images in the two eyes (thereby creating binocular disparity) can provide the user with a compelling percept of depth through stereopsis. Because these devices are stereoscopic, they can also be used as high-tech synoptophores, in which the images to the two eyes differ in contrast, luminance, size, position, and content for measuring and treating binocular anomalies. The inclusion of eye tracking in VR adds an additional dimension to its utility in measuring and treating binocular vision anomalies, as well as other conditions. This paper describes the essential requirements for testing and treating binocular anomalies and reviews current studies in which XR devices have been used to measure and treat binocular vision anomalies.

Mixed Reality or Simply Mobile? A Case Study on Enabling Less Skilled Workers to Perform Routine Maintenance Tasks

The availability of skilled workers has become a severe issue for production companies in developed countries like Germany in recent years. Specialists for maintenance tasks are in strong demand and their workforce should be steered towards the most challenging tasks in maintenance. In this case study we analyze, whether less skilled or not specifically instructed workers can perform routine maintenance tasks without being directly supported by experts. In our setup ten participants try to handle a maintenance routine checklist in three different ways: paper-based printouts, on a mobile tablet, and with a Microsoft HoloLens 2 Mixed Reality (MR) device. The results show that workers fail to complete the task with a mere paper-based approach, but are able to succeed both with the mobile tablet and MR application. Quantitative time measurements and qualitative analyses indicate that the mobile tablet app was easier to learn, did not impair the worker as much as the HoloLens and worked more stable. However, the HoloLens solution showed advantages as hands-free and spatial guiding device, which could be important aspects when better MR hardware will be available and MR interaction principles become common knowledge.

Development of a distributed MR-IoT method for operations and maintenance of underground pipeline network

The underground pipeline network (UPN) is an essential infrastructure and plays an irreplaceable role in national defense and urban activities. The complexity of structural environment and management makes its operation and maintenance difficult. To solve this problem, a distributed mixed reality (MR) and internet of things (IoT) system is developed through game thinking. Firstly, digital models are created based on design drawings and real-world environments, and then an MR system for the UPN is built by the game engine and the OpenXR platform. Secondly, an IoT cloud platform is built to connect with the MR system based on the API sets and cloud services; the data communication between sensors and MR devices is linked with the Socket method, and the data filtering model is constructed by the Kalman algorithm to realize the information exchange between the field workers and the backend managers. Finally, the National Center for Materials Service Safety at the University of Science and Technology Beijing (NCMS_USTB) is used as the experimental site to test this system, and its underground sewage and rainwater pipeline network are used to simulate the key problems in the operation and maintenance. The effect of the application shows that there is potential technical complementarity between the MR and IoT, and the distributed MR-IoT approach can be used as a new technical reference for the operation and maintenance of the UPN.