HoloLens Research Articles

A novel training session: mixed reality evisceration surgery webinar.

Evisceration surgery is an important part of an ophthalmologist's skillset but is an uncommon procedure. Therefore, it is important for training ophthalmologists to have adequate knowledge and confidence in this procedure to maximise learning opportunities. Mixed reality involves the intertwining of real and virtual environments and has been used successfully in surgical training. To improve participants' knowledge of the relevant anatomy, steps and confidence in performing evisceration surgery using an extended reality online webinar coupled with conventional teaching. The teaching session comprised of a short lecture introducing evisceration and explaining mixed reality technology including the Microsoft HoloLens headset. This was followed by a 3D demonstration of the orbit and globe, then the live evisceration procedure on a porcine cadaver, with the surgeon's point of view via the headset. There was expert live running commentary during the procedure with the use of virtual signposts on relevant anatomy. Participants were encouraged to be interactive and given the ability to share their annotations on-screen. Ten doctors ranging from Foundation year 1 to Ophthalmology specialist training year 3 attended the 90-min session and completed pre- and post- session questionnaires (5-point Likert scales). Results showed statistically significant improvements in the knowledge, steps and confidence in performing the evisceration surgery (paired two-sample T-tests, SPSS version 29). This study has demonstrated that utilisation of mixed reality with conventional teaching methods can provide a useful initial learning experience which may facilitate effective further learning of the evisceration surgery in a wet-lab or patient setting.

Enhancing Cognitive Fit: Exploring the Potential of Mixed Reality for Developing Mental Rotation Skills

Mixed Reality is a promising venue for spatial ability training, allowing participants to engage in problem-solving through gesture-based interactions with holographic objects. 3-D measurement of learning in Mixed Reality environments may result in a better cognitive fit than 2-D. This mixed-method study measures and enhances mental rotation ability and investigates user experience in Mixed Reality. To assess mental rotation ability, we adapted the Purdue Spatial Visualization Test:Rotation for Microsoft HoloLens 2, creating a measurement tool for our Mixed Reality-based training program, Holomental. Comparing 2-D and 3-D tests, we explored how stimulus dimensionality influences accuracy and cognitive load. Our findings indicate that Holomental enhances mental rotation performance, both in 2-D and 3-D. Cognitive load in the 3-D test was lower than in the 2-D test. Semi-structured interviews revealed participants’ appreciation for the representational and interactional affordances of the training environment. This study provides valuable insights into the effectiveness of 3-D spatial training, with implications and suggestions for future design considerations.

Nested Semi-Transparent Isosurface Simulated Volume-Rendering (NESTIS-VR) – An efficient on-device rendering approach for Augmented Reality headsets increasing surgeon confidence of kidney donor arterial anatomy

Background and objectiveVolume-renderings of computed tomography or magnetic resonance angiograms (MRAs) are routinely used by surgeons in the preoperative assessment of vascular anatomy in kidney donors. Stereoscopic headsets (OST-HMD) like Microsoft HoloLens allow intuitive interaction with three-dimensional content for more intuitive comprehension, but do not allow real-time ray-casting volume-rendering of medical volume datasets on-device due to computational limitations. MethodsWe introduce NEsted Semi-Transparent Isosurface Simulated Volume-Rendering (NESTIS-VR), as an on-device alternative to ray-casting volume-rendering and developed an application for HoloLens to render kidney donor MRAs with interactive control of fundamental rendering parameters. We compared NESTIS-VR with current standard pre-calculated 2D ray-cast volume-renderings in an observational study with 2 expert kidney transplant surgeons, measuring their confidence in pre-operatively assessing the kidney pedicle arterial anatomy in 20 potential donors. We also compared it against other 3D rendering techniques to understand which features contributed most to any improvements. ResultsReal-time stereoscopic three-dimensional (3D) NESTIS-VR in Augmented Reality significantly improves surgeons’ confidence compared with pre-calculated conventional two-dimensional (2D) ray-casting volume-rendered images (p = 0.0415/p = 0.00003). 2D non-stereoscopic NESTIS-VR was significantly superior to pre-calculated 2D ray-casting volume-rendered images for both surgeons (p = 0.044/p = 0.0003). Single isosurface 2D rendering was significantly superior than pre-calculated 2D volume-rendered images for one surgeon. There was no significant difference between binocular 3D display over 2D views with NESTIS-VR or between constrained and unconstrained vantage points for 2D viewing. ConclusionNESTIS-VR provides a new approach to rendering medical datasets in computationally limited OST-HMD headsets and significantly increases surgeons’ confidence of kidney donor arterial anatomy. The principal confidence benefit arises from providing surgeons interactive control over rendering parameters compared to pre-calculated renderings at preset parameters whilst rendering on-device and keeping the OST-HMD untethered from a workstation.

Evaluation of Optical Tracking to Distinguish Surgeon Experience During Endoscopic Stone Surgery.

Introduction and Objectives: Optical tracking (OT) has shown potential in assessing surgical skill but has yet to be evaluated for endoscopic urologic surgery. We sought to evaluate the potential for OT to distinguish expert and trainee surgeons during flexible ureteroscopy (fURS) for kidney stone treatment in both simulated and live surgical settings. Methods: We performed OT analysis of six surgeons performing stone localization during fURS in two settings. In the first setting, surgeons were tasked with fiducial localization in three separate kidney phantoms during fURS. In the second setting, surgeons performed stone localization via fURS in five separate patients. Surgeons were categorized as "expert" (n = 3, endourologist, average case volume of >100 fURS per year) or trainee (n = 3, trainee, <100 fURS per year). OT metrics were recorded for both settings using the Microsoft HoloLens 2© as the surgeons viewed the surgical monitor during fURS. Standard OT metrics of experts and trainees were compared, and included: area of eye gaze movement, gaze distance traveled, number of gaze fixation points, percentage of gaze fixation dwell time, and number of saccades. Results: In the simulated setting, the average time for stone localization was greater for trainees compared to experts (318 seconds vs 52 seconds, p < 0.01). Additionally, the mean area of eye gaze movements with area of interest (AOI) was greater for trainees compared to experts (1430 cm2 vs 1060 cm2, p < 0.01). The total gaze distance traveled was also greater for trainees compared to experts (1480 cm vs 730 cm, p < 0.01). In the live surgical setting, average time for stone localization was similar for trainees and experts (74 seconds vs 51 seconds, p = 0.05). The total area of eye gaze movements in AOI was greater for trainees compared to experts (700 cm2 vs 30 cm2, p < 0.01). Additionally, the total gaze distance traveled was greater for trainees compared to experts (14,000 cm vs 680 cm, p < 0.01). This suggests more varied and less point specific concentration on the surgical screen by trainees. There was no difference in percentage of gaze fixation dwell time and number of saccades between experts and trainees in either setting. Conclusions: OT analysis can objectively distinguish surgical experience between experts and trainee surgeons performing fURS in both simulated and live surgical settings. These findings may play a role in future surgical training and skills assessment.

Initial framework design of a digital twin mixed-reality-application on human-robot bi-directional collaboration for forming double curvature plate

This paper explores and identifies commonalities among recent Digital Twin (DT) technology definitions. additionally, it introduces a framework for a digital twin of Human-Robot Collaboration (HRC) focusing on English wheel manufacturing as its foundational context. To achieve this, two 3D simulators, namely Grasshopper and Rhino, were employed to faithfully replicate the English wheel process digitally. Subsequently, optimizing these simulators’ performance paved the way for a comprehensive visualization of the English wheel using the Microsoft HoloLens. This visualization integrates digital and physical interactions, resulting in a mixed reality environment. The ultimate goal of this research is to enable a bi-directional collaboration within the digital representation of the English wheel, bridging the gap between the virtual and physical realms.

Design of a Mixed-Reality Application to Reduce Pediatric Medication Errors in Prehospital Emergency Care

Children in prehospital emergency care are particularly vulnerable to medication errors, often with serious consequences. A prior study analyzing prehospital pediatric medication dosing errors, conducted after the implementation of a statewide pediatric drug-dosing reference for emergency medical services (EMS), identified an alarmingly high error rate. This significant finding led to the current study, which aims to develop technological interventions to reduce the frequency of medication errors for children during treatment by EMS. The current study focuses on the design and development of a safety strategy to automate medication administration using mixed-reality technology. Simulations were conducted to inform the design process, focusing on three scenarios: cardiac arrest, seizure, and burns. The design team included medical and engineering researchers, paramedics, and emergency medical technicians from multiple emergency medical service agencies. Root cause analysis (RCA) and failure mode and effects analysis (FMEA) were conducted after the simulations were completed. The RCA and FMEA were used to identify and prioritize failure points, which were then addressed in a mixed-reality solution using Microsoft HoloLens 2 to automate and enhance pediatric medication administration in prehospital emergency care. The resulting application will provide real-time assistance to guide paramedics through the complicated medication dosing and administration process using a detailed step-by-step guide, aiming to decrease medication errors and improve medication dosing accuracy.

Augmented reality for point-of-care ultrasound-guided vascular access in pediatric patients using Microsoft HoloLens 2: a preliminary evaluation.

Conventional ultrasound-guided vascular access procedures are challenging due to the need for anatomical understanding, precise needle manipulation, and hand-eye coordination. Recently, augmented reality (AR)-based guidance has emerged as an aid to improve procedural efficiency and potential outcomes. However, its application in pediatric vascular access has not been comprehensively evaluated. We developed an AR ultrasound application, HoloUS, using the Microsoft HoloLens 2 to display live ultrasound images directly in the proceduralist's field of view. We presented our evaluation of the effect of using the Microsoft HoloLens 2 for point-of-care ultrasound (POCUS)-guided vascular access in 30 pediatric patients. A custom software module was developed on a tablet capable of capturing the moving ultrasound image from any ultrasound machine's screen. The captured image was compressed and sent to the HoloLens 2 via a hotspot without needing Internet access. On the HoloLens 2, we developed a custom software module to receive, decompress, and display the live ultrasound image. Hand gesture and voice command features were implemented for the user to reposition, resize, and change the gain and the contrast of the image. We evaluated 30 (15 successful control and 12 successful interventional) cases completed in a single-center, prospective, randomized study. The mean overall rendering latency and the rendering frame rate of the HoloUS application were 139.30ms and 30 frames per second, respectively. The average procedure completion time was 17.3% shorter using AR guidance. The numbers of puncture attempts and needle redirections were similar between the two groups, and the number of head adjustments was minimal in the interventional group. We presented our evaluation of the results from the first study using the Microsoft HoloLens 2 that investigates AR-based POCUS-guided vascular access in pediatric patients. Our evaluation confirmed clinical feasibility and potential improvement in procedural efficiency.

Enhancing surgical navigation: a robust hand-eye calibration method for the Microsoft HoloLens 2.

Optical-see-through head-mounted displays have the ability to seamlessly integrate virtual content with the real world through a transparent lens and an optical combiner. Although their potential for use in surgical settings has been explored, their clinical translation is sparse in the current literature, largely due to their limited tracking capabilities and the need for manual alignment of virtual representations of objects with their real-world counterparts. We propose a simple and robust hand-eye calibration process for the depth camera of the Microsoft HoloLens2, utilizing a tracked surgical stylus fitted with infrared reflective spheres as the calibration tool. Using a Monte Carlo simulation and a paired-fiducial registration algorithm, we show that a calibration accuracy of 1.65mm can be achieved with as little as 6 fiducial points. We also present heuristics for optimizing the accuracy of the calibration. The ability to use our calibration method in a clinical setting is validated through a user study, with users achieving a mean calibration accuracy of 1.67mm in an average time of 42s. This work enables real-time hand-eye calibration for the Microsoft HoloLens2, without any need for a manual alignment process. Using this framework, existing surgical navigation systems employing optical or electromagnetic tracking can easily be incorporated into an augmented reality environment with a high degree of accuracy.

Using Mixed Reality for Control and Monitoring of Robot Model Based on Robot Operating System 2

This article presents the design and implementation of an innovative human–machine interface (HMI) in mixed reality for a robot model operating within Robot Operating System 2 (ROS 2). The interface is specifically developed for compatibility with Microsoft HoloLens 2 hardware and leverages the Unity game engine alongside the Mixed Reality Toolkit (MRTK) to create an immersive mixed reality application. The project uses the Turtlebot 3 Burger model robot, simulated within the Gazebo virtual environment, as a representative mechatronic system for demonstration purposes. Communication between the mixed reality application and ROS 2 is facilitated through a publish–subscribe mechanism, utilizing ROS TCP Connector for message serialization between nodes. This interface not only enhances the user experience by allowing for the real-time monitoring and control of the robotic system but also aligns with the principles of Industry 5.0, emphasizing human-centric and inclusive technological advancements. The practical outcomes of this research include a fully functional mixed reality application that integrates seamlessly with ROS 2, showcasing the potential of mixed reality technologies in advancing the field of industrial automation and human–machine interaction.

Scanning indoor environments with Microsoft HoloLens 2 for interactive generation of floor plans

Scanning indoor environments with Microsoft HoloLens 2 for interactive generation of floor plans

Prototype development of a multi-user augmented reality framework for collaborative medical visualization in shared environments

Abstract This study developed and evaluated a prototype for augmented reality (AR) interaction among multiple users sharing a three-dimensional hologram within a shared physical space. A systems requirements analysis was conducted to determine necessary interactions, followed by an iterative implementation plan with testing at each step. The application for the HoloLens 2 (HL2) headset was developed using Unity for 3D design, C# for programming, and Photon for multiplayer capabilities. The prototype was evaluated in a user study utilizing custom and NASA TLX questionnaires, comparing participants' self-reported task success with recorded data to assess the accuracy of selfassessments. The prototype effectively facilitated multi-user interactions within a shared environment, achieving acceptable synchronization for two users. Most participants successfully completed tasks, with TLX scores indicating low cognitive and mechanical load during AR task execution. This study demonstrates AR's viability for displaying and manipulating complex three-dimensional structures within a collaborative medical context among multiple individuals in the same physical space.

Utilizing Augmented Reality to Evaluate Service Member Team Performance.

Decision-making is a complex process that relies on situational awareness and experience to create a potential list of actions while weighing the risks and benefits of each action. There is a paucity of data evaluating decision-making for individual service members (SM) during the performance of team-based military-relevant activities. Understanding individual performance and decision-making within the context of a team-based activity has potential to aid in the detection and management of mild traumatic brain injuries and assist with safe and timely return-to-duty decision making. The aim of this project was to evaluate cognitive and motor performance in healthy SM during an augmented reality military specific, team-based activity. Data from 110 SMs from Fort Moore Georgia were analyzed for this project. Service members completed 3 augmented reality room breaching and clearing scenarios (Empty Room, Civilian/Combatant, and Incorrect Position of a unit member) with 3 avatar team members. Participants wore a Microsoft HoloLens 2 (HL2) device and used a replica M4 weapon (Haptech Defense Systems) during scenarios. Three-dimensional position data from the HL2 headset was used to compute temporal measures of room breaching and clearing events while the number and timing of weapon discharge was monitored by the M4. Temporal outcomes included time to enter room, time to fire first shot, time in fatal funnel, and total trial time while motor outcomes were distance traveled and average movement velocity. Pairwise comparisons between the Incorrect Position scenario and the Civilian/Combatant scenario demonstrated no difference in time to enter the room (2.36 seconds in both scenarios). Time to fire the first shot in the Civilian/Combatant scenario was longer (0.97 seconds to 0.58 seconds) while time in fatal funnel (2.58 seconds to 3.31 seconds) and time to trial completion (7.46 seconds to 8.41 seconds) were significantly longer in the Incorrect Position scenario. Reaction time to fire the first shot, time in the fatal funnel, and total trial time reflect a change in information-processing and decision-making capabilities during military specific, ecological, team-based scenarios when altering the environment inside of the room and modifying avatar movements. Future studies are planned to evaluate the effects of mild traumatic brain injury on specific aspects of military team performance.

Mixed reality simulation training in the assessment and management of acutely unwell patients in undergraduate medical education: a pilot study

Background: Development of the undergraduate simulation experience is needed to continue developing aspiring clinicians’ ability to assess and manage unwell patients. However, its heavy resource demand and ‘Generation Z’s’ desire for technologically enhanced learning means novel simulation modalities must now be considered. Mixed reality (MR) provides a platform to deliver such simulation; however, it is yet to be studied in this context. We conducted an observational cohort study with the aim to assess ift MR simulation improves students’ perceived ability to assess and manage unwell patients. Methods: We recruited 32 undergraduate medical students and asked them to rate their ability to assess and manage acutely unwell patients, out of 10. An MR simulation was then delivered using the Microsoft HoloLens and HoloScenario software produced by GigXR. Students then rated their assessment and management ability again, alongside their experience of the system and its usability. Data analysis used paired t-tests to assess for significant differences. Results: By attending the simulation, students showed a significant improvement in perceived ability to assess (p &lt; 0.001) and manage (p &lt; 0.001) acutely unwell patients. Mean assessment scores improved by 1.09/10 (95% confidence interval [CI 0.67, 1.52]) with 89% of students feeling more confident. Mean management scores improved by 1.63/10 (95% CI [1.15, 2.10]) with 84% of students reporting increased confidence levels. When considering usability, 69% of students did not find the MR system easy to use, with 75% of students having technical issues and 38% experiencing side effects. Overall, 88% of students believed the experience was beneficial to their learning. Conclusions: MR shows promise in its ability to deliver simulation training and improve students’ perceived ability to assess and manage unwell patients. Advances in software availability and content are required for integration into undergraduate medical curricula. Further research is required to assess if these results are replicated objectively.

Comparative Analysis of Mixed Reality and PowerPoint in Education: Tailoring Learning Approaches to Cognitive Profiles.

The term immersive technology refers to various types of technologies and perspectives that are constantly changing and developing. It can be used for different purposes and domains such as education, healthcare, entertainment, arts, and engineering. This paper aims to compare the effectiveness of immersive technologies used in education, namely mixed reality, generated with Microsoft HoloLens 2, with traditional teaching methods. The experiment involves comparing two groups of students who received different training methods: the first group saw a PowerPoint slide with an image of the human muscular system, while the second group saw a 3D hologram of the human body that showed the same muscle groups as in the PowerPoint (PPT). By integrating the Intelligence Quotient (IQ) levels of the participants as a predictive variable, the study sought to ascertain whether the incorporation of mixed reality technology could significantly influence the learning outcomes and retention capabilities of the learners. This investigation was designed to contribute to the evolving pedagogical landscape by providing empirical evidence on the potential benefits of advanced educational technologies in diverse learning environments. The main finding of this study indicates that while MR has potential, its effectiveness is closely tied to its interactivity. In cases where the content remains static and non-interactive, MR does not significantly enhance in-formation retention compared to traditional PPT methods. Additionally, the study highlights that instructional strategies should be adapted to individual cognitive profiles, as the technology type (MR or PPT) alone does not significantly impact learning outcomes when the information presented is identical.

Robot-Assisted Augmented Reality (AR)-Guided Surgical Navigation for Periacetabular Osteotomy.

Periacetabular osteotomy (PAO) is an effective approach for the surgical treatment of developmental dysplasia of the hip (DDH). However, due to the complex anatomical structure around the hip joint and the limited field of view (FoV) during the surgery, it is challenging for surgeons to perform a PAO surgery. To solve this challenge, we propose a robot-assisted, augmented reality (AR)-guided surgical navigation system for PAO. The system mainly consists of a robot arm, an optical tracker, and a Microsoft HoloLens 2 headset, which is a state-of-the-art (SOTA) optical see-through (OST) head-mounted display (HMD). For AR guidance, we propose an optical marker-based AR registration method to estimate a transformation from the optical tracker coordinate system (COS) to the virtual space COS such that the virtual models can be superimposed on the corresponding physical counterparts. Furthermore, to guide the osteotomy, the developed system automatically aligns a bone saw with osteotomy planes planned in preoperative images. Then, it provides surgeons with not only virtual constraints to restrict movement of the bone saw but also AR guidance for visual feedback without sight diversion, leading to higher surgical accuracy and improved surgical safety. Comprehensive experiments were conducted to evaluate both the AR registration accuracy and osteotomy accuracy of the developed navigation system. The proposed AR registration method achieved an average mean absolute distance error (mADE) of 1.96 ± 0.43 mm. The robotic system achieved an average center translation error of 0.96 ± 0.23 mm, an average maximum distance of 1.31 ± 0.20 mm, and an average angular deviation of 3.77 ± 0.85°. Experimental results demonstrated both the AR registration accuracy and the osteotomy accuracy of the developed system.

An efficient image classification and segmentation method for crime investigation applications

AbstractThe field of forensic science is experiencing significant growth, largely driven by the increasing integration of holographic and immersive technologies, along with their associated head-mounted displays. These immersive systems have become increasingly vital in resolving critical crimes as they facilitate communication, interaction, and collaboration. Given the sensitive nature of their work, crime investigators require substantial technical support. There is a pressing need for accurate documentation and archiving of crime scenes, which can be addressed by leveraging 3D scanned scenes to accurately represent evidence and expected scenarios. This study aims to develop an enhanced AR. system that can be deployed on hologram facilities such as the Microsoft HoloLens. The proposed system encompasses two main approaches, namely image classification and image segmentation. Image classification utilizes various deep learning models, including lightweight convolutional neural networks (CNNs) and convolutional Long-Short Term Memory (ConvLSTM). Additionally, the image segmentation approach is based on the fuzzy active contour model (FACM). The effectiveness of the proposed system was evaluated for both classification and segmentation tasks, utilizing metrics such as accuracy, sensitivity, precision, and F1 score. The simulation results indicate that the proposed system achieved a 99% accuracy rate in classification and segmentation tasks, positioning it as an effective solution for detecting bloodstain patterns in AR applications.

Using a mixed-reality headset to elicit and track clinically relevant movement in the clinic.

21st century neurology will require scalable and quantitative tools that can improve neurologic evaluations over telehealth and expand access to care. Commercially available mixed-reality headsets allow for simultaneous presentation of stimuli via holograms projected into the real world and objective and quantitative measurement of hand movement, eye movement, and phonation. We created 6 tasks designed to mimic standard neurologic assessments and administered them to a single participant via the Microsoft HoloLens 2 mixed-reality headset. The tasks assessed postural hand tremor, finger tapping, pronation and supination of hands, hand and eye tracking of a center-out task, hand and eye tracking of a random motion task, and vocal assessment. We show the utility of the HoloLens for commonly used neurological exams. First, we demonstrate that headset-derived holograms can project hand movements and objects in 3D space, providing a method to accurately and reproducibly present test stimuli to reduce test-test variability. Second, we found that participant hand movements closely matched holographic stimuli using a variety of metrics calculated on recorded movement data. Third, we showed that the HoloLens can record and playback exam tasks for visual inspection, sharing with other medical providers, and future analysis. Fourth, we showed that vocal recordings and analysis could be used to profile vocal characteristics over time. Together, this demonstrates the versatility of mixed reality headsets and possible applications for neurological assessment. Administering components of the neurologic exam via a self-contained and commercially available mixed-reality headset has numerous benefits including detailed kinematic quantification, reproducible stimuli presentation from test to test, and can be self-administered expanding access to neurological care and saving hospital time and money. This work was supported by grants from the National Institutes of Health (NIH) (F30AG063468) (E.L.), (F31NS113395) (D.J.C), and the Pilot Grant Award from the University of Colorado Movement Disorders Center (D.J.C).

979 Enhancing Surgical Training Through Augmented Reality: A Pilot Study on the Use of the Microsoft HoloLens for Live-Streaming from Theatre

979 Enhancing Surgical Training Through Augmented Reality: A Pilot Study on the Use of the Microsoft HoloLens for Live-Streaming from Theatre

The feasibility and usability of mixed reality teaching in a hospital setting based on self-reported perceptions of medical students

BackgroundClinical teaching during encounters with real patients lies at the heart of medical education. Mixed reality (MR) using a Microsoft HoloLens 2 (HL2) offers the potential to address several challenges: including enabling remote learning; decreasing infection control risks; facilitating greater access to medical specialties; and enhancing learning by vertical integration of basic principles to clinical application. We aimed to assess the feasibility and usability of MR using the HL2 for teaching in a busy, tertiary referral university hospital.MethodsThis prospective observational study examined the use of the HL2 to facilitate a live two-way broadcast of a clinician-patient encounter, to remotely situated third and fourth year medical students. System Usability Scale (SUS) Scores were elicited from participating medical students, clinician, and technician. Feedback was also elicited from participating patients. A modified Evaluation of Technology-Enhanced Learning Materials: Learner Perceptions Questionnaire (mETELM) was completed by medical students and patients.ResultsThis was a mixed methods prospective, observational study, undertaken in the Day of Surgery Assessment Unit. Forty-seven medical students participated. The mean SUS score for medical students was 71.4 (SD 15.4), clinician (SUS = 75) and technician (SUS = 70) indicating good usability. The mETELM Questionnaire using a 7-point Likert Scale demonstrated MR was perceived to be more beneficial than a PowerPoint presentation (Median = 7, Range 6–7). Opinion amongst the student cohort was divided as to whether the MR tutorial was as beneficial for learning as a live patient encounter would have been (Median = 5, Range 3–6). Students were positive about the prospect of incorporating of MR in future tutorials (Median = 7, Range 5–7). The patients’ mETELM results indicate the HL2 did not affect communication with the clinician (Median = 7, Range 7–7). The MR tutorial was preferred to a format based on small group teaching at the bedside (Median = 6, Range 4–7).ConclusionsOur study findings indicate that MR teaching using the HL2 demonstrates good usability characteristics for providing education to medical students at least in a clinical setting and under conditions similar to those of our study. Also, it is feasible to deliver to remotely located students, although certain practical constraints apply including Wi-Fi and audio quality.

Designing and evaluation of a mixed reality system for crime scene investigation training: a hybrid approach

Police investigation in real-life crime scenes is an essential aspect of forensic science education. However, the practicality of bringing young investigators to actual crime scenes is often hindered by the costs and challenges involved. In order to overcome these obstacles, new technologies such as mixed reality (MR) are being explored as potential solutions. MR technology offers an interactive and cost-effective way to simulate real-life crime scenes, providing a valuable training experience for young investigators. This paper presents a novel design of a MR system using Microsoft HoloLens 2.0, which is tailored to work in a spatial 3D scanned and reconstructed crime scene using FARO point cloud 3D scanner X130 blended with photogrammetry techniques. The system was developed through the lens of Experiential Learning Theory and designed using a participatory approach, providing a cost-effective solution to help trained Kuwaiti police officers enhance their investigative skills. In order to evaluate the system’s user experience and user interaction, the Questionnaire of User Interaction Satisfaction and User Experience Questionnaire were utilised. Forty-four young police officers evaluated the system. Police students showed positive levels of satisfaction with user interaction and overall user experience with minimal negative feedback. Female students showed higher satisfaction with the overall impression compared to male students. Based on the positive feedback regarding the system expansion, the system will be taken into the commercialisation stage in the future to be provided as an essential tool for crime scene education and investigation practices.