Mixed Reality Simulation Research Articles

In Search of Deliberate Practice: Simulating Teaching in Three Teacher Education Programs

Background: Mixed reality simulations (MRSs) offer a platform for novice teachers to practice teaching in the absence of real children. Though prior research suggests simulated practice can be beneficial for novice teachers, these studies describe widely varying designs and implementations across teacher education programs (TEPs). As such, we lack clarity about the relative benefits of different approaches to simulated practice for novices who are entering teaching via a growing array of pathways. Purpose: To better understand the influence of context on novice teachers’ experiences in simulations, we conducted a study of three teacher education programs that used simulations as a component of their teacher education curriculum. Specifically, we asked the following research questions: How do novices in three differently structured TEPs experience a series of parallel practice opportunities in MRSs? What contextual features into, through, and beyond the approximation contribute to variation, if any, in novices’ experiences of practice in MRSs? Research Design: This mixed methods study leverages postsimulation survey data from novices ( n = 160) in three TEPs to understand how they experienced a parallel series of simulation tasks paired with support in the form of coaching or self-reflection. We elaborate on survey findings with qualitative analysis of each teacher education program and interviews with a purposive sample of participants from each program ( n = 15). Conclusions: Findings highlight the influence of context and supports on novices’ simulation experiences. In particular, the sequencing of practice with other preparatory opportunities into and beyond the approximation, as well as the provision of feedback and level of challenge through the practice can promote more useful practice opportunities in teacher education.

The Integration of Mixed Reality Simulation into Reading Literacy Modules

The Integration of Mixed Reality Simulation into Reading Literacy Modules

Understanding Mixed Reality Simulation as a Training Environment for Behavior and Special Education Knowledge and Skill Acquisition

Concerningly, pre-service teachers might utilize practices during practicum and student teaching experiences in ways that are less than satisfactory, or even harmful, to students, peers, or mentors. Mixed reality simulation (MRS), an educational tool for practicing teaching skills, may be a warranted approach to field experience that limits wrongdoing. However, few studies have researched the impact and meaningfulness of MRS teaching experiences. Therefore, this study combined both survey results and interview findings to report the extent to which MRS might improve the knowledge and skills of pre-service educators and whether MRS was considered an appropriate field experience alternative. Within one simulated scenario design, participants ( n = 94) were asked to engage five elementary-aged avatar students who presented challenging behaviors and asked questions or made comments about disability. Implications of findings are discussed in relation to mitigating common barriers of field experiences in higher education programs suggesting that, potentially, MRS may be an innovative way to respond to attrition concerns in the field of special education.

Research on Mixed Reality Hand Interaction Technology in Flight Simulation Teaching

Abstract The integration of mixed reality simulation training in institutional education and teaching processes can yield significant quality benefits. Non-wearable interactive methods represent a more natural form of hand interaction within flight cockpits, covering various emerging fields such as mixed reality, computer vision, and human-computer interaction. This paper is guided by a “human-centered” approach to natural human-computer interaction and proposes a virtual hand mixed reality interaction scheme based on hand natural feature points. This study introduces a hand detection, segmentation, and recognition algorithm based on skin colour and key hand feature points. Initially, the algorithm uses the Otsu adaptive threshold algorithm in the YCbCr space for skin colour detection to accommodate varying image brightness levels. Subsequently, it employs a hand Keypoint model for rapid hand detection within skin-like areas while excluding interference from other regions. During the process of hand tracking, optimization prediction is performed using the particle filter algorithm combined with the artificial fish swarm algorithm to achieve tracking accuracy within six pixels in both horizontal and vertical directions. Gesture recognition involves extracting Fourier descriptive contour features of the hand bone structure through identification of feature points. Furthermore, this study combines the optimization capabilities of the artificial fish swarm algorithm to enhance support vector machine model parameter optimization for improved recognition rates. Testing involved 800 processed images as part of a test set for gesture recognition; only five commonly used gestures within flight cockpits were recognized and classified with an accuracy rate reaching up to 98%. Finally, this research presents virtual representations of common operational gestures including natural hand states, control stick manipulation, and throttle handling positions, button activations, and rotational movements via head-mounted displays or screens. The proposed mixed reality-based hand gesture interaction system addresses challenges related to complex background interference during hand detection in cockpit environments as well as issues arising from lighting disturbances. Additionally it resolves problems associated with insufficient particles leading to tracking failures while significantly enhancing registration effects during tracking processes. Moreover it mitigates issues related to Fourier descriptive features being influenced by background variations or changes in pose along with limitations pertaining to expressing singular forms of gestures thereby improving overall classification accuracy.

Integrating Mixed-Reality Simulations in TESOL Teacher Preparation Programs: Principles, Strengths, and Weaknesses

Integrating Mixed-Reality Simulations in TESOL Teacher Preparation Programs: Principles, Strengths, and Weaknesses

Mixed reality simulation training in the assessment and management of acutely unwell patients in undergraduate medical education: a pilot study

Background: Development of the undergraduate simulation experience is needed to continue developing aspiring clinicians’ ability to assess and manage unwell patients. However, its heavy resource demand and ‘Generation Z’s’ desire for technologically enhanced learning means novel simulation modalities must now be considered. Mixed reality (MR) provides a platform to deliver such simulation; however, it is yet to be studied in this context. We conducted an observational cohort study with the aim to assess ift MR simulation improves students’ perceived ability to assess and manage unwell patients. Methods: We recruited 32 undergraduate medical students and asked them to rate their ability to assess and manage acutely unwell patients, out of 10. An MR simulation was then delivered using the Microsoft HoloLens and HoloScenario software produced by GigXR. Students then rated their assessment and management ability again, alongside their experience of the system and its usability. Data analysis used paired t-tests to assess for significant differences. Results: By attending the simulation, students showed a significant improvement in perceived ability to assess (p < 0.001) and manage (p < 0.001) acutely unwell patients. Mean assessment scores improved by 1.09/10 (95% confidence interval [CI 0.67, 1.52]) with 89% of students feeling more confident. Mean management scores improved by 1.63/10 (95% CI [1.15, 2.10]) with 84% of students reporting increased confidence levels. When considering usability, 69% of students did not find the MR system easy to use, with 75% of students having technical issues and 38% experiencing side effects. Overall, 88% of students believed the experience was beneficial to their learning. Conclusions: MR shows promise in its ability to deliver simulation training and improve students’ perceived ability to assess and manage unwell patients. Advances in software availability and content are required for integration into undergraduate medical curricula. Further research is required to assess if these results are replicated objectively.

Effect of Infection Control Simulation Based on a Negative Pressure Isolation Room Using Mixed Reality.

This study aimed to examine the effectiveness of an infection control simulation using mixed reality, comparing simulation fidelity with a high-fidelity mannequin (MN) group and problem-based learning with written cases group. This study used a three-group pretest-posttest quasi-experimental design. Two universities with similar curricula were conveniently selected, and a total of 72 nursing students were recruited. Participants were randomly assigned to three groups of 24 each. In the final analysis, there were 22 participants in the mixed reality groups, 21 in the mannequin groups, and 23 in the problem-based learning with written cases groups. Data were analyzed using descriptive statistics and the χ 2 , Kruskal-Wallis, and Wilcoxon signed rank tests. The mixed reality groups had a significantly positive effect on clinical reasoning ability and clinical competence than the problem-based learning with written cases groups, whereas the mannequin groups had a significantly positive effect on clinical competence than the problem-based learning with written cases groups. Mixed reality simulation is an appropriate simulation method that enhances learning immersion, satisfaction, and self-confidence in simulation. Additionally, it can substitute for mannequin simulation in terms of clinical reasoning ability and clinical competence. This study suggests that it is important to the various approaches in simulation fidelity to diversely enhance the competency of nursing students in simulation outcomes.

Using Mixed Reality to Teach Medical Students Lumbar Punctures

BackgroundGaining familiarity with bedside procedures during medical school is essential for preparing medical students for intern year. However, this is often not a core component of undergraduate medical education, and students often feel unprepared to perform bedside procedures upon graduation from medical school. Extended reality technologies have rapidly developed in the past few years, become more accessible, and show great promise in education, most recently in healthcare. We aim to show the feasibility and usability of a mixed-reality lumbar puncture simulated experience. MethodsWe created the mixed-reality simulation by filming a procedure conducted by an expert on a lumbar puncture simulator from the first-person view. We performed a study to assess the feasibility and usability of mixed-reality lumbar puncture training compared with traditional faculty-led high-fidelity mannikin-based simulation training. Fourth-year medical students were randomly selected to participate in the mixed-reality lumbar puncture training. Students watched the virtual first-person teaching via a virtual reality headset while mimicking hand movements on a lumbar puncture mannikin. Faculty taught students in the traditional high-fidelity group in small groups before practicing on the high-fidelity manikin. Students in both groups then performed the procedure entirely with performance assessed via a standardized checklist. Number of attempts to achieve a predetermined minimum passing score were recorded, and students were also surveyed regarding their experience with both simulations. ResultsFifty-nine percent of students felt comfortable using mixed reality as a training modality with less than 6% having any prior extended reality experience. More than 55% felt the mixed-reality experience was as effective as the traditional high-fidelity simulation training. Many students (72%) desired more mixed-reality opportunities. In terms of performance, 58.8% of students in the mixed-reality group vs 65.5% in the traditional group completed the checklist in 1 attempt (P = .79). ConclusionsWe demonstrate that mixed reality is an acceptable and effective modality to train health care professionals in lumbar punctures with opportunities to further enhance the immersive learning experience.

Depth perception optimization of mixed reality simulation systems based on multiple‐cue fusion

AbstractMixed reality technology can be applied to simulation training, improve surgical performance, enhance 3D game experience, and so on, attracting extensive attention of researchers. The perception of the user about head‐mounted display MR using HoloLens is critical, especially for precision applications such as virtual hoisting. Designing and adding appropriate depth cues in MR scenes is an effective way to improve users' depth perception. In this study, taking virtual hoisting training system as an example, the depth perception strategy of multi‐cue fusion is proposed to improve the perception effect. Based on the mechanism of human depth perception, five kinds of depth cues are designed. The depth perception effect of adding single cue is studied by perceptual matching experiment. Based on the principle of fuzzy clustering, a multiple‐cue comprehensive depth optimization strategy on viewing distance scale is proposed. Finally, the perceptual matching results demonstrate the effectiveness of the multi‐cue fusion strategy, and the average error is reduced by 20.68% compared with the single‐cue strategy, which can significantly improve the spatial depth perception. This research can provide a reference for improving users' depth perception in interactive MR simulation systems.

Authentic learning and job readiness: Are mixed-reality simulations effective tools for preparing business students for the real world?

Authentic learning and job readiness: Are mixed-reality simulations effective tools for preparing business students for the real world?

Mixed-Reality Simulation to Prepare Preservice Teachers to Use Culturally Responsive Teaching Strategies: A Pilot Study

By 2031, culturally and linguistically diverse students will represent approximately 55% of students enrolled in United States public schools (National Center for Educational Statistics, 2023). Despite increased ethnic diversity among students, most teachers are White. Consequently, many new teachers report feeling ill-equipped to implement culturally responsive teaching strategies to meet the needs of culturally and linguistically diverse students. To prepare culturally responsive preservice teachers, teacher preparation programs can embed technologies such as mixed-reality simulation paired with explicit instruction and performance feedback to support the preservice teachers’ implementation of culturally responsive teaching strategies. This mixed methods (convergent parallel design) pilot study evaluated how using mixed-reality simulation, explicit teaching, and performance feedback impacts preservice teachers’ knowledge of culturally responsive teaching strategies and self-efficacy. Results indicated that participants increased their knowledge, implementation, and self-efficacy. We discuss the findings and limitations and offer recommendations to improve mixed-reality simulation within teacher preparation programs and recommendations for future research.

A Comparison of Role-play v. Mixed-Reality Simulation on Pre-Service Teachers’ Behavior Management Practices

Students with disabilities are being educated in the general education setting at an ever-increasing rate; therefore, it becomes imperative that educator preparation programs ensure that pre-service teachers are equipped to meet the needs of all students. Teacher education scholars have encouraged educator preparation programs to maximize the potential of technology to improve the use of evidence-based and high-leverage practices. This study sought to examine the effects of role-play versus mixed-reality simulation on the development of pre-service teachers’ ability to implement high-probability practices while setting group expectations for behavior. Using a switching replications design, results suggested mixed-reality simulation was more effective than role-play on the development of pre-service teachers’ ability to implement high-probability practices (i.e., opportunities to respond, feedback) while setting group expectations for behavior. Additionally, student perception regarding the feasibility and acceptability of mixed-reality simulation to learn classroom management strategies was greater than that of role-play. Implications of results and future research are discussed.

Visualising mixed reality simulation for multiple users

Blended reality seeks to encourage co-presence in the classroom, blending student experience across virtual and physical worlds. In a similar way, Mixed Reality, a continuum between virtual and real environments, is now allowing learners to work in both the physical and the digital world simultaneously, especially when combined with an immersive headset experience. This experience provides innovative new experiences for learning, but faces the challenge that most of these experiences are single user, leaving others outside the new environment. The question therefore becomes, how can a mixed reality simulation be experienced by multiple users, and how can we present that simulation effectively to users to create a true blended reality environment? This paper proposes a study that uses existing screen production research into the user and spectator to produce a mixed reality simulation suitable for multiple users. A research method using Design Based Research is also presented to assess the usability of the approach.

Preparing General Education Teachers for Inclusive Settings: Integrating High-Leverage Practices and Mixed-Reality Simulation in Pre-Service Coursework

Preparing General Education Teachers for Inclusive Settings: Integrating High-Leverage Practices and Mixed-Reality Simulation in Pre-Service Coursework

Using Mixed Reality Simulation to Improve Junior Medical Trainees’ Preparedness to Manage High-Acuity Trauma

High-acuity trauma necessitates experienced and rapid intervention to prevent patient harm. However, upskilling junior trainees through hands-on management of real trauma cases is rarely feasible without compromising patient safety. This...

Pre-Service Teacher Candidates’ Perceptions of Classroom-Based Mixed-Reality Simulations

Background: Mixed-reality simulations (MRS) have been available for some time. However, teacher education programs in the United States are now introducing MRS as part of teacher training. Therefore, this study sought to determine teacher candidates’ perceptions of MRS and their possible benefits for education. Objectives: The purpose of this case study was to determine factors associated with a positive simulation experience, the simulation improvements or concerns, and what the teacher candidates learned from the live session. Methods: A qualitative methodological approach was employed. Feedback results were collected from 57 teacher candidates who participated in the MRS session, which were analyzed using an Excel document to identify the emergent themes. Results and Conclusions: The qualitative data revealed three themes: the real-life experiences were beneficial in acquiring pedagogical skills; the simulation was an effective training resource; and there was a need to improve the technology to ensure more realistic experiences. The simulation enables pre-service teachers to engage, think critically, and apply teaching skills with a small group of students. Conducting only one simulation was not enough to acquire knowledge on best teaching practices. Therefore, there is a need to implement additional MRS scenarios at the university level, so that teacher candidates can practice and feel confident teaching students in a safe environment.

GPT AI in Virtual Patient Facilitates Communication Training of Medical First Responders: A Usability Study in Mixed Reality Simulation (Preprint)

GPT AI in Virtual Patient Facilitates Communication Training of Medical First Responders: A Usability Study in Mixed Reality Simulation (Preprint)

Effects of team-based mixed reality simulation program in emergency situations.

This study aimed to demonstrate the effectiveness of a team-based mixed reality simulation program about emergencies. A nonequivalent control group pretest-posttest design was utilized. We recruited 32 nurses for the experimental group and 32 for the control group, resulting in a total of 61 nurses ultimately included as subjects for analysis. This mixed reality program is designed to facilitate responses in cardiopulmonary resuscitation situations simultaneously using four HoloLens. With shared content visible to all four trainees, the participants could engage in simulation while freely communicating. The experimental group engaged in cardiopulmonary resuscitation emergency simulation while wearing the HoloLens, while the control group was provided with written CPR materials. There were significant increases in the experimental group's critical thinking (p < .001), learning transfer motivation (p = .006), communication confidence (p = .033), and learning immersion (p < .001) compared to the control group. The program developed in this study presents an effective educational strategy that can enhance nurses' emergency competencies and leverage the practicality of mixed reality.

Enhancing Mixed Reality Simulation Training Technology with Real-Time Performance Visualization: Mixed Methods Study with Medical First Responders (Preprint)

Enhancing Mixed Reality Simulation Training Technology with Real-Time Performance Visualization: Mixed Methods Study with Medical First Responders (Preprint)

Special Education Teachers’ Preservice Experiences With Mixed-Reality Simulation: A Systematic Review

Mixed-reality simulation is a technology-based tool used to augment preservice teacher education in special education; as such, researchers have conducted several studies to assess mixed-reality simulation for its effectiveness and efficacy. Though these researchers have concluded that mixed-reality simulation can be an effective tool, there is limited information regarding a synthesis of the study characteristics (e.g., participants’ demographic information, amount of time, skills taught using simulation, and participants’ perceptions). Understanding these components is critical to the continued success and growth of using mixed-reality simulation in special education coursework. We identified and synthesized eight mixed-reality simulation studies with a total of 256 participants from 2005 to 2021. Together, the eight studies suggest that mixed-reality simulation is a valuable practice-based experience for preservice special educators. Implications for practice and future research are discussed.