Use Of Mixed Reality Technologies Research Articles

Application of mixed reality technology in free fibular flap transplantation for repairing mandibular defects

To explore the feasibility and effectiveness of mixed reality technology for localizing perforator vessels in the repair of mandibular defects using free fibular flap. Between June 2020 and June 2023, 12 patients with mandibular defects were repaired with free fibular flap. There were 8 males and 4 females, with an average age of 61 years (range, 35-78 years). There were 9 cases of ameloblastomas and 3 cases of squamous cell carcinomas involving the mandible. The disease duration ranged from 15 days to 2 years (median, 14.2 months). The length of mandibular defects ranged from 5 to 14 cm (mean, 8.5 cm). The area of soft tissue defects ranged from 5 cm×4 cm to 8 cm×6 cm. Preoperative enhanced CT scans of the maxillofacial region and CT angiography of the lower limbs were performed, and the data was used to create three-dimensional models of the mandible and lower limb perforator vessels. During operation, the mixed reality technology was used to overlay the three-dimensional model of perforator vessels onto the body surface for harvesting the free fibular flap. The length of the fibula harvested ranged from 6 to 15 cm, with a mean of 9.5 cm; the size of the flap ranged from 6 cm×5 cm to 10 cm×8 cm. The donor sites were sutured directly in 7 cases and repaired with free skin grafting in 5 cases. Thirty perforator vessels were located by mixed reality technology before operation, with an average of 2.5 vessels per case; the distance between the exit point of the perforator vessels located before operation and the actual exit point ranged from 1 to 4 mm, with a mean of 2.8 mm. All fibular flaps survived; 1 case had necrosis at the distal end of flap, which healed after dressing changes. One donor site had infection, which healed after anti-inflammatory dressing changes; the remaining incisions healed by first intention, and the grafts survived smoothly. All patients were followed up 8-36 months (median, 21 months). The repaired facial appearance was satisfactory, with no flap swelling. Among the patients underwent postoperative radiotherapy, 2 patients had normal bone healing and 1 had delayed healing at 6 months. In free fibular flap reconstruction of mandibular defects, the use of mixed reality technology for perforator vessel localization can achieve three-dimensional visualization, simplify surgical procedures, and reduce errors.

Virtual Laboratory in Mixed Reality with 3D Modelling for Sustainable Design

Immersive experiences through Mixed Reality (MR) are revolutionizing the way people connect and interact with each other and things around them. MR is the umbrella term that binds all spatial computing technologies which blends real and virtual environments. It holds the potential to provide things beyond imagination while simultaneously making deeper and more meaningful engagement possible. A new engineer to be equipped with the know-how of an OP (Operational Procedure) of machinery, this virtual laboratory helps to know the intricate details of the machinery and tools. The virtual laboratory combines the use of MR technologies, such as augmented reality (AR) and virtual reality (VR), with advanced 3D modeling tools. This virtual laboratory has the potential to revolutionize the sustainable design process by providing an immersive and flexible platform for experimentation, analysis, and collaboration. This project focuses on helping the user to visualize the machine (i.e., Single Drum Piston) and its functionalities through an immersive experience and understand the visual empathy of it. Through 3D Modelling technology, Plane detection and algorithms written with C# script we built this solution with the incorporation of Autodesk Maya and Unity Game Engine. The built solution can be experienced immersive by viewing it through a mobile placed in a headset (Aryzon Headset). This MR experience provides realistic virtual presentations which eases learning of complex modules while analytics ensure understanding of outcomes. It can empower designers and researchers to explore innovative design solutions, optimize resource utilization, and ultimately contribute to the development of more sustainable built environments. This can be incorporated in various other fields like automobile manufacturing, machine industries, construction tool manufacturing, real estate buildings & landscapes, home appliances and furniture, interior design, medical institution, and educational institutions.

The use of mixed reality technology within the donor collection experience.

Mixed reality (MR), a form of virtual reality (VR), provides an immersive and interactive experience for the user. Given VR's benefits in patients undergoing needle insertion procedures, MR's usability, impact on anxiety, and safety were evaluated in the blood donation setting. Whole blood donors ≥18 years old (yo) were enrolled at two blood centers and provided a MR headset with independently developed software to wear during blood donation. Pre- and post-donation questionnaires were conducted, and reaction data were reviewed. A post-study questionnaire was also completed by staff who assisted donors with MR. Descriptive statistics, bivariate analyses, and multinomial logistic regression were performed, and p values determined statistical significance between variables. A total of 282 donors completed the study. 84% wanted to try MR because it seemed fun/different/cool/interesting, and most staff (69%) and donors (68%) found MR easy to use. Baseline subjective anxiety, reported by 50.3% (more often in females, first-time donors, and donors <20 yo), was reduced by MR in 68.4% of donors, and there was a 3.6 times higher odds of anxiety reduction with MR. 54% of donors with baseline anxiety would use MR again with the highest future interest in young donors. Donor reactions while using MR were mild and included pre-faint reactions and hematomas. This study demonstrates the potential of MR in reducing donor anxiety, its feasibility during blood donation, and its safety in blood donors. MR is an innovative technology that holds promise to increase donor engagement, satisfaction, and retention.

From Augmented to Virtual Reality in Plastic Surgery: Blazing the Trail to a New Frontier.

Augmented reality (AR) and virtual reality (VR)-termed mixed reality-have shown promise in the care of operative patients. Currently, AR and VR have well-known applications for craniofacial surgery, specifically in preoperative planning. However, the application of AR/VR technology to other reconstructive challenges has not been widely adopted. Thus, the purpose of this investigation is to outline the current applications of AR and VR in the operative setting. The literature pertaining to the use of AR/VR technology in the operative setting was examined. Emphasis was placed on the use of mixed reality technology in surgical subspecialities, including plastic surgery, oral and maxillofacial surgery, colorectal surgery, neurosurgery, otolaryngology, neurosurgery, and orthopaedic surgery. Presently, mixed reality is widely used in the care of patients requiring complex reconstruction of the craniomaxillofacial skeleton for pre- and intraoperative planning. For upper extremity amputees, there is evidence that VR may be efficacious in the treatment of phantom limb pain. Furthermore, VR has untapped potential as a cost-effective tool for microsurgical education and for training residents on techniques in surgical and nonsurgical aesthetic treatment. There is utility for mixed reality in breast reconstruction for preoperative planning, mapping perforators, and decreasing operative time. VR has well- documented applications in the planning of deep inferior epigastric perforator flaps by creating three-dimensional immersive simulations based on a patient's preoperative computed tomography angiogram. The benefits of AR and VR are numerous for both patients and surgeons. VR has been shown to increase surgical precision and decrease operative time. Furthermore, it is effective for patient-specific rehearsal which uses the patient's exact anatomical data to rehearse the procedure before performing it on the actual patient. Taken together, AR/VR technology can improve patient outcomes, decrease operative times, and lower the burden of care on both patients and health care institutions.

The Impact of Occlusion on Depth Perception at Arm's Length.

This paper investigates the accuracy of Augmented Reality (AR) technologies, particularly commercially available optical see-through displays, in depicting virtual content inside the human body for surgical planning. Their inherent limitations result in inaccuracies in perceived object positioning. We examine how occlusion, specifically with opaque surfaces, affects perceived depth of virtual objects at arm's length working distances. A custom apparatus with a half-silvered mirror was developed, providing accurate depth cues excluding occlusion, differing from commercial displays. We carried out a study, contrasting our apparatus with a HoloLens 2, involving a depth estimation task under varied surface complexities and illuminations. In addition, we explored the effects of creating a virtual "hole" in the surface. Subjects' depth estimation accuracy and confidence were assessed. Results showed more depth estimation variation with HoloLens and significant depth error beneath complex occluding surfaces. However, creating a virtual hole significantly reduced depth errors and increased subjects' confidence, irrespective of accuracy enhancement. These findings have important implications for the design and use of mixed-reality technologies in surgical applications, and industrial applications such as using virtual content to guide maintenance or repair of components hidden beneath the opaque outer surface of equipment. A free copy of this paper and all supplemental materials are available at https://bit.ly/3YbkwjU.

P‐BB‐76 | The Use of Mixed Reality Technology within the Donor Collection Experience

P‐BB‐76 | The Use of Mixed Reality Technology within the Donor Collection Experience

Application and prospect of mixed reality technology in orthopedics

ABSTRACT In recent years, mixed reality (MR) technology has emerged as a new digital visualization tool and has been increasingly used in orthopedics. In comparison to traditional methods, MR provides a deeper integration of virtual information with the real world, offering users an intuitive display of anatomical details and facilitating the transition from two-dimensional to three-dimensional imaging data. With these significant features, MR has the potential to play a vital role in orthopedics. This article outlines the unique characteristics of MR and explores its technical advantages in doctor-patient communication, medical education, and surgical navigation within the field of orthopedics. This paper aims to inspire greater use of MR technology in the era of intelligent medicine.

Effect of Mixed Reality on Delivery of Emergency Medical Care in a Simulated Environment

Mixed-reality (MR) technology has the potential to enhance care delivery, but there remains a paucity of evidence for its efficacy and feasibility. To assess the efficacy and feasibility of MR technology to enhance emergency care delivery in a simulated environment. This pilot randomized crossover trial was conducted from September to November 2021 at a single center in a high-fidelity simulated environment with participants block randomized to standard care (SC) or MR-supported care (MR-SC) groups. Participants were 22 resident-grade physicians working in acute medical and surgical specialties prospectively recruited from a single UK Academic Health Sciences Centre. Data were analyzed from September to December 2022. Participants resuscitated a simulated patient who was acutely unwell, including undertaking invasive procedures. Participants completed 2 scenarios and were randomly assigned to SC or MR-SC for the first scenario prior to crossover. The HoloLens 2 MR device provided interactive holographic content and bidirectional audiovisual communication with senior physicians in the MR-SC group. The primary outcome was error rate assessed via the Imperial College Error Capture (ICECAP) multidimensional error-capture tool. Secondary outcomes included teamwork (Observational Teamwork Assessment for Surgery [OTAS]; range, 0-6 and Teamwork Skills Assessment for Ward Care [T-SAW-C]; range, 1-5), scenario completion, stress and cognitive load (NASA Task Load Index [NASA-TLX; range 0-100]), and MR device user acceptability. A total of 22 physicians (15 males [68.2%]; median [range] age, 28 [25-34] years) were recruited. MR technology significantly reduced the mean (SD) number of errors per scenario compared with SC (5.16 [3.34] vs 8.30 [3.09] errors; P = .003), with substantial reductions in procedural (0.79 [0.75] vs 1.52 [1.20] errors; P = .02), technical (1.95 [1.40] vs 3.65 [2.03] errors; P = .01), and safety (0.37 [0.96] vs 0.96 [0.85] errors; P = .04) domains. MR resulted in significantly greater scenario completion rates vs SC (22 scenarios [100%] vs 14 scenarios [63.6%]; P = .003). It also led to significant improvements in the overall quality of teamwork and interactions vs SC as measured by mean (SD) OTAS (25.41 [6.30] vs 16.33 [5.49]; P < .001) and T-SAW-C (27.35 [6.89] vs 18.37 [6.09]; P < .001) scores. As reported via mean (range) NASA-TLX score, there were significant reductions for MR-SC vs SC in participant temporal demands (38 [20-50] vs 46 [30-70]; P = .03) and significant improvements in self-reported task performance (50 [30-60] vs 39 [10-70]; P = .01). Overall, 19 participants (86.4%) reported that they were more confident in making clinical decisions and undertaking clinical procedures with MR support. This study found that the use of MR technology reduced error, improved teamwork, and enhanced practitioner confidence when used to support the delivery of simulated emergency medical care. ClinicalTrials.gov Identifier: NCT05870137.

Familiarization with Mixed Reality for Individuals with Autism Spectrum Disorder: An Eye Tracking Study.

Mixed Reality (MR) technology is experiencing significant growth in the industrial and healthcare sectors. The headset HoloLens 2 displays virtual objects (in the form of holograms) in the user's environment in real-time. Individuals with Autism Spectrum Disorder (ASD) exhibit, according to the DSM-5, persistent deficits in communication and social interaction, as well as a different sensitivity compared to neurotypical (NT) individuals. This study aims to propose a method for familiarizing eleven individuals with severe ASD with the HoloLens 2 headset and the use of MR technology through a tutorial. The secondary objective is to obtain quantitative learning indicators in MR, such as execution speed and eye tracking (ET), by comparing individuals with ASD to neurotypical individuals. We observed that 81.81% of individuals with ASD successfully familiarized themselves with MR after several sessions. Furthermore, the visual activity of individuals with ASD did not differ from that of neurotypical individuals when they successfully familiarized themselves. This study thus offers new perspectives on skill acquisition indicators useful for supporting neurodevelopmental disorders. It contributes to a better understanding of the neural mechanisms underlying learning in MR for individuals with ASD.

Exploring the Clinical Benefits of Mixed-Reality Technology for Breast Lumpectomy

This study determined the value of mixed-reality (MR) technology for doctor-patient communication, preoperative planning, intraoperative navigation, and tumor localization in treating patients with breast cancer. Fifty-eight patients with breast space-occupying lesions (16 benign and 42 malignant) who underwent breast lumpectomy at the People’s Liberation Army General Hospital of China were included in this study. The patients were randomly divided into the MR group and the computed tomography (CT) group. In the MR group, a 3D reconstruction of whole-breast ultrasound was used to localize the spatial position of the breast lesion and was combined with the 3D reconstruction of breast MRI to determine the lesion boundaries. To improve the preciseness of surgery, a postoperative survey was conducted. The MR group exhibited a higher level of patient knowledge regarding the disease, treatment, and diagnosis (26.207 ± 1.698 points) than the CT group (19.228 ± 4.889 points) (T = 7.033; P &lt; 0.01 ), and patient satisfaction with surgical results, treatment confidence, and communication methods (4.448 ± 0.572) was also higher than that of the CT group (3.172 4.448 ± 0.572) 0.602) P &lt; 0.05 . In addition, doctors were significantly more satisfied with surgical planning and intraoperative localization when the MR technique was used (T = 8.273; P &lt; 0.01 ). The use of MR technology in lumpectomy has improved patients’ understanding of surgical procedures and surgical results and has achieved positive results. This technique may provide clinical benefits.

The use of mixed reality technology for the objective assessment of clinical skills: a validation study

BackgroundMixed Reality technology may provide many advantages over traditional teaching methods. Despite its potential, the technology has yet to be used for the formal assessment of clinical competency. This study sought to collect validity evidence and assess the feasibility of using the HoloLens 2 mixed reality headset for the conduct and augmentation of Objective Structured Clinical Examinations (OSCEs).MethodsA prospective cohort study was conducted to compare the assessment of undergraduate medical students undertaking OSCEs via HoloLens 2 live (HLL) and recorded (HLR), and gold-standard in-person (IP) methods. An augmented mixed reality scenario was also assessed.ResultsThirteen undergraduate participants completed a total of 65 OSCE stations. Overall inter-modality correlation was 0.81 (p = 0.01), 0.98 (p = 0.01) and 0.82 (p = 0.01) for IP vs. HLL, HLL vs. HLR and IP vs. HLR respectively. Skill based correlations for IP vs. HLR were assessed for history taking (0.82, p = 0.01), clinical examination (0.81, p = 0.01), procedural (0.88, p = 0.01) and clinical skills (0.92, p = 0.01), and assessment of a virtual mixed reality patient (0.74, p = 0.01). The HoloLens device was deemed to be usable and practical (Standard Usability Scale (SUS) score = 51.5), and the technology was thought to deliver greater flexibility and convenience, and have the potential to expand and enhance assessment opportunities.ConclusionsHoloLens 2 is comparable to traditional in-person examination of undergraduate medical students for both live and recorded assessments, and therefore is a valid and robust method for objectively assessing performance. The technology is in its infancy, and users need to develop confidence in its usability and reliability as an assessment tool. However, the potential to integrate additional functionality including holographic content, automated tracking and data analysis, and to facilitate remote assessment may allow the technology to enhance, expand and standardise examinations across a range of educational contexts.

Augmented Reality ASL for 11:11 at Theatre Passe Muraille

In November 2020, Theatre Passe Muraille produced a workshop as a part of their Accessibility Labs (funded through the Toronto Arts Council’s Open Door Project). This workshop connected the development of Samson Bonkeabantu Brown’s play 11:11, directed by Tsholo Visions Khalema and choreographed by Mafa Makhubalo, with a collaboration between technology collectives Toasterlab and Cohort to experiment with the use of augmented reality to support the American Sign Language interpretation and live captioning in consultation with Courage Bacchus, Marcia Adolphe, and Carmelle Cachero, Jenelle Rouse, and Gaitrie Persaud. The collaborators spent a week building an interpretation distribution system that used affordable technology to create a proof-of-concept experiment to see what it would look like to move both text and sign interpretation from just offstage into the visual field of the performance. Using Cohort’s media distribution app developed for the synchronous delivery of media to mobile devices, this project explored live and pre-recorded versions of performance interpretation for a Deaf audience. These were displayed on the audience’s phone screens and in simple augmented reality headsets. The goal was to explore expanding the opportunities for an audience to access interpretation, making the experience more customizable through different forms of engagement, and seeking to provide more universal access across all performances by making a recorded alternative available. The workshop also explored the dramaturgical and scenographic implications of adding this information into the direct visual field of an audience member. This article, in the form of a compiled oral history of the workshop, documents the process, the findings, and the follow-up questions that the team identified over our short time together to provide a baseline for further exploration into the use of mixed-reality technologies in support of accessible performance spaces. It considers situated identity and Deaf culture in relationship to translating interpreted performance to a technological solution as it both outlines the practical steps that allowed this to happen and explores artist, interpreter, and technologist perspectives on what was learned.

The Space between the Screen and Architecture

This article presents the historical and contemporary relationship that cinema and urban-architectural space have developed. Its purpose is to show the reciprocal influence between objective and virtual reality projected in science fiction films like Blade Runner, by director Ridley Scott (1982), and Ready Player One, by Steven Spielberg (2011), regarding the current perception of architectural space. The idea behind establishing the relationship between these films is highlighting the prospective used by their directors around user-architecture interaction and technology. Some of the literary and cinematographic works that address the topic of postmodern cities and the cities of the future are cited as background, either from a Utopian vision or from the perspective of a dystopia. Methodologically, a critical review of both films was carried out based on an analysis layout on the conception of architecture through its scenography or staging (Zavala, 2003), as well as the similarity with the way in which we currently travel through the architectural space with the use of mixed reality technology. In this way, the results show how mixed reality (physical and virtual) influences the perception and behavior of users within physical spaces. The conclusion expresses the inescapable presence of mixed reality in spaces as complex as those of cinema and architecture.

Exploring L2 Teacher Identities in an Intercultural Telecollaborative Mixed-Reality Teaching Environment

This study investigates the identities produced by three L2 teachers of different backgrounds and experiences in an intercultural telecollaborative project that integrated the use of mixed-reality technology. Content-analysis (Hoffman et al., 2011) and multimodal (inter)action analysis (Norris, 2011) were employed to identify the identity elements produced during their multimodal interactions. Using Darvin and Norton’s (2015) model of investment (MOI) as an interpretative lens, this study uncovers the different identities held by the participants, and the ways in which they aspired to become good teachers. Findings also show that mixed-reality simulated teaching provided a safe environment for exercising professional agency, thereby facilitating identity development and promoting the sense of getting there. Interactive reflection provided a site of struggle where the value of capital and imagined identities shifted. Dialoguing among the participants allowed them to reassess the value of pre-acquired capital and move to (re)imagining new teacher identities and restructuring ideological structures. Mixed-reality teaching and intercultural professional learning sensitizes the less experienced teachers to the socially and culturally complex roles required for L2 teachers. The mixed-reality simulation technology still has its limitations, mainly where classroom interaction is concerned, because the number of avatar students is small and they cannot move around physically.

Intraoperative Use of Mixed Reality Technology in Median Neck and Branchial Cyst Excision

The paper considers the possibilities, prospects, and drawbacks of the mixed reality (MR) technology application using mixed reality smartglasses Microsoft HoloLens 2. The main challenge was to find and develop an approach that would allow surgeons to conduct operations using mixed reality on a large scale, reducing the preparation time required for the procedure and without having to create custom solutions for each patient. Research was conducted in three clinical cases: two median neck and one branchial cyst excisions. In each case, we applied a unique approach of hologram positioning in space based on mixed reality markers. As a result, we listed a series of positive and negative aspects related to MR surgery, along with proposed solutions for using MR in surgery on a daily basis.

Effects of Nursing Simulation Using Mixed Reality: A Scoping Review.

Mixed reality (MR) has recently been suggested as a new educational tool for nursing simulation. However, few studies have assessed the use and evaluation of MR nursing simulations. Therefore, this review identified studies of MR nursing simulations through a scoping review using the framework proposed by Arksey and O’Malley. The studies reviewed were found through DBpia, RISS, PubMed, CINAHL, and Google Scholar. Each study was analyzed, and data were abstracted into publication characteristics, simulation program details, device use, and simulation evaluation. A total of 10 studies were reviewed. Senses that were mainly used in MR nursing simulation included audition and haptics via motion, in addition to visual information. Simulations were evaluated using various outcome variables. Knowledge was most commonly evaluated, followed by clinical performance and satisfaction. This study is significant as it analyzed trends in research on MR nursing simulations in Korea and other countries and suggested directions for the use of MR technology in nursing simulations based on the findings. Additional studies are required to develop scenarios for the effective use of MR in nursing education and to evaluate the effects of MR nursing simulations.

A New Model for Image Segmentation Based on Deep Learning

Image segmentation of the medical image and its conversion into anatomical models is an important technique and main point in computer vision (CV) and image processing (IP), training tools that are used routinely in the fields of medicine and surgery. Segmenting images and converting them into a model that depends on its work on the different algorithms and the extent of technological advancement and method of application. The advancement of segmentation algorithms has led to the possibility of creating three-dimensional models for the patient to study without endangering his life. This paper describes a combination of two fields of solving segmentation problem to convert through the workflow of a hybrid algorithm structure Convolutional neural network (CNN, Active Contour &amp;amp; Deep Multi-Planar) and seg3d2 to switch DICOM medical rays “Digital Imaging and Communications in Medicine” into a 3Dimintional model, using data from active contour to be the input of deep learning. This research will be using are human liver DICOM images and is divided into two stages (medical image segmentation - retinal model optimization). This is to help doctors and surgeons to study the patient’s condition with accuracy and efficiency through the use of mixed reality technology in liver surgery [living donor liver transplantation (LDLT)], all implement by Seg3D2 and Python.

COVID-19 and the Digitalisation of Cardiovascular Training and Education-A Review of Guiding Themes for Equitable and Effective Post-graduate Telelearning.

The coronavirus disease-2019 (COVID-19) pandemic has had an unprecedented impact leading to novel adaptations in post-graduate medical education for cardiovascular and general internal medicine. Whilst the results of initial community COVID-19 vaccination are awaited, continuation of multimodality teaching and training that incorporates telelearning will have enduring benefit to post-graduate education and will place educational establishments in good stead to nimbly respond in future pandemic-related public health emergencies. With the rise in innovative virtual learning solutions, medical educators will have to leverage technology to develop electronic educational materials and virtual courses that facilitate adult learning. Technology-enabled virtual learning is thus a timely progression of hybrid classroom initiatives that are already adopted to varying degrees, with a need for faculty to serve as subject matter experts, to host and moderate online discussions, and to provide feedback and overall mentorship. As an extension from existing efforts, simulation-based teaching (SBT) and learning and the use of mixed reality technology should also form a greater core in the cardiovascular medicine curriculum. We highlight five foundational themes for building a successful e-learning model in cardiovascular and general post-graduate medical training: (1) digital solutions and associated infrastructure; (2) equity in access; (3) participant engagement; (4) diversity and inclusion; and (5) patient confidentiality and governance framework. With digitalisation impacting our everyday lives and now how we teach and train in medicine, these five guiding principles provide a cognitive scaffold for careful consideration of the required ecosystem in which cardiovascular and general post-graduate medical education can effectively operate. With due consideration of various e-learning options and associated infrastructure needs; and adoption of strategies for participant engagement under sound and just governance, virtual training in medicine can be effective, inclusive and equitable through the COVID-19 era and beyond.

Heart Disease Simulation with Mixed Reality Technology

Heart disease is one of the leading causes of death in the world. Various types of heart disease include Coronary Heart Disease, Arrhythmias, Heart Failure, Heart Valves and Heart Muscle. The Ministry of Health of the Republic of Indonesia calls on the public to be smarter in responding to the terror of heart disease. One of them through the introduction and early prevention of heart disease. As time has evolved, technology has become part of society. One of the things that can be done to prevent and recognize heart disease is to utilize the use of Mixed Reality technology as a medium for learning and learning. The use of Mixed Reality technology used is a device from Microsoft is namely Hololens. By using this tool, the developer can build applications to display 3D objects and their interactions. To interact, hololens use a targetting mechanism based on the view from the user’s side, a circular viewfinder as a cursor that can be directed with head movements and 2 gestures namely airtap and bloom by the user directing the hand forward until the sensor is detected from Hololens. In this case, the researchers visualize the type of heart disease in 3D and interact by applying hand gestures or voice input as interactions.

Developing Interpersonal and Counseling Skills Through Mixed-Reality Simulation in Communication Sciences and Disorders

Purpose Simulation is increasingly becoming a valuable tool for training and educating students in communication sciences and disorders (CSD). The purpose of this study is to examine the impact of a mixed-reality simulation on CSD students' perceptions of their ability to apply clinical and counseling skills. Additionally, this study sought to investigate the overall efficacy and acceptance of this type of clinical simulation experience for undergraduate CSD students. Method A total of 29 undergraduate students participated in a clinical simulation experience that used actor-controlled avatars in a mixed-reality simulation environment to practice collecting case history information and delivering diagnostic news to parents of a child client. All students completed pre- and postsimulation questionnaires that rated the impact of the clinical simulation experience on their confidence in demonstrating targeted clinical skills and their general attitudes about their participation. Additionally, five lead participants participated in follow-up interviews to gather data to best describe students' perceptions. Quantitative and qualitative data were recorded and analyzed. Results The majority of the students exhibited positive attitudes toward the mixed-reality clinical simulation experience and reported an increase in their perceived ability to apply several counseling skills (e.g., listening and selective feedback) following the session. Analysis showed that the perceived confidence levels on seven out of the 17 targeted skill items increased on the postevent questionnaire to a level of statistical significance and that the simulation experience was described as meaningful and supportive for increasing confidence. Conclusions Mixed-reality clinical simulation may be a useful tool for teaching interpersonal communication and counseling skills for students, including undergraduates, in CSD. Additionally, the use of mixed-reality technology in this study produced similar results seen with other clinical simulation methods such as standardized patients or computer-based simulations.