Mixed Reality Platform Research Articles

Augmented reality in plastic surgery education

BackgroundMixed reality devices have made a big impact on the medical profession by creating a virtual world. Visual perception methods and 3D Touch can be used in teaching institutions to impart knowledge and help perform plastic surgery easilyMethodsThis study has been done to evaluate the systematic review and merits of Augmented Reality technology in plastic surgery for 3 years in the Department of Burn and Plastic Surgery in All India Institute of Medical Sciences, Rishikesh using PubMed Ovid and Google Scholar. This is a scoping review to compare the advent and utility of mixed reality platforms in plastic surgery in the world by searching the PubMed database.ResultsThe records of 55 articles could be retrieved in plastic surgery from Pubmed and Ovid database.26 were review articles,10 were on haptic devices,3 on 3D printing,15 case reports, and one RCT. Haptic devices are a part of training skills such as bone drilling, burring, and cutting. Education is enhanced by virtual reality platforms incorporating learning soft tissue skills with haptic devicesConclusionTechnological advancement in imaging modalities provides a convenient way of teaching anatomy, mapping of perforators to design flaps, and perform virtual osteotomies. Virtual consoles can be designed to teach endoscopic plastic surgery with 3D scanning.

The Role of Advanced Endoscopic Resection of Diverse Skull Base Malignancies: Technological Analysis during an 8-Year Single Institutional Experience.

Background Resection of skull base malignancies poses complex pathological and treatment-related morbidities. Recent technological advancements of endoscopic endonasal surgery (EES) offer the ability to reexamine traditional treatment paradigms with endoscopic procedures. The utility of EES was quantitatively examined in a longitudinal series with attention to morbidities and postoperative outcomes. Methods A single-center retrospective review was performed of all malignant sinonasal tumors from 2010 to 2018. Patients with purely EES were selected for analysis. Disease features, resection extent, complications, adjuvant treatment, recurrence, and survivability were assessed. Despite the mixed pathological cohort, analysis was performed to identify technical aspects of resection. Results A total of 68 patients (47.6% males and 52.4% females, average age: 60.3 years) were included. A diversity of histotypes included clival chordoma (22.1%), olfactory neuroblastoma (14.7%), squamous cell carcinoma (11.8%), and adenoid cystic carcinoma (11.8%). Gross total resection (GTR) was achieved in 83.8% of cases. Infection (4.4%) and cerebrospinal fluid leak (1.5%) were the most common postoperative complications. Total 46 patients (67.6%) underwent adjuvant treatment. The average time between surgery and initiation of adjunctive surgery was 55.7 days. Conclusion In our 8-year experience, we found that entirely endoscopic resection of mixed pathology of malignant skull base tumors is oncologically feasible and can be accomplished with high GTR rates. There may be a role for EES to reduce operative morbidity and attenuate time in between surgery and adjuvant treatment, which can be augmented through recent mixed reality platforms. Future studies are required to systematically compare the outcomes with those of open surgical approaches.

Letter: Emergency Response Plan During the COVID-19 Pandemic: The University of Alabama at Birmingham Experience.

Letter: Emergency Response Plan During the COVID-19 Pandemic: The University of Alabama at Birmingham Experience.

Use of Mixed Reality Visualization in Endoscopic Endonasal Skull Base Surgery.

Neuronavigation systems assist with spatial orientation during endoscopic transnasal skull base surgery, but they require a correlation of 3-dimensional (3D) views with 2-dimensional (2D) radiology studies. To outline an initial experience with a novel technology platform that provides intraoperative navigation using 3D reconstructions of patient anatomy for endoscopic surgery. A retrospective study of endoscopic anterior skull base and complex paranasal procedures was performed. Data from preoperative computed tomography and magnetic resonance imaging scans were fused to create 3D digital models of patient anatomy. Using the technology developed by Surgical Theater (Mayfield Village, Ohio), these reconstructions were designed to highlight particular anatomic regions of interest. The models were studied to guide the surgical approach and anticipate critical structures.The reconstructions were linked with the navigational technology created by Brainlab (Munich, Germany) during endoscopic surgery. A dynamic image of the reconstruction was displayed alongside a matching endoscopic camera view. These 2 views could be overlaid to provide an immersive, mixed reality image of the patient's anatomy. A total of 134 cases were performed. The pathologies included tumors of the anterior skull base or sinonasal cavity, inflammatory sinus disease, and cerebrospinal fluid leaks. Specific anatomic structures, such as the internal carotid arteries and optic nerves, were chosen for enhancement. Surgeons felt that the technology helped to guide the extent of bony dissection and to identify critical structures. We describe the first clinical series of complex skull base pathologies treated using a novel mixed reality platform.

Mixed reality as a time-efficient alternative to cadaveric dissection

Objectives: The extent of medical knowledge increases yearly, but the time available for students to learn is limited, leading to administrative pressures to revise and reconfigure medical school curricula. The goal of the present study is to determine whether the mixed reality platform HoloAnatomy represents an effective and time-efficient modality to learn anatomy when compared to traditional cadaveric dissection.Methods: This was a prospective, longitudinal study of medical students completing a musculoskeletal anatomy course at Case Western Reserve University School of Medicine. Participants were divided into two groups based on learning platform (HoloAnatomy versus traditional cadaveric dissection) and content area (upper limb versus lower limb anatomy). Time spent in lab and end of course practical exam scores were compared between groups.Results: The average study time of 48 medical students who completed study requirements was 4.564 h using HoloAnatomy and 7.318 h in the cadaver lab (p = 0.001). No significant difference was found between exam scores for HoloAnatomy and cadaver learners (p = 0.185).Conclusions: Our results indicate that HoloAnatomy may decrease the time necessary for anatomy didactics without sacrificing student understanding of the material.

The Design of Interplay Between Virtual Reality and Augmented Reality: A Case Study and Post Mortem of VRsus guARdian

VRsus guARdian is the result of an amalgamation of two different gameplay mediums that challenged the gap separating different mixed reality platforms, namely virtual reality (VR) and augmented reality (AR). The game utilized each medium’s approach towards immersion as a design principle in building a natural, asymmetric play for both players. The game also constructed a compelling fantasy atop each medium’s unique interactive capability to build a dynamic narrative. Lastly, the cross-platform nature of VRsus guARdian caused the game to be highly dependent upon Unity, a game engine with highly accessible, platform-agnostic development capabilities.

AnatomyMR: A Multi-User Mixed Reality Platform for Medical Education

Mixed Reality (MR) has opened up new means of imparting education, especially in the medical field. Students could view the whole human anatomy and it’s components at multiple levels of scale to learn their location and naming nomenclature, and also interact with them separately to familiarize themselves with their functions. In this paper we propose a platform for Medical education in MR which creates shared experiences, providing support for teaching within classrooms. The application would also work as a standalone experience, augmenting important information over the anatomy model for the student’s reference when a shared experience is not in session, thereby facilitating self-study. The platform would be built in such a way that the instructors would also have the option to add 3D models and relevant content to the existing courseware and have it rendered in the mixed reality space.

The impact of different play activity designs on students’ embodied learning

PurposeThis paper aims to build on work that has demonstrated the value of play or game-based learning environments and to further unpack how different kinds of play activities can support learning of academic concepts. To do so, this paper explores how students learn complex science concepts through collective embodied play by comparing two forms of play labeled as Inquiry Play and Game Play.Design/methodology/approachThis study builds off of previous research that uses the Science Through Technology Enhanced Play (STEP) technology platform (Authors et al., 2015). STEP is a mixed reality platform that allows learners to playfully explore science phenomena, such as the rules of particle behavior in solid, liquid and gas, through collective embodied activity. A combination of interaction analysis and qualitative coding of teacher and student interactions are used to examine patterns in the learning processes during embodied play activities.FindingsBoth forms of play led to similar learning gains. However, Inquiry Play promoted more emergent, flexible modeling of underlying mechanisms while Game Play oriented students more towards “winning”.Originality/valueBy contrasting play environments, this paper provides new insights into how different features of play activities, as well as how teachers orient their students according to these different features, support students’ learning in collective activity. As a result, these findings can provide insights into the design of future play-based learning environments that are intended to support the learning of academic concepts.

Making Augmented and Virtual Reality Work for the Plastic Surgeon.

Augmented and virtual reality is an evolving technology at the forefront of medicine. It can provide physicians with hands-free, real-time access to the vast resources of the Internet and electronic medical records, allowing simultaneously recording of clinical encounters or procedures. Mixed reality platforms can be applied as a clinical tool, educational resource, or as an aid in enhancing communication in health care. This article will explore how various augmented and virtual reality platforms have enabled real-time visualization of patient information, recording of surgical cases, point-of-view photography, and intraoperative consults-all while remaining sterile in the operating room. Although this technology is of potential value to a number of different surgical and medical specialties, plastic surgery is ideally suited to lead this charge.

Development of a Mixed Reality Platform for Lateral Skull Base Anatomy.

A mixed reality (MR) headset that enables three-dimensional (3D) visualization of interactive holograms anchored to specific points in physical space was developed for use with lateral skull base anatomy. The objectives of this study are to: 1) develop an augmented reality platform using the headset for visualization of temporal bone structures, and 2) measure the accuracy of the platform as an image guidance system. A combination of semiautomatic and manual segmentation was used to generate 3D reconstructions of soft tissue and bony anatomy of cadaver heads and temporal bones from 2D computed tomography images. A Mixed-Reality platform was developed using C# programming to generate interactive 3D holograms that could be displayed in the HoloLens headset. Accuracy of visual surface registration was determined by target registration error between seven predefined points on a 3D holographic skull and 3D printed model. Interactive 3D holograms of soft tissue, bony anatomy, and internal ear structures of cadaveric models were generated and visualized in the MR headset. Software user interface was developed to allow for user control of the virtual images through gaze, voice, and gesture commands. Visual surface point matching registration was used to align and anchor holograms to physical objects. The average target registration error of our system was 5.76 mm ± 0.54. In this article, we demonstrate that an MR headset can be applied to display interactive 3D anatomic structures of the temporal bone that can be overlaid on physical models. This technology has the potential to be used as an image guidance tool during anatomic dissection and lateral skull base surgery.

Using mixed reality displays for observational learning of motor skills: A design research approach enhancing memory recall and usability

When learning an action sequence, observing a demonstration informs the knowledge of movement execution and enhances the efficiency of motor skill acquisition. Three-dimensional (3D) virtual learning environments offer more opportunities for motor skill training as they afford observational learning. Mixed reality platforms (virtual reality, desktop PC, etc.) that render 3D virtual environments can therefore increase accessibility of observational content. To explore the effectiveness of these platforms so as to facilitate observational learning of action sequences, we developed the Recovery Position Application [ 1 ] (RPA) at the Interactive System Studio, University of Plymouth. The RPA was originally designed for mobile virtual reality. The RPA displays two virtual avatars performing the steps of the recovery position. We present the design of content and interaction informed by research into observational learning of motor skills. To formatively evaluate the current functional prototype, and potential use within an educational context, RPA was tested on three different platforms. Mobile VR (N=20), desktop PC (N=20) and video recording (N=21). Memory recall of movements was recorded and the usability of the RPA was investigated. Across all three platforms, the average recall of demonstrated information was 61.88%, after using the application for 10 min. No significant differences between recall rates were identified between platforms. Participant responses were positive or very positive for both application effectiveness as a learning resource and for ease of use. These results are discussed with regard to the future development of the RPA and guidelines for virtual demonstration content. Published: 27 November 2018 This paper is part of the special collection Mobile Mixed Reality Enhanced Learning , edited by Thom Cochrane, Fiona Smart, Helen Farley and Vickel Narayan. More papers from this collection can be found here . Citation: Research in Learning Technology 2018, 26 : 2129 - http://dx.doi.org/10.25304/rlt.v26.2129

User Experience of Augmented Reality System for Astronaut's Manual Work Support.

This paper introduces Augmented Reality (AR) system to support an astronaut's manual work, it has been developed in two phases. The first phase was developed in Europeans Space Agency's (ESA) project called “EdcAR—Augmented Reality for Assembly, Integration, Testing and Verification, and Operations” and the second phase was developed and evaluated within the Horizon 2020 project “WEKIT—Wearable Experience for Knowledge Intensive Training.” The main aim is to create an AR based technological platform for high knowledge manual work support, in the aerospace industry with reasonable user experience. The AR system was designed for the Microsoft HoloLens mixed reality platform, and it was implemented based on a modular architecture. The purpose of the evaluation of the AR system is to prove that reasonable user experience of augmented reality can reduce performance errors while executing a procedure, increase memorability, and improve cost, and time efficiency of the training. The main purpose of the first phase evaluation was to observe and get feedback from the AR system, from user experience point-of-view for the future development. The use case was a filter change in International Space Station (ISS)—Columbus mock-up in the ESA's European Astronaut Centre (EAC). The test group of 14 subjects it included an experienced astronaut, EAC trainers, other EAC personnel, and a student group. The second phase the experiment consisted of an in-situ trial and evaluation process. The augmented reality system was tested at ALTEC facilities in Turin, Italy, where 39 participants were performing an actual real astronaut's procedure, the installation of Temporary Stowage Rack (TSR) on a physical mock-up of an ISS module. User experience evaluation was assessed using comprehensive questionnaires, and interviews, gathering an in-depth feedback on their experience with a platform. This focused on technology acceptance, system usability, smart glasses user satisfaction, user interaction satisfaction, and interviews, gathering an in-depth feedback on their experience with a platform. The analysis of the questionnaires and interviews showed that the scores obtained for user experience, usability, user satisfaction, and technology acceptance were near the desired average. Specifically, The System Usability Scale (SUS) score was 68 indicating that the system usability is already nearly acceptable in the augmented reality platform.

When Mixed Reality Meets Internet of Things

From panoramic paintings and stereoscopic photos in the early 19th century, there has been a centurylong effort to realize mixed reality, interweaving real and virtual worlds that interact with each other. Recently, over the past few years, we have witnessed the first wave of "affordable" mixed reality platforms, such as Oculus Rift and Microsoft HoloLens hitting the market. In particular, 2017 was the showcase year of mixed reality technologies: The Academy awarded its first Oscar to virtual reality storytelling1; AAA caliber virtual reality games started to hit the market with impacts2. Furthermore, major mobile operating systems, including Android and iOS, began to support augmented reality at the platform level (e.g., Android ARCore, Apple ARKit). Looking down the road, a recent forecast by Orbis Research projects over $40 billion mixed reality market worldwide by 20203.

Accessible Play in Everyday Spaces

The advent of affordable and powerful mobile technology has allowed for explorations in mixed reality that merges virtual and physical space. However, the social and entertainment value and efficacy of mixed reality platforms for adult powered chair users has not been widely explored. In this article, we introduce the Mobility Games project, which aims to produce a series of inclusive entertainment technologies and services for people who use powered chairs. We describe our first offering: an accessible, social mixed reality game for co-located mobile play in everyday spaces. Findings from two exploratory field studies and a post hoc observer survey show that adult powered chair users found the game to be entertaining and used a variety of path strategies as they learned to play the game. An initial set of theoretically and empirically informed guidelines for making mobile mixed reality games accessible to adult powered chair users with diverse abilities is proposed.

“Learn what we’re going through”: attitudes of older powered chair users towards mixed reality games that involve power mobility

Older powered chair users' perceptions on and attitudes towards mixed reality and modern facilitating technologies, such as tablets and smartphones, was explored to inform the design of mixed reality games that involve power mobility. Eleven older powered chair users (aged 55 and over) were interviewed in focus groups about their knowledge of, adoption of, experiences with, attitudes about, preferences for, and interest in these topics. Questionnaire and interview data were analysed using descriptive statistics and thematic analysis. Findings suggest that community-dwelling older powered chair users are a technologically forward group that use or are interested in trying new technologies and interaction paradigms, with key barriers being identity mismatch, affordability, and social acceptability. Although they did not see their use of powered chairs, new technologies, and digital entertainment as an integrated system, participants were receptive to the idea of synchronizing assistive and non-assistive technologies for the purpose of social entertainment, advanced training, and/or promoting understanding and empathy in others. In particular, the concept of a game-based mixed reality platform designed around performance mastery for older powered chair users and empathy training for able-bodied friends and family members was well-received. Initial resistance due to identity mismatch or lack of knowledge was mitigated through explanation and examples.

Mixed reality web service platform

This paper presents a RESTful Web service platform for applications for both Web browsers and mobile clients. Having a common service backend makes creating applications fast, simple, and open to third parties. The paper presents two mixed reality applications that have been built on the platform. It summarizes requirements for a mixed reality platform and defines a mixed reality domain model that the platform and applications share. In addition, it describes how the clients can use the REST interface to perform operations on user-generated content, as well as access real-life commercial geo-content such as street-view panoramas and building models.

Simple Mixed Reality Infrastructure for Experimental Analysis of Route Choice Behavior

Microscopic traffic simulation models offer virtual reproduction of full-scale traffic networks with individual vehicle and driver resolution. However, relevant behavioral aspects, such as route-choice behavior, are based on some theoretical assumptions. Alternatively, driving simulators allow for direct testing of real subjects. However, the virtual driving environment is typically a fairly rudimentary representation of the road network and traffic conditions, focusing on the immediate surroundings of the test vehicle. As such, both tools were integrated to create a mixed reality traffic analysis environment. The objective is to enhance the credibility of in-lab-simulated route-choice experiments under various intelligent transportation system applications. The mixed reality system allows a human subject to “drive” a vehicle in a microscopic traffic simulation model of an actual physical network. An externally controlled driving capability was integrated into the widely used Quadstone Paramics microscopic traffic simulator. The developed mixed reality platform was used at the Toronto (Canada) Intelligent Transportation System Centre to test subjects while driving in a simulated model of a main downtown Toronto corridor. Drivers were given descriptive and prescriptive traffic information while their reactions were monitored. For comparative purposes, they were also tested using a more classical map-based point-and-click route-choice procedure. Results of the analysis highlight the potential of the developed mixed reality platform to enhance the realism of in-lab-simulated route-choice experiments and hence improve the credibility of collected data. Both the virtual reproduction of the choice environment and the tangible consequences of choice decisions contribute to an enhanced experimental environment.

Mixed-Reality Simulation of Minimally Invasive Surgeries

Our mixed-reality platform helps train surgeons in minimally invasive surgery and objectively assesses their performance. The platform uses multicamera stereo inside a patient manikin to measure the 3D positions of unmodified surgical instruments. It uses this information to drive a mixed-reality, computer-mediated learning system and provide objective measures of surgical skill.