Mixed Reality Research Articles

Narrative Inquiry into the Challenges of Implementing XR Technologies in Education: Voices from Future Hub 2023

This research explores the multifaceted challenges and opportunities inherent in the integration of Extended Reality (XR) technologies—comprising Augmented Reality (AR), Virtual Reality (VR), and Mixed Reality (MR)—within educational contexts. Utilizing a narrative inquiry methodology, this study draws upon the experiences and perspectives of participants at the Future Hub Forum 2023, a gathering of researchers, educators, and technologists specializing in immersive learning technologies. Through the collection and analysis of narrative materials, including interviews, presentations, and observational data, the study aims to elucidate the complex interplay of technological, pedagogical, and infrastructural challenges that influence the adoption and effective use of XR technologies in education. Key findings reveal significant barriers to widespread XR integration, including issues of access and affordability, which exacerbate existing digital divides; challenges in training and securing buy-in from stakeholders; content creation and deployment hurdles; and infrastructural and administrative obstacles. Despite these challenges, the study also uncovers a collective optimism about the transformative potential of XR technologies to revolutionize pedagogical approaches and learning outcomes, provided these barriers can be overcome. The implications of this research extend beyond the immediate context of XR technology implementation, offering insights into broader questions of educational equity, the evolving landscape of digital pedagogy, and the future of immersive learning experiences. By highlighting the voices and experiences of those at the forefront of XR adoption in education, this paper contributes to a deeper understanding of the conditions necessary for the successful integration of these technologies into diverse educational settings. Through its narrative approach, the study also underscores the value of storytelling and personal narratives in uncovering the nuanced realities of educational technology adoption, offering a rich, contextualized perspective that quantitative methods alone may not capture.

Is Native Naïve? Comparing Native Game Engines and WebXR as Immersive Analytics Development Platforms.

Native game engines have long been the 3-D development platform of choice for research in mixed and augmented reality. For this reason, they have also been adopted in many immersive visualization and immersive analytics systems and toolkits. However, with the rapid improvements of WebXR and related open technologies, this choice may not always be optimal for future visualization research. In this article, we investigate common assumptions about native game engines versus WebXR and find that while native engines still have an advantage in many areas, WebXR is rapidly catching up and is superior for many immersive analytics applications.

Immersive Reality in Education: Transforming Teaching and Learning through AR, VR, and Mixed Reality Technologies

Background and Aims: Understanding how immersive technologies like AR, VR, and MR can transform education by enabling interactive and experiential learning. By addressing adoption challenges and highlighting successful case studies, it aims to help educators and policymakers effectively integrate these technologies while promoting equitable access and informed decision-making. Thus, this paper aims to explore the role of AR, VR, and MR in enhancing learning experiences. Methodology: The methodology ensures a thorough review by taking a systematic approach to data collection and analysis from various sources, with a focus on recent advances in immersive technologies. By combining qualitative and quantitative analyses, the paper aims to provide a comprehensive overview of how AR, VR, and MR affect educational practices and outcomes. Results: Augmented Reality (AR), Virtual Reality (VR), and Mixed Reality (MR) improve education by making difficult concepts more accessible and engaging. AR makes abstract concepts tangible, VR provides immersive experiences for deeper understanding, and MR connects the digital and physical worlds. These technologies work together to create interactive learning environments that meet a variety of learning needs, promote critical thinking, and encourage creativity. Conclusion: Augmented Reality (AR), Virtual Reality (VR), and Mixed Reality (MR) all improve education by making complex concepts more understandable and engaging through interactive experiences. These technologies create a dynamic learning environment by combining AR's tangibility, VR's immersion, and MR's integration of the digital and physical worlds.

FEATURES OF ORGANIZING STEAM EDUCATIONAL PROJECTS USING IMMERSIVE TECHNOLOGIES

Formulation of the problem. The rapid development of technologies in the modern world opens up new horizons for education. Integrating science, technology, engineering, arts, and mathematics (STEAM) holds particular significance as a transformative approach, fostering interdisciplinary learning and preparing students for the challenges of a knowledge-based and digitized society. The transition from a traditional approach to STEAM education becomes even more captivating with the incorporation of immersive technologies. This direction, merging technical and creative aspects, requires an innovative approach to the organization of educational projects, considering the new opportunities and challenges presented by virtual reality (VR), augmented reality (AR), and mixed reality (MR). Materials and methods. To achieve the goal of our research, we used the following methods: systematic and comparative analysis of pedagogical, psychological, philosophical, and sociological works, methodic literature, and analysis of the pedagogical experience of using immersive technologies to organize STEAM projects at school; synthesis and generalization to formulate the main provisions of the study; interviews and questionnaires of teachers regarding their understanding and attitude to immersive technologies as a means of implementing STEAM projects at school; interpretation of research results. We analyzed 18 scientific sources, including 14 Scopus-indexed articles, two reports (the educational report from Horizon 2020 and the "State of the Industry Report" from the XR Association), and two scientific studies by Ukrainian researchers. Eighty-seven teachers participated in interviews and surveys conducted through Google Forms. Results. Creating and organizing a STEAM educational project using immersive technologies involves the following teacher actions: forming a project name by the student's learning goals, educational content, and the project result as its product; creating a reference project; determining the subject, topic, age of students, project preparation time, learning time, essential immersive technologies; forming the purpose of the first lesson and project tasks to immerse and motivate students about the project; identifying problematic issues that should reflect the actual context or problem. We have characterized teachers' attitudes towards immersive technologies when organizing and conducting such projects. Based on the study's results, we have identified the features of organizing STEAM projects using immersive technologies: expanding the field of creativity with AR and VR, immersion in scientific concepts and conducting experiments through the use of virtual laboratories, visualizing abstract ideas through the use of augmented reality, interacting with technology through mixed reality projects, and teamwork in immersive environments. Conclusions. Immersive technologies in STEAM education have advantages (e.g., gaining practical experience, forming interdisciplinary connections through interdisciplinary learning, cooperation, communication, adaptive environment, and request for creativity) in teaching.

Use of Fluorescence Imaging in Liver Transplant Surgery.

Liver transplant surgery is a complex procedure that demands high knowledge of surgical anatomy and the precise recognition and preservation of structures. To address this, the use of fluorescence imaging has facilitated the identification of anatomical structures such as biliary ducts, arteries, and liver segmentation. Indocyanine green is among the most commonly utilized fluorescent agents, not just during surgery but also in the pre- and postoperative phases, where it is used to assess graft failure by measuring the plasma disappearance rate. New advancements such as artificial intelligence paired with fluorescence imaging have the potential to enhance patient outcomes. Additionally, technologies such as augmented reality and mixed reality could be integrated into surgical procedures, broadening the scope of possibilities for improving patient safety.

Mixed Reality-Based Navigation for Pedicle Screw Placement: A Preliminary Study Using a 3D-Printed Spine Model.

Background and objectives Mixed reality (MR) is one of the image processing technologies that allows the user to manipulate three-dimensional (3D) virtual images (hologram). The aim of this study was to evaluate the accuracy of MR-based pedicle screw (PS) placement using 3D spine models. Materials and methods Using the preoperative CT data of a patient with adolescent idiopathic scoliosis (AIS) who had undergone posterior spinal fusion in our hospital, a 3D-printed spine model was created. On the other hand, a 3D hologram of the same patient was automatically created using the preoperative CT data uploaded to the Holoeyes MD service website (Holoeyes Inc., Tokyo, Japan). Using a Magic Leap One® headset (Magic LeapInc., Plantation, FL), the 3D hologram with lines of predetermined PS trajectories was superimposed onto the 3D-printed spine model and PS were inserted bilaterally along with the trajectory lines from T5 to L3. As a control, we used a readymade 3D spine model of AIS and inserted PS bilaterally with a freehand technique from T4 to L3. The rate of pedicle violation was compared between the MR-based and freehand techniques. Results A total of 22 and 24 PS were placed into the 3D-printed spine model of our patient and the readymade 3D spine model, respectively. The rate of pedicle violation was 4.5% (1/22 screws) in the MR-based technique and 29.2% (7/24 screws) in the freehand technique (P = 0.049). Conclusions We demonstrated a significantly lower rate of PS misplacement in the MR-based technique than in the freehand technique. Therefore, an MR-assisted system is a promising tool for PS placement in terms of feasibility, safety, and accuracy, warranting further studies including cadaveric and clinical studies.

Innovative Applications of XR in Architectural Sustainability: Current Trends and Future Prospects

Due to the limited global resources and the threat of climate change, sustainable development has become a critical issue. This study aims to delve into the various applications of Extended Reality (XR), including Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR), in sustainable architectural design. Through virtual experiences, visualisation of architectural ecosystems, BIM integration, and environmental simulations, XR technology provides designers with a more comprehensive dataset and perspective. However, despite showcasing the potential of XR technology, further research and practical implementation are necessary to deepen our understanding of its role in architectural sustainability and address potential technological and application challenges. The article emphasizes the need to holistically consider multiple dimensions of data when applying XR technology in architectural design, enabling a more comprehensive evaluation of the sustainability of design solutions.

Exploring Data Input Problems in Mixed Reality Environments: Proposal and Evaluation of Natural Interaction Techniques

Data input within mixed reality environments poses significant interaction challenges, notably in immersive visual analytics applications. This study assesses five numerical input techniques: three benchmark methods (Touch-Slider, Keyboard, Pinch-Slider) and two innovative multimodal techniques (Bimanual Scaling, Gesture and Voice). An experimental design was employed to compare these techniques’ input efficiency, accuracy, and user experience across varying precision and distance conditions. The findings reveal that multimodal techniques surpass slider methods in input efficiency yet are comparable to keyboards; the voice method excels in reducing cognitive load but falls short in accuracy; and the scaling method marginally leads in user satisfaction but imposes a higher physical load. Furthermore, this study outlines these techniques’ pros and cons and offers design guidelines and future research directions.

Extended Reality (XR) Toward Building Immersive Solutions: The Key to Unlocking Industry 4.0

When developing XR applications for Industry 4.0, it is important to consider the integration of visual displays, hardware components, and multimodal interaction techniques that are compatible with the entire system. The potential use of multimodal interactions in industrial applications has been recognized as a significant factor in enhancing humans’ ability to perform tasks and make informed decisions. To offer a comprehensive analysis of the current advancements in industrial XR, this review presents a structured tutorial that provides answers to the following research questions: (R.Q.1) What are the similarities and differences between XR technologies, including augmented reality (AR), mixed reality (MR), Augmented Virtuality (AV), and virtual reality (VR) under Industry 4.0 consideration? (R.Q.2) What types of visual displays and hardware devices are needed to present XR for Industry 4.0? (R.Q.3) How did the multimodal interaction in XR perceive and relate to Industry 4.0? (R.Q.4) How have modern adaptations of XR technologies dealt with the theme of Industry 4.0? (R.Q.5) How can XR technologies in Industry 4.0 develop their services and usages to be more solution-inclusive? This review showcases various instances that demonstrate XR’s potential to transform how humans interact with the physical world in Industry 4.0. These advancements can increase productivity, reduce costs, and enhance safety.

Exploration of Animation Design Collaboration Mechanism Combining Mixed Reality and Intelligent CAD System

Exploration of Animation Design Collaboration Mechanism Combining Mixed Reality and Intelligent CAD System

Ghost image analysis for pancake virtual reality systems

Pancake optics (also known as polarization-based catadioptric system) has been widely used as the imaging lens for virtual reality (VR) and mixed reality (MR) headsets because of its compact formfactor and excellent image quality. However, such a folded pancake optics not only dramatically lowers the optical efficiency to 25% because of the utilization of a half mirror, but also suffers from ghost images due to the stray light from multiple surface reflections and imperfect polarization control inside the optical system. In this paper, the origins including the light paths of the ghost images are explored by experiment and then analyzed by simulation. The effect of different incident angles on the intensity of each ghost is also investigated.

Server-enabled mixed reality for flood risk communication: On-site visualization with digital twins and multi-client support

This study introduces a novel mobile mixed reality (MR) system to improve communication among stakeholders regarding climate-change-driven flood risk. This MR system enables stakeholders to visualize flood risks on common mobile devices, such as smartphones and tablets, offering perspectives from their respective environments. The system overcomes the computational limitations of previous mobile MR systems by rendering the MR images on a server. Furthermore, distributing users across multiple servers enables many users to simultaneously perform server-enabled MR, facilitating widespread engagement in flood risk communication. The system uses a viewport transformation to accurately display virtual floods and correct for real-time posture changes. A prototype tested in a high-risk flood area achieved real-time MR visualization at more than 16 frames per second, even on multiple devices. Future research will aim to promote stakeholder participation through more accurate visualization of virtual floods and improved interactive information sharing among users.

Revolutionizing Higher Education Teaching Evaluation with Interactive Mobile and Mixed Reality: A Methodological and Applied Analysis

In higher education, it is imperative to maintain educational quality, which requires necessitates a thorough evaluation of teaching methods. The advent of mixed reality (MR) technology, combined with interactive mobile capabilities, introduces innovative possibilities for traditional educational assessment frameworks. This study is dedicated to investigating the application of MR and interactive mobile technologies in higher education teaching evaluations, with a focus on assessing their effectiveness and implementation outcomes. Research to date has explored MR’s educational applications. However, the integration of interactive mobile technologies alongside MR in developing specific methodologies and evaluative tools for teaching evaluations has not been fully realized. This underutilizes the combined potential of these technologies. This paper is anchored in two principal research endeavors. Initially, it delves into the construction of a meta-model facilitated by MR-aided and interactive mobile-enhanced teacher-student interactions, utilizing a hierarchical remote interaction pyramid model. This process involves developing extensible associative functions, determining index weights, and establishing evaluation levels, which collectively enhance the evaluation’s multidimensionality, interactivity, and scientific precision. Subsequently, the analytic hierarchy process (AHP) is employed to quantitatively assess the educational impact of MR and interactive mobile experiences on students. This approach provides support for customized and precise teaching evaluations. The findings reveal that integrating MR technology with interactive mobile capabilities significantly enhances the interactivity and systematic scientific approach of teaching evaluations. This fosters a more multidimensional and interactive framework for educational evaluation.

Performance Analysis of Meta-UI on Membatik Mixed Reality Application Using Nielsen’s Heuristic Method

Moving beyond immersive learning, the rapid advancement of Mixed Reality (MR) technology is diversifying its applications across various sectors, including education, healthcare, and corporate training. However, standalone MR devices pose challenges related to performance and visualization realism. In this study, we focused on enhancing immersive learning by optimizing the User Interface (UI) of an MR batik-making application using the Meta-UI principle. This research aimed to assess the impact of these UI approaches on user satisfaction, measured through Nielsen's usability methods. User feedback was collected and analyzed to refine the UI design. Nielsen's usability heuristics guided the evaluation of UI attributes such as clarity, efficiency, and user satisfaction. The utilization of Meta-UI demonstrated a notable increase in user satisfaction. Nielsen's usability guidelines indicated ratings ranging from 3.93 (good) to 4.30 (very good), signifying users' positive perceptions of learnability, efficiency, memorability, error handling, and overall satisfaction within the MR batik-making application. This research also addressed these issues by meticulously optimizing 3D assets within MR applications. The results indicated that the application's CPU utilization is around 22%, attributed to its startup processes, resource allocation, and any initial computations it performed. However, the application continued to run within 4 GB of installed memory with the network bandwidth usage at 0.27 Mbps. User comfortability was enhanced, which is evident in improved ratings for importance and satisfaction levels (4.26 and 4.16, respectively) in the MR application.

Learning Reimagined: AI and Mixed Reality Revolutionize Indian Education

Artificial intelligence and mixed reality are realities now, not just ideas from science fiction films. AI analyzes data and can provide interpretation, just like the human brain. The efficiency and accuracy of this are causing the globe to adopt it presently. As opposed to mixed reality (MR), which combines the actual and virtual worlds to create new locations and scenarios, where things are both digital and tangible and interact with each other in real time. With a focus on Indian education, this paper provides a thorough overview of artificial intelligence (AI) and mixed reality technologies and their prospective educational applications. Together, artificial intelligence (AI) and mixed reality can revolutionize education by enhancing learning outcomes and bringing fresh resources to the classroom that have the power to change traditional methods of instruction. Like any new technology, there are hazards involved in using it, though. Experts are even in danger because of its immense power, which has the potential to fundamentally alter several industries and cause a large number of job losses. which has the potential to fundamentally alter several industries and cause a large number of job losses. Keywords -- Mixed Reality, Artificial Intelligence, Indian Education

A scoping review of the ethics frameworks describing issues related to the use of extended reality

The use of extended reality (XR) / immersive technologies such as virtual, augmented, mixed reality and virtual worlds (Metaverse) raises issues of ethical concern. The various issues, if left unaddressed, may impact human wellbeing over time. Immersive technologies are used in entertainment, commerce, training, education, health, and the military among others. Subsequently, there is a broad spectrum of users with various degrees of competencies and vulnerabilities. Special attention regarding long-term effects of immersive technologies on children and the lack of consideration of inclusivity for all persons in society is essential. Several publications have highlighted ethical issues related to immersive technologies, and some have sought to address these issues by proposing solutions or approaches in the form of frameworks, codes of conduct or best practices. This review examined literature between 2000 and 2023 to identify proposed or adopted ethical frameworks, codes of conduct or best practices for immersive technologies. Qualitative research method was applied, using a scoping review approach. Twenty-eight papers were selected for analysis. Approximately 70% of the selected papers were published between 2020 and 2022. Using an inductive thematic analysis method, seven fundamental values and twenty-two corresponding principles were generated. The main values are respect for persons, well-being, safety, integrity and trust, justice, and responsiveness. The dominant principles identified are privacy, informed consent, responsibility, transparency, and freedom. The authors of the papers were predominantly academic researchers. The normative approaches to addressing ethical issues were organised into four domains: society and governance, industry, research/academic organisations, and individuals. Recommendations are: 1) development and/or application of laws or guidelines to ethical, legal, and social issues with immersive technologies; 2) adoption of inclusive approaches to design and development; 3) minimisation of risk for research participants; 4) empowerment of users of immersive technologies; and 5) promotion of responsibility and sincerity in the use of virtual space, especially in matters concerning identity and conduct.

Understanding Multi-user, Handheld Mixed Reality for Group-based MR Games

Research has identified applications of handheld-based VR, which utilizes handheld displays or mobile devices, for developing systems that involve users in mixed reality (MR) without the need for head-worn displays (HWDs). Such systems can potentially accommodate large groups of users participating in MR. However, we lack an understanding of how group sizes and interaction methods affect the user experience. In this paper, we aim to advance our understanding of handheld-based MR in the context of multiplayer, co-located games. We conducted a study (N = 38) to understand how user experiences vary by group size (2, 4, and 8) and interaction method (proximity-based or pointing-based). For our experiment, we implemented a multiuser experience for up to ten users. We found that proximity-based interaction that encouraged dynamic movement positively affected social presence and physical/temporal workload. In bigger group settings, participants felt less challenged and less positive. Individuals had varying preferences for group size and interaction type. The findings of the study will advance our understanding of the design space for handheld-based MR in terms of group sizes and interaction schemes. To make our contributions explicit, we conclude our paper with design implications that can inform user experience design in handheld-based mixed reality contexts.

Application of mixed reality navigation technology in primary brainstem hemorrhage puncture and drainage surgery: a case series and literature review

Application of mixed reality navigation technology in primary brainstem hemorrhage puncture and drainage surgery: a case series and literature review

Research needs for enhancing pre-occupancy evaluation of buildings

PurposeMany buildings fail to meet user expectations, causing a performance gap. Pre-occupancy evaluation (PrOE) is believed to have the potential to close the gap. It enables designers to obtain end-user feedback in the design phase and improve the design for better performance. However, PrOE implementation faces challenges due to still maturing knowledgebase. This study aims to understand the state-of-the-art knowledge of PrOE, thereby identifying future research needs to advance the domain.Design/methodology/approachA systematic literature review following the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) framework was conducted. A thorough search in five databases and Google Scholar retrieved 90 articles, with 30 selected for systematic review after eliminating duplicates and irrelevant articles. Bibliometric analyses were performed using VOSviewer and Biblioshiny on the article metadata, and thematic analyses were conducted on their contents.FindingsPrOE is a vehicle for engaging building end-users in the design phase to address the credibility gap caused by the discrepancies between the expected and actual performance of buildings. PrOE has gained limited applications in healthcare, residential, office and educational building design for two broad purposes: design management and marketing. Using virtual reality technologies for PrOE has demonstrated significant benefits. Yet, the PrOE domain needs to mature in multiple perspectives to serve its intended purpose effectively.Originality/valueThis study identifies four knowledge gaps for future research to advance the PrOE domain: (1) developing a holistic PrOE framework, integrating comprehensive performance evaluation criteria, useable at different stages of the design phase and multi-criteria decision algorithms, (2) developing a mixed reality tool, embodying the holistic PrOE framework, (3) formulating a PrOE framework for adaptive reuse of buildings and (4) managing uncertainties in user requirements during the lifecycle in PrOE decisions.

Psychological Factors, Digital Health Technologies, and Best Asthma Management as Three Fundamental Components in Modern Care: A Narrative Review

New digital interventions have shown potential in managing asthma and improving patients’ quality of life compared with conventional interventions. Our objective was to conduct an exhaustive survey of the application of digital health technologies in evaluating, treating, and self-managing psychological and psychopathological elements linked to asthma. We analyzed a compendium of research papers pertaining to asthma, encompassing themes such as outdoor air pollution, early life wheezing disorders, atopic dermatitis, digital strategies for asthma self-management, psychiatric conditions and asthma, familial impacts on pediatric asthma, and the utilization of mobile health apps for managing asthma. We scrutinized six chosen studies to evaluate the capacity of digital health technologies to enhance the management and treatment outcomes of psychological factors related to asthma. The studies under review indicate that eHealth interventions, mixed reality instruments, mHealth technology-augmented nurse-led interventions, and smartphone apps incorporating Bluetooth-enabled sensors for asthma inhalers can markedly enhance self-management of symptoms, quality of life, and mental health outcomes, particularly in children and adolescents with asthma. Nonetheless, additional research is required to ascertain their efficacy and practicability across diverse populations and environments. Customized interventions that cater to the unique needs and preferences of asthma patients and associated psychological elements are vital to ensure the persistent and equitable adoption of these technologies. We underscore the significance of addressing psychological elements in the management and treatment of asthma and advocate for ongoing research and development in this domain.