Design, development, and evaluation of an immersive augmented virtuality training system for transcatheter aortic valve replacement

The adoption of immersive technologies is increasing in sensory, consumer, and marketing research, yet existing extended reality (XR) systems face limitations in realism, ease of product interaction, presence, data collection, and scalability. This study presents Sense-AV, an augmented virtuality (AV) system designed for large-scale sensory and consumer tests with enhanced immersion and realism. 102 participants evaluated two foods and one beverage across two sessions: a conventional sensory booth and the Sense-AV system, which simulated a sports bar environment. Real-time data collection was supported through API-linked mobile questionnaires, audio prompts via the head-mounted display (HMD), and open comments recorded by voice. Sense-AV was rated highly for usability, efficiency, satisfaction, presence, and sensory awareness. Older participants reported greater ease in handling products, while some difficulties with mobile input were noted but had minimal impact on the overall user experience (UX). Interviews emphasized immersion, intuitive use, and minor technical adjustments. No significant differences in overall product liking were found between methods, except for the mayonnaise, which scored higher in the immersive setting. Although food intake was lower in Sense-AV, oral feedback was more detailed and expressive. The system demonstrates innovation by improving realism and external validity in large-scale sensory evaluations.

As of 2025, extended reality (XR) refers to a spectrum of immersive technologies, including augmented reality (AR), augmented virtuality (AV), mixed reality (MR), and virtual reality (VR). Terminology in the XR field continues to evolve alongside technological capabilities and emerging use cases. As a result, the meanings of these terms are shifting, which can lead to ambiguity and confusion not only for the casual end user but for developers, researchers, instructors, and decision makers. In the spirit of exploration and open communication, this discussion panel with feature experts and enthusiasts to converse on XR terminologies and uses, how different understandings of hardware and software can impact collaboration, how we can better communicate, and which terms we love (and love to hate). The goal is to provide attendees with both conceptual clarity and actionable insights for advancing XR in real-world training and operational contexts.

ABSTRACT This paper presents research on the role of Augmented Virtuality (AV) within collaborative educational environments. While AV remains a rather underexplored dimension in the Virtuality-Reality Continuum, this study lays the groundwork for its pedagogical integration, investigating how real-world data can enhance collaborative learning and offer valuable insights into students’ collaborative interactions. The paper provides a background on current approaches to collaborative learning, particularly assessing collaboration to understand interactions, indicators and behaviours that indicate a possible degree of collaboration between students. The paper describes the iterative experimental studies conducted, aimed at assessing learners’ states during learning activities, which played a key role in refining the initial requirements for the proposed framework, design and development of the system. The key results from the validation experiments with students and consultation with teaching practitioners are presented. The results from validation experiments showed a positive correlation between students’ perceived collaboration levels and those calculated by the algorithm developed, indicating the potential for the framework proposed to promote student self-awareness and potentially more enhanced collaborative interactions. The scope of this study is to contribute as an intersection of AV technologies and education, providing a foundation for educational practitioners to create collaborative environments enriched by AV technologies.

Poor students’ academic performance at all educational level nowadays is a significant challenge to educational stakeholders globally. A country’s development is directly connected to the academic performance of its student. There are multiple factors that can influence students’ academic performance, however, teachers’ instructional strategy play a significant role. Immersive education is a pedagogical approach that uses virtual reality, augmented reality, mixed reality, augmented virtuality, 3D and 360 degree immersive technologies for knowledge and skill acquisition. First, the paper introduce immersive education and its categories to distinguish between these technologies. This study was guided by preferred reporting Items for systematic reviews and Meta-Analyses (PRISMA) protocol. The review protocol was designed to guide the conduct of the systematic literature review. The review method comprised of research questions, inclusion and exclusion criteria, literature search strategy, study selection process, data extraction, data synthesis and report writing. This study evaluated empirical literature published in research community on immersive education from 2020 to 2024 by systematic literature review. The findings shows that despite the novelty and importance of immersive education, studies on this innovation is on decline in recent years. Therefore, it is recommended that further research be conducted on this technologies especially on augmented virtuality, 3D and 360 degree immersive technologies to leverage on its benefits in teaching and learning in education systems. .

Robotic ultrasound systems have the potential to improve medical diagnostics, but patient acceptance remains a key challenge. To address this, we propose a novel system that combines an AI-based virtual agent, powered by a large language model (LLM), with three mixed reality visualizations aimed at enhancing patient comfort and trust. The LLM enables the virtual assistant to engage in natural, conversational dialogue with patients, answering questions in any format and offering real-time reassurance, creating a more intelligent and reliable interaction. The virtual assistant is animated as controlling the ultrasound probe, giving the impression that the robot is guided by the assistant. The first visualization employs augmented reality (AR), allowing patients to see the real world and the robot with the virtual avatar superimposed. The second visualization is an augmented virtuality (AV) environment, where the real-world body part being scanned is visible, while a 3D Gaussian Splatting reconstruction of the room, excluding the robot, forms the virtual environment. The third is a fully immersive virtual reality (VR) experience, featuring the same 3D reconstruction but entirely virtual, where the patient sees a virtual representation of their body being scanned in a robot-free environment. In this case, the virtual ultrasound probe, mirrors the movement of the probe controlled by the robot, creating a synchronized experience as it touches and moves over the patient's virtual body. We conducted a comprehensive agent-guided robotic ultrasound study with all participants, comparing these visualizations against a standard robotic ultrasound procedure. Results showed significant improvements in patient trust, acceptance, and comfort. Based on these findings, we offer insights into designing future mixed reality visualizations and virtual agents to further enhance patient comfort and acceptance in autonomous medical procedures.

An immersive spatially consistent multi-modal augmented virtuality human-machine interface for telerobotic systems

Drawing on social influence and behavioral intention theories, coworkers' risk-taking serves as an "extra motive" -an exogenous factor-for risk-taking behaviors among workers in the workplace. Social influence theories have shown that social factors, such as social pressure and coworker risk-taking, may predict risk-taking behaviors and significantly affect decision-making. While immersive technologies have been widely used to create close-to-real simulations for construction safety-related studies, there is a paucity of research considering the impact of social presence in evaluating workers' risk decision-making within immersive environments. To bridge this gap, this study developed a state-of-the-art Augmented Virtuality (AV) environment to investigate roofers' risk-taking behaviors when exposed to social stressors (working alongside a safe/unsafe peer). In this augmented virtuality environment, a virtual peer with safe and unsafe behaviors was simulated in order to impose peer pressure and increase participants' sense of social presence. Participants were asked to install asphalt shingles on a physical section of a roof (passive haptics) while the rest of the environment was projected virtually. During shingle installation, participants' cognitive and behavioral responses were captured using psychophysiological wearable technologies and self-report measures. The results demonstrated that the developed AV model could successfully enhance participants' sense of presence and social presence while serving as an appropriate platform for assessing individuals' decision-making orientations and behavioral changes in the presence of social stressors. Such information shows the value of immersive technologies to examine the naturalistic responses of individuals without exposing them to actual risks.

Digital simulation in dental education has substantially evolved, addressing several educational challenges in dentistry. Following global lockdowns and sustainability concerns, dental educators are increasingly adopting digital simulation to enhance or replace traditional training methods. This review aimed to contribute to a uniform taxonomy for extended reality (XR) simulation within dental education. This scoping review followed the PRISMA and PRISMA-ScR guidelines. PubMed/MEDLINE, EMBASE, Web of Science and Google Scholar were searched. Eligible studies included English-written publications in indexed journals related to digital simulation in dental/maxillofacial education, providing theoretical descriptions of extended reality (XR) and/or immersive training tools (ITT). The outcomes of the scoping review were used as building blocks for a uniform of XR-simulation taxonomy. A total of 141 articles from 2004 to 2024 were selected and categorised into Virtual Reality (VR), Mixed Reality (MR), Augmented Reality (AR), Augmented Virtuality (AV) and Computer Simulation (CS). Stereoscopic vision, immersion, interaction, modification and haptic feedback were identified as recurring features across XR-simulation in dentistry. These features formed the basis for a general XR-simulation taxonomy. While XR-simulation features were consistent in the literature, the variety of definitions and classifications complicated the development of a taxonomy framework. VR was frequently used as an umbrella term. To address this, operational definitions were proposed for each category within the virtuality continuum, clarifying distinctions and commonalities. This scoping review highlights the need for a uniform taxonomy in XR simulation within dental education. Establishing a consensus on XR-related terminology and definitions facilitates future research, allowing clear evidence reporting and analysis. The proposed taxonomy may also be of use for medical education, promoting alignment and the creation of a comprehensive body of evidence in XR technologies.

Mixed Reality allows us to integrate virtual and physical content into users' environments seamlessly. Yet, how this fusion affects perceptual and cognitive resources and our ability to find virtual or physical objects remains uncertain. Displaying virtual and physical information simultaneously might lead to divided attention and increased visual complexity, impacting users' visual processing, performance, and workload. In a visual search task, we asked participants to locate virtual and physical objects in Augmented Reality and Augmented Virtuality to understand the effects on performance. We evaluated search efficiency and attention allocation for virtual and physical objects using event-related potentials, fixation and saccade metrics, and behavioral measures. We found that users were more efficient in identifying objects in Augmented Virtuality, while virtual objects gained saliency in Augmented Virtuality. This suggests that visual fidelity might increase the perceptual load of the scene. Reduced amplitude in distractor positivity ERP, and fixation patterns supported improved distractor suppression and search efficiency in Augmented Virtuality. We discuss design implications for mixed reality adaptive systems based on physiological inputs for interaction.

Mixed reality enables users to immerse themselves in high-workload interaction spaces like office work scenarios. We envision physiologically adaptive systems that can move users into different mixed reality manifestations, to improve their focus on the primary task. However, it is unclear which manifestation is most conducive for high productivity and engagement. In this work, we evaluate whether physiological indicators for engagement can be discriminated for different manifestations. For this, we engaged participants in a typing task in three different mixed reality manifestations (augmented reality, augmented virtuality, virtual reality) and monitored physiological correlates (EEG, ECG, and eye tracking) of users' engagement and workload. We found that users achieved best typing performances in augmented reality and augmented virtuality. At the same time, physiological engagement peaked in augmented virtuality, while workload decreased. We conclude that augmented virtuality strikes a good balance between the different manifestations, as it facilitates displaying the physical keyboard for improved typing performance and, at the same time, allows one to block out the real world, removing many real-world distractors.

Supplemental Material for Immersive Virtual Reality and Augmented Virtuality in Sport and Performance Psychology: Opportunities, Current Limitations, and Practical Recommendations

While Mixed Reality allows the seamless blending of digital content in users' surroundings, it is unclear if its fusion with physical information impacts users' perceptual and cognitive resources differently. While the fusion of digital and physical objects provides numerous opportunities to present additional information, it also introduces undesirable side effects, such as split attention and increased visual complexity. We conducted a visual search study in three manifestations of mixed reality (Augmented Reality, Augmented Virtuality, Virtual Reality) to understand the effects of the environment on visual search behavior. We conducted a multimodal evaluation measuring Fixation-Related Potentials (FRPs), alongside eye tracking to assess search efficiency, attention allocation, and behavioral measures. Our findings indicate distinct patterns in FRPs and eye-tracking data that reflect varying cognitive demands across environments. Specifically, AR environments were associated with increased workload, as indicated by decreased FRP - P3 amplitudes and more scattered eye movement patterns, impairing users' ability to identify target information efficiently. Participants reported AR as the most demanding and distracting environment. These insights inform design implications for MR adaptive systems, emphasizing the need for interfaces that dynamically respond to user cognitive load based on physiological inputs.

Dementia impacts millions worldwide and is challenging individuals' ability to engage in daily activities. Active living is crucial in mitigating dementia's neurodegenerative effects, yet people with dementia often struggle to initiate and complete tasks independently. Technologies offer promising solutions to engage people with dementia in activities of active living and improving their quality of life through prompting and cueing. It is anticipated that developments in sensor and wearable technologies will result in mixed reality technology becoming more accessible in everyday homes, making them more deployable. The possibility of mixed reality technologies to be programmed for different applications, and to adapt them to different levels of impairments, behaviours and contexts, will make them more scalable. The study aimed to develop a better understanding of modalities of prompts that people with dementia perceive successfully and correctly in mixed reality environments. It investigated interactions of people with dementia with different types of visual (graphics, animation, etc.) and sound (human voice, tones, etc.) prompts in mixed reality technologies. We used the Research through Design (RtD) method in this study. This paper describes the findings from the user research carried out in the study. We conducted observation studies with twenty-two people with dementia playing games on off-the-shelf mixed reality technologies, including both Augmented Reality (HoloLens, ArKit on iPhone) and Augmented Virtuality (Xbox Kinect and Osmo) technologies. The interactions with the technologies during the gameplay were video recorded for thematic analysis in Noldus Observer XT (version 16.0) for successful and correct perception of prompts. A comparison of the probability estimates of correct perception of the prompts by people with dementia suggests that human voice, graphic symbols and text are the most prominently perceived modalities of prompts. Feedback prompts for every action performed by people with dementia on the technology are critical for successful perception and should always be provided in the design. The study has resulted in recommendations and guidelines for designers to design prompts for people with dementia in mixed-reality environments. The work lays the foundation for considering mixed reality technologies as assistive tools for people with dementia, fostering discussions on their accessibility and inclusive design in technology development.

Sensory analysis methodologies are performed in sensory booths designed to minimise external stimuli, lacking ecological validity. Immersive environments are used to introduce contextual cues, but there is a lack of studies using mixed reality systems. The main goal of this study was to evaluate an augmented virtuality (AV) system where participants are inserted into a virtual environment and evaluate a real product, being able to interact with both dimensions. A panel of 102 consumers evaluated five samples of commercial peach nectars in three sessions, each in a different environment: public food court, living room (AV environments), and laboratory (traditional sensory booth). Consumers rated overall liking, followed by open comments, and also answered an Engagement (EQ) and a Presence Questionnaire (PQ). The type of environment only affected hedonic discrimination among samples, with the laboratory setting being the only one with sample discrimination. Nonetheless, each sample was not evaluated differently across the different environments. Concerning engagement, the environment only significantly influenced the EQ's 'Affective Value' factor, being higher when using an AV system. The level of presence in the virtual environment was significantly higher in the public food court, being significantly correlated with the EQ factor scores.

Abstract Technologies like virtual reality (VR) and augmented virtuality (AV) based on head-mounted display (HMD) have made immersive interactive narrative experience possible. To investigate the differences between HMD VR and HMD AV for interactive storytelling, an app entitled the Extended Journey has been designed, developed and deployed on both VR headsets and optical see-through (OST) mixed reality (MR) headsets. Subsequently, a within-subjects design experiment with sixty participants was conducted to measure different factors of user experience in HMD VR and HMD AV conditions. Additionally, a between-subjects design experiment with 62 participants was conducted to evaluate narrative immersion and presence. Quantitative results from the experiments reveal that HMD VR narrative had statistically significantly better narrative immersion, presence, enjoyment and perceived affinity of computer-generated (CG) characters compared with HMD AV narrative in the context of galleries and museums. Based on Sundar's Modality-Agency-Interactivity-Navigability (MAIN) model, we proposed and validated that sense of being-there and interaction mediated the relationship between immersive media modality and narrative immersion as well as empathy. Qualitative data and behavioral tests showed that young people have a higher preference for HMD VR over HMD AV, but for specific use scenarios, there is still a certain possibility of choosing HMD AV due to security and space limitations. These findings have theoretical implications for the psychology of VR and AV, and practical applications for HMD-based immersive narrative in general.

This study explores the anticipated changes in team collaboration driven by the rise of AI-reliant businesses in the metaverse. The research is motivated by the rapid technological advancements and increasing interest from major tech companies in creating shared virtual environments for distributed teams. The main objective is to investigate how team collaboration will evolve in this new context, providing insights for future metaverse team members and leaders. The qualitative study employs semi-structured, in-depth interviews in January-February 2022 with team members from the virtual world of Second Life, focusing on payment satisfaction, globalized team culture, training and learning, and fictional identities. This research period is contemporary, reflecting current trends and capabilities. The primary methods include thematic analysis of interview data to identify emerging patterns. Data is analyzed using “Dedoose”, a web application for mixed methods research, and collected in several codes and subcodes that will eventually form a “code tree”. Descriptive validity, interpretative validity, theoretical validity and generalizable validity were confirmed. Key findings reveal that metaverse-native teams need innovative payment methods and a focus on authentic self-expression. A globalized team culture will require balancing work and social life in virtual environments, with Corporate Social Responsibility being vital to attracting Generation Alpha. Payment satisfaction will shift towards fulfilment and unique experiences rather than just financial security, and businesses should explore alternative payment methods beyond cryptocurrency. Fictional identities will necessitate building metaverse-specific reputations for professionalism and trust. Training and learning will connect virtual and real-life scenarios, enhancing skills and quality. The study confirms that traditional collaboration models may only partially apply in augmented virtuality. These results have practical significance for businesses and professionals preparing to operate in the metaverse, offering a foundation for future research and development in virtual team collaboration.

When developing XR applications for Industry 4.0, it is important to consider the integration of visual displays, hardware components, and multimodal interaction techniques that are compatible with the entire system. The potential use of multimodal interactions in industrial applications has been recognized as a significant factor in enhancing humans’ ability to perform tasks and make informed decisions. To offer a comprehensive analysis of the current advancements in industrial XR, this review presents a structured tutorial that provides answers to the following research questions: (R.Q.1) What are the similarities and differences between XR technologies, including augmented reality (AR), mixed reality (MR), Augmented Virtuality (AV), and virtual reality (VR) under Industry 4.0 consideration? (R.Q.2) What types of visual displays and hardware devices are needed to present XR for Industry 4.0? (R.Q.3) How did the multimodal interaction in XR perceive and relate to Industry 4.0? (R.Q.4) How have modern adaptations of XR technologies dealt with the theme of Industry 4.0? (R.Q.5) How can XR technologies in Industry 4.0 develop their services and usages to be more solution-inclusive? This review showcases various instances that demonstrate XR’s potential to transform how humans interact with the physical world in Industry 4.0. These advancements can increase productivity, reduce costs, and enhance safety.

Effect of Designer- versus User-driven Network-monitoring Dashboard Design on User Flow Experience and Performance: The Role of Augmented Virtuality

As modern information technology advances and equipment devices update, extended reality (XR) technologies, including virtual reality (VR), augmented reality (AR), and augmented virtuality (AV) have witnessed an increasing use and application in construction training. This review aims to comprehensively examine the evolution of XR training in the construction domain. To achieve this, a systematic literature review of 74 journal papers from the Scopus database was conducted. This paper outlines the progression of XR training from 2009 to 2023, detailing related technologies like development platforms, display devices, and input devices. The literature review reveals that XR application in construction training spans five main areas: (1) safety management, (2) skill/knowledge acquisition, (3) equipment operation, (4) human–computer collaboration, and (5) ergonomics/postural training. Additionally, this review explores the impact of trainee roles on XR training outcomes and identifies the challenges faced by XR technology in construction training applications. The findings of this literature review are hoped to assist researchers and construction engineering trainers in understanding the latest advancements and challenges in XR, thereby providing valuable insights for future research.