Augmented Reality Display Research Articles

Exploring the Role of Location-Based Games in Managing Tourist Destinations Under Climate Change Challenges: A Gap Analysis Review

Global climate disruptions pose escalating threats to tourism networks, necessitating innovative resilience solutions tailored for regional interdependencies. This review examines research on location-based games for enhancing climate resilience across interconnected tourism economies. Analyzing 75 studies, strengths and limitations are delineated. While confirming augmented reality, virtual reality, and geo-tagging versatility for promotion, analysis, and experience enhancement, findings reveal gaps in leveraging these technologies for systemic coordination, participatory governance, embodied vulnerability assessment, and social learning. Immersive climate visualizations, policy simulations, and multiplayer interfaces emerge as frontiers enabling collaborative adaptation. The top priorities are (1) integrating localized climate projections with human perceptions through interactive visualizations to create tangible threats, (2) designing policy simulations for participatory governance of resilience investments across sectors, (3) developing embodied social learning vulnerability assessments highlighting differential exposures, and (4) designing multiplayer games to facilitate the co-creation of equitable, robust adaptation strategies by communities. Targeted research advancing location-based platforms to link science, policy, and community priorities is essential for tourism networks to navigate intensifying climate disruptions collaboratively. This review thus delineates critical next steps in utilizing geo-technologies’ participatory, experiential promise to inform and connect stakeholders in steering tourism toward resilient pathways.

HuBar: A Visual Analytics Tool to Explore Human Behavior based on fNIRS in AR Guidance Systems.

The concept of an intelligent augmented reality (AR) assistant has significant, wide-ranging applications, with potential uses in medicine, military, and mechanics domains. Such an assistant must be able to perceive the environment and actions, reason about the environment state in relation to a given task, and seamlessly interact with the task performer. These interactions typically involve an AR headset equipped with sensors which capture video, audio, and haptic feedback. Previous works have sought to facilitate the development of intelligent AR assistants by visualizing these sensor data streams in conjunction with the assistant's perception and reasoning model outputs. However, existing visual analytics systems do not focus on user modeling or include biometric data, and are only capable of visualizing a single task session for a single performer at a time. Moreover, they typically assume a task involves linear progression from one step to the next. We propose a visual analytics system that allows users to compare performance during multiple task sessions, focusing on non-linear tasks where different step sequences can lead to success. In particular, we design visualizations for understanding user behavior through functional near-infrared spectroscopy (fNIRS) data as a proxy for perception, attention, and memory as well as corresponding motion data (acceleration, angular velocity, and gaze). We distill these insights into embedding representations that allow users to easily select groups of sessions with similar behaviors. We provide two case studies that demonstrate how to use these visualizations to gain insights about task performance using data collected during helicopter copilot training tasks. Finally, we evaluate our approach by conducting an in-depth examination of a think-aloud experiment with five domain experts.

Development of Chemis 3D Augmented Reality Media for Discovery Learning Model in Chemical Bonding

The goal of this research and development (R&D) study was to develop a 3D Chemis media product that included cards and applications for three-dimensional display. The 4-D development model by Thiagarajan et al., which comprised the phases of defining, designing, developing, and distributing, was consulted in the creation of this 3D chemistry media. The study's findings included the following: 1) two media experts and two material experts' evaluations of the media's viability and validity fell into the extremely valid group; 2) During the limited testing phase, which involved 29 students from class X MIA 5 SMAN 16 Makassar, each student received an average learning outcome test score of 83.3 out of 100, with 86.2% of the class completing the course. Additionally, the evaluation of student motivation for this media fell into the very high category, suggesting that the media was effective; 3) both students' and teachers' responses to the developed learning media fell into the very high category, suggesting that the media was useful. Following a validation stage by experts and validators, development revisions, and a small number of trials, this learning material was found to be reliable, useful, and efficient. The Chemis 3D media model was shown to be reliable, useful, and efficient.

Analysis of wave aberrations and optical modeling for off-axis augmented reality display

Analysis of wave aberrations and optical modeling for off-axis augmented reality display

A novel 3D image registration technique for augmented reality vision in minimally invasive thoracoscopic pulmonary segmentectomy.

In this feasibility study, we aimed to create a dedicated pulmonary augmented reality (AR) workflow to enable a semi-automated intraoperative overlay of the pulmonary anatomy during video-assisted thoracoscopic surgery (VATS) or robot-assisted thoracoscopic surgery (RATS). Initially, the stereoscopic cameras were calibrated to obtain the intrinsic camera parameters. Intraoperatively, stereoscopic images were recorded and a 3D point cloud was generated from these images. By manually selecting the bifurcation key points, the 3D segmentation (from the diagnostic CT scan) was registered onto the intraoperative 3D point cloud. Image reprojection errors were 0.34 and 0.22 pixels for the VATS and RATS cameras, respectively. We created disparity maps and point clouds for all eight patients. Time for creation of the 3D AR overlay was 5min. Validation of the point clouds was performed, resulting in a median absolute error of 0.20mm [IQR 0.10-0.54]. We were able to visualize the AR overlay and identify the arterial bifurcations adequately for five patients. In addition to creating AR overlays of the visible or invisible structures intraoperatively, we successfully visualized branch labels and altered the transparency of the overlays. An algorithm was developed transforming the operative field into a 3D point cloud surface. This allowed for an accurate registration and visualization of preoperative 3D models. Using this system, surgeons can navigate through the patient's anatomy intraoperatively, especially during crucial moments, by visualizing otherwise invisible structures. This proposed registration method lays the groundwork for automated intraoperative AR navigation during minimally invasive pulmonary resections.

Utilization of Augmented and Mixed Reality in Training Mining Machines

Augmented reality (AR) and mixed reality (MR) are used in many disciplines, especially education, science, health, safety and engineering. It is based on the visualization and interaction of virtual graphic designs in the natural environment with its own hardware and software. In this study, the effect of augmented and mixed reality applications in the training of open-pit mining machines is examined. In the application called AR Book, a booklet containing machine visuals and technical information was created. In the project developed in the Unity real-time development engine, the database created in the Vuforia AR engine and 3D machine models matched with them were used. An image-target-based augmented reality application was implemented in this booklet using Android devices. In another application, a ground-plane-based application was developed for the mixed reality device MS Hololens 2. With the Hand Interactions feature, machines positioned in the natural environment can be controlled with drag, rotate and scale operations, enlarged to gigantic sizes and examined. The evaluations of the training group regarding the use of AR and MR applications together with traditional education were sought. They were also asked to make comparisons between the mobile device and Hololens 2. There have been highly positive results in integrating imaging technologies into education.

Design and Evaluation of Ecological Interface of Driving Warning System Based on AR-HUD.

As the global traffic environment becomes increasingly complex, driving safety issues have become more prominent, making manual-response driving warning systems (DWSs) essential. Augmented reality head-up display (AR-HUD) technology can project information directly, enhancing driver attention; however, improper design may increase cognitive load and affect safety. Thus, the design of AR-HUD driving warning interfaces must focus on improving attention and reducing cognitive load. Currently, systematic research on AR-HUD DWS interfaces is relatively scarce. This paper proposes an ecological interface cognitive balance design strategy for AR-HUD DWS based on cognitive load theory and environmental interface design theory. The research includes developing design models, an integrative framework, and experimental validation suitable for warning scenarios. Research results indicate that the proposed design effectively reduces cognitive load and significantly decreases driver response and comprehension times, outperforming existing interfaces. This design strategy and framework possess promotional value, providing theoretical references and methodological guidance for AR-HUD warning interface research.

AR Smart Glasses: The Feeling of Groundedness Mediator Effect

Augmented Reality (AR) smart glasses strengthen user interaction by merging real-world visuals with digital information. This study explores the impact of wearability, usability, and social influence on the feeling of groundedness and purchase intentions for AR smart glasses moderated by privacy concerns. Text mining and PLS-SEM were employed to analyze 3,164 Amazon smart glasses online reviews. Results suggest that the feeling of groundedness is influenced by usability and social influence, while wearability showed no significant effect. Privacy concerns moderate the relationship between groundedness and purchase intentions, emphasizing the need for privacy measures to improve user trust. These findings suggest that AR smart glasses create meaningful connections for users, affecting their preferences, choices, and willingness to pay.

Priority Design in Multi-Target AR-HUD Warning: Evidence from Eye Movement and Behavior of the Novice Driver

Technically, Augmented Reality Head-Up Display (AR-HUD) technology can augment multiple targets in the scene. However, existing literature predominantly focuses on scenarios involving single-target augmentation. The understanding of multi-target augmentation and studies exploring effective presentation methods under such conditions are notably scarce. This study evaluates the efficacy of integrating color-based warning priority design across multi-target scenarios. 45 Participants in different warning modes (Equivalent, Hierarchical, and Baseline) view AR-augmented driving videos and respond to risky targets. Their behavioral performance and eye-tracking data are compared. Findings indicate that the equivalent warning mode, lacking in priority design, adversely affects driver performance, prolongs reaction times, and elevates saccade counts, and gaze entropy. Conversely, the hierarchical warning mode significantly ameliorates driver reaction times and the time to first fixation, while also reducing saccade counts and gaze entropy, demonstrating the efficacy of the warning priority design. The findings provide insight into the design of AR-HUD with multi-target augmentation.

Investigating Effects of Future Path Visualisation on Path Choices During Collision Encounters

Safe navigation choices made in walkable spaces highly depend on how a traveller perceives and understands the threat from surrounding travellers’ movements. Hence, if a visual medium like an AR headset provides an augmented view where future paths of others are already shown virtually, this could influence the way a person decides to avoid a potential collision. In this work, taking walking conflicts as an example, we first propose different ways of showing the future path in AR and then investigate whether people decide to walk more safety-consciously when seeing AR information. For this, we conducted a web-based user study (n =27), in which participants sketched a walking path that they preferred when avoiding collisions in different crossing scenes. Each scene in the study contained another person walking and crossing the participant’s expected path, while the prediction of the another person’s future walking path was either not visualised or augmented with a virtual arrow. Participants were then expected to sketch a path to their destination while not colliding with the person in the scene. By applying a sketch transformation pipeline, the participants’ drawings were transformed to trajectories of the preferred walking paths, which where then simulated to estimate the conflict severity using the Post Encroachment Time (PET) metric. The study verified that people choose to adapt their walking paths when seeing the future path information, and also react differently to different types of 3D arrow visualisations that represent the same future path.

A randomized controlled trial on evaluating clinician-supervised generative AI for decision support

BackgroundThe integration of generative artificial intelligence (AI) as clinical decision support systems (CDSS) into telemedicine presents a significant opportunity to enhance clinical outcomes, yet its application remains underexplored. ObjectiveThis study investigates the efficacy of one of the most common generative AI tools, ChatGPT, for providing clinical guidance during cardiac arrest scenarios. MethodsWe examined the performance, cognitive load, and trust associated with traditional methods (paper guide), autonomous ChatGPT, and clinician-supervised ChatGPT, where a clinician supervised the AI recommendations. Fifty-four subjects without medical backgrounds participated in randomized controlled trials, each assigned to one of three intervention groups: paper guide, ChatGPT, or supervised ChatGPT. Participants completed a standardized CPR scenario using an Augmented Reality (AR) headset, and performance, physiological, and self-reported metrics were recorded. Main FindingsResults indicate that the Supervised-ChatGPT group showed significantly higher decision accuracy compared to the paper guide and ChatGPT groups, although the scenario completion time was longer. Physiological data showed a reduced LF/HF ratio in the Supervised-ChatGPT group, suggesting potentially lower cognitive load. Trust in AI was also highest in the supervised condition. In one instance, ChatGPT suggested a risky option, highlighting the need for clinician supervision. ConclusionOur findings highlight the potential of supervised generative AI to enhance decision-making accuracy and user trust in emergency healthcare settings, despite trade-offs with response time. The study underscores the importance of clinician oversight and the need for further refinement of AI systems to improve safety. Future research should explore strategies to optimize AI supervision and assess the implementation of these systems in real-world clinical settings.

Polarization conversion effect in cholesteric liquid crystal-based polarization volume gratings

Following the recent experimental discovery of a new polarization conversion phenomenon in polarization volume gratings (PVGs), in this paper, we investigate its underlying physical mechanisms in cholesteric liquid crystal (CLC) reflectors and PVGs. We examine the transition of eigenstates from circular to linear polarization as the incident angle deviates from the helical axis, which originates such an anomalous polarization conversion effect. While this transition enables PVGs to double the in-coupling efficiency and brightness uniformity in waveguide-based augmented reality (AR) displays, it also degrades the polarization selectivity and alters the transmitted polarization state in both CLC reflectors and PVGs, which in turn narrows down their spectral and angular bandwidth in the multi-layer design. These findings help not only deepen the understanding of polarization behaviors in CLC-based optical elements but also provide valuable insights to optimize their performances for emerging AR smart glasses.

A systemic survey of the Omniverse platform and its applications in data generation, simulation and metaverse

Nvidia’s Omniverse platform represents a paradigm shift in the realm of virtual environments and simulation technologies. This paper presents a comprehensive examination of the Omniverse platform, a transformative force in virtual environments and simulation technologies. We offer a detailed systematic survey of the Omniverse’s impact across various scientific fields, underscoring its role in fostering innovation and sculpting the technological future. Our focus includes the Omniverse Replicator for generating synthetic data to address data insufficiency, and the utilization of Isaac Sim with its Issac Gym and software development kit (SDK) for robotic simulations, alongside Drive Sim for autonomous vehicle emulation. We further investigate the Extended Reality (XR) suite for augmented and virtual realities, as well as the Audio2Face application, which translates audio inputs into animated facial expressions. A critical analysis of Omniverse’s technical architecture, user-accessible applications, and extensions are provided. We contrast existing surveys on the Omniverse with those on the metaverse, delineating their focus, applications, features, and constraints. The paper identifies potential domains where the Omniverse excels and explores its real-world application capabilities by discussing how existing research papers utilize the Omniverse platform. Finally, we discuss the challenges and hurdles facing the Omniverse’s broader adoption and implementation, mitigating the lack of surveys solely focusing on the Omniverse.

Intraoperative Surgical Guidance for DIEP Flap Harvest using Augmented Reality

SUMMARY: Augmented reality (AR) systems for surgical navigation provides the capability to project preoperative CT scans and segmented anatomical structures directly into the surgeon’s field of view, along with virtual displays akin to traditional monitors. Utilizing the Meta Quest 3 consumer AR headset, we found that it can achieve clinically acceptable accuracy in surgical navigation for deep inferior epigastric perforator (DIEP) surgeries. Notably, the Quest 3 can operate independently thanks to a novel registration technique employing hand tracking, suitable for use in sterile environments. The DIEP flap is a widely favored option for autologous breast reconstructions. Intraoperatively, the perforator arteries are selected to maximize tissue perfusion and minimize donor site morbidity. Conventionally, preoperative CT angiograms are used to locate perforators and visualize vessel courses, thus reducing DIEP flap harvest time and complication rates. Though, there are currently very limited means to reliably translate information found on preoperative imaging for use in the operating theater. The Meta Quest 3 was trialed in three DIEP flap surgeries by displaying segmented deep inferior epigastric arteries and DIEP arteries during preoperative and intraoperative phases of surgery. The AR projections facilitated the visualization of the DIEP arteries and efficient pedicle dissection by guiding the placement and length of the fascial incision. Furthermore, this work supports the hypothesis that AR will improve flap design, reduce harvest time, and improve safety during DIEP flap harvesting.

PlantView: Integrating deep learning with 3D modeling for indoor plant augmentation

Indoor plant recognition poses significant challenges due to the variability in lighting conditions, plant species, and growth stages. Despite the growing interest in applying deep learning techniques to plant data, there still needs to be more research focused on the automatic recognition of indoor plant species, highlighting the need for real-time, automated solutions. To address this gap, this study introduces a novel approach for real-time identification and visualization of indoor plants using a Convolutional Neural Network (CNN)-based model called PlantView, integrated with Augmented Reality (AR) for enhanced user interaction. The proposed PlantView model not only accurately classifies the plant species but also visualizes them in a 3D AR environment, allowing users to interact with virtual plant models seamlessly integrated into their real-world surroundings. We developed a custom dataset comprising over 28,000 images of 48 different plant species at various growth stages, captured under diverse lighting conditions and camera settings. Our proposed approach achieves an impressive accuracy of 98.20 %. To validate the effectiveness of PlantView model, we conduct extensive experiments and compared its performance against state-of-the-art methods, demonstrating its superior accuracy and processing speed. The results indicate that our method is not only highly effective for real-time indoor plant recognition but also offers practical applications for enhancing indoor plant care and visualization. This research offers a comprehensive solution for indoor plant enthusiasts and professionals, combining advanced computer vision techniques with immersive AR visualization to revolutionize the way indoor plants are identified, visualized, and integrated into living spaces.

Understanding User Intention to Adopt Augmented Reality Head-Up Display: A Study Based on Design Factors

Augmented Reality Head-Up Display (AR-HUD) is designed to engender immersive experiences and possess vast potential in enhancing vehicular safety. In the context of advancing digital technologies, the intention to use such innovations represents a significant field of study. Drawing on the Perceived Value Theory and the Stimulus-Organism-Response (S-O-R) framework, this study formulates a model to evaluate user intentions regarding the use of automotive Augmented Reality Head-Up Display (AR-HUD). This model integrates design elements, including user interface attractiveness, sense of presence, controllability, and responsiveness, along with perceived utilitarian and hedonic values. Through Structural Equation Modeling (SEM) of 230 valid questionnaire results, the analysis identifies both utilitarian and hedonic values as pivotal in influencing AR-HUD adoption, with utilitarian value being preeminent. Additionally, it is evidenced that design attributes influence the usage intention via perceived utilitarian and hedonic values. Consequently, this study presents design strategies for auto manufacturers and designers aimed at amplifying the advantages of AR-HUD, thus fostering greater user adoption willingness.

Virtual anatomical atlas of the deep brain nuclei.

This study aims to improve understanding of the anatomy of the deep brain nuclei relevant to deep brain stimulation as well as stereotactic lesioning procedures, including radio frequency, high-focused ultrasound, and radiosurgery. We created interactive, three-dimensional virtual models from cadaveric dissections and radiological segmentation. We used five brain specimens (ten hemispheres) obtained from routine autopsies, prepared according to Klingler's method. Dissections were done from lateral to medial, medial to lateral, and superior to inferior to expose deep brain stimulation targets and adjacent structures. Using photogrammetry, we scanned the specimens to create detailed three-dimensional models. These models were uploaded to an online platform for free global access. Radiological models were also generated from atlas-based regions using the Montreal Neurological Institute template. We produced 16 high-quality cadaveric models at various stages of dissection. These and the radiological models were examined and interacted with through augmented reality and virtual reality headsets. This approach allowed comprehensive visual access to the anatomical structures and delineated their spatial relationships. These three-dimensional models provide detailed anatomical representations that can enhance anatomical orientiation, improve spatial perception, and serve as valuable educational tools for clinicians and students.

Enhancing manual inspection in semiconductor manufacturing with integrated augmented reality solutions

On-site routine inspection often remains a manual operation in the semiconductor manufacturing industry because implementing automated solutions can be costly and technically challenging in such a highly controlled and complex environment. The manual inspection is prone to errors due to the impact of demanding physical and mental workloads. This paper presents an integrated Augmented Reality (AR) solution developed to assist manual inspection tasks in the supporting areas of semiconductor manufacturing, referred to as the sub-fab. The solution is accessible to a human worker wearing an AR headset during the inspection process at the location. We propose a system framework to deploy computational intelligences of varying granularity provided by the solution across cloud, edge, and device levels, accommodating constraints within the sub-fab. A machine maintenance module helps estimate and monitor the health condition of running scrubbers. Incorrect intentions performed by the worker on the scrubber control panel are detected through hand gesture recognition. This instantly prompts warning messages in the AR headset to prevent subsequent wrong actions. The solution can also identify abnormal device states through 6D pose estimation of objects enabled by machine learning models. A test scenario demonstrates how these functional features enhance the inspection efficiency and quality by reducing human workloads. This work demonstrates that semiconductor manufacturing may require AR-assisted functions different from those needed or common in other industrial sectors. It also highlights the potential of AR technology for reducing operational human errors in manual tasks.

Intraoperative Position System guided with Augmented Reality Improves the Learning Curve of Endovascular Navigation in Endovascular Naïve Operators

ObjectiveThis study aimed to compare the completion of gate cannulation task performed by participants of varying experience using fluoroscopy, the Intra-Operative Positioning System (IOPS) – an FDA-cleared endovascular navigation system that has been developed to reduce dependence on fluoroscopy – or an investigational augmented reality electromagnetic navigation technology based on IOPS. MethodsThe task consisted in the cannulation of the gate of a GORE Excluder AAA endoprosthesis bifurcated aortic stent graft (W.L. GORE & Associates, Flagstaff, AZ USA) deployed into a 3D printed abdominal aortic aneurysm (AAA) model connected to a 7000 MDX flow pump (Sarns Inc/3M, Ann Arbor, MI USA) reproducing physiological conditions. The procedure was performed in a hybrid operating room (GE Allia IGS 7). Each participant performed the cannulation task with fluoroscopy, standard IOPS guidance with flat screen display (IOPS-FS), and the investigational IOPS with augmented reality headset (IOPS-AR), in a randomly assigned order. All participants used the same sensorized guidewire and steerable 6Fr catheter during their three cannulation tasks. A total of 26 participants were classified in three groups of experience: Group 1 (endovascular naïve; n = 13), Group 2 (surgeon in training; n = 12) and Group 3 (one expert surgeon). Primary endpoints included cannulation time and technical success, which was defined as the advancement of the catheter over the guidewire within the main body of the aortic stent graft within a maximum 15 minutes cutoff time for each trial. ResultsIn group 1, the mean cannulation time was shorter using IOPS-AR versus fluoroscopy (4.3 ±4.4 min vs 7.1±4.9 min; P=.04), but not statistically different when comparing IOPS-FS and fluoroscopy (6.3±4.5 min vs 7.1±4.9 min; P=.63). In group 1, technical success was 77% with fluoroscopy and 92% with both IOPS-FS and IOPS-AR (P=.59). In group 2, although there was no significant difference between cannulation time amongst the three different endovascular approaches, there was a trend towards shorter cannulation times with IOPS-FS or IOPS-AR as compared to fluoroscopy (mean time of 2.5±0.9, 4.4±4.0 and 5.2±4.5 min, respectively). In group 2, technical success was 92% with fluoroscopy and 100% with both IOPS-FS and IOPS-AR (P>.99). The expert vascular surgeon repeated the cannulation task four times for each endovascular approach, with 100% technical success and no difference in mean cannulation time between the imaging modalities (P=.89). ConclusionsAugmented reality allows for reducing the gate cannulation time as compared to fluoroscopy in participants with no previous exposure to any endovascular procedure. This suggests that augmented reality can be beneficial for individuals early in their career and can mitigate the learning curve. As individuals become experts, their ability to adapt to different endovascular modalities increases, eliminating the learning curve altogether.

Efficient and Accurate Semi-Automatic Neuron Tracing with Extended Reality.

Neuron tracing, alternately referred to as neuron reconstruction, is the procedure for extracting the digital representation of the three-dimensional neuronal morphology from stacks of microscopic images. Achieving accurate neuron tracing is critical for profiling the neuroanatomical structure at single-cell level and analyzing the neuronal circuits and projections at whole-brain scale. However, the process often demands substantial human involvement and represents a nontrivial task. Conventional solutions towards neuron tracing often contend with challenges such as non-intuitive user interactions, suboptimal data generation throughput, and ambiguous visualization. In this paper, we introduce a novel method that leverages the power of extended reality (XR) for intuitive and progressive semi-automatic neuron tracing in real time. In our method, we have defined a set of interactors for controllable and efficient interactions for neuron tracing in an immersive environment. We have also developed a GPU-accelerated automatic tracing algorithm that can generate updated neuron reconstruction in real time. In addition, we have built a visualizer for fast and improved visual experience, particularly when working with both volumetric images and 3D objects. Our method has been successfully implemented with one virtual reality (VR) headset and one augmented reality (AR) headset with satisfying results achieved. We also conducted two user studies and proved the effectiveness of the interactors and the efficiency of our method in comparison with other approaches for neuron tracing.