Augmented Reality Research Articles

Incorporating augmented reality into teaching marketing strategies: Perspectives from business education teachers and students

This study investigates the perceptions of business education instructors and their students regarding using augmented reality (AR) for teaching marketing strategies. Given the changes in education technologies in recent times, teaching strategies and approaches have changed. The study foresees that using AR, a digital resource essential for exposing students to real-life learning experiences, may improve students' learning experiences. The study employed a mixed-method research design to gather information from 112 individuals on the study's objectives. The researchers collected and evaluated data to investigate the participants' perspectives on utilizing augmented reality. The findings of this study reveal that although most studies assert that AR is effective in improving students' learning experiences, students and teachers acknowledged several drawbacks. This study indicates that if AR is used without effective teaching procedures and methodologies, it may not achieve the desired learning outcomes. Other findings were used to make conclusions on the study's objectives.

Intraoperative Position System guided with Augmented Reality Improves the Learning Curve of Endovascular Navigation in Endovascular Naïve Operators

ObjectiveThis study aimed to compare the completion of gate cannulation task performed by participants of varying experience using fluoroscopy, the Intra-Operative Positioning System (IOPS) – an FDA-cleared endovascular navigation system that has been developed to reduce dependence on fluoroscopy – or an investigational augmented reality electromagnetic navigation technology based on IOPS. MethodsThe task consisted in the cannulation of the gate of a GORE Excluder AAA endoprosthesis bifurcated aortic stent graft (W.L. GORE & Associates, Flagstaff, AZ USA) deployed into a 3D printed abdominal aortic aneurysm (AAA) model connected to a 7000 MDX flow pump (Sarns Inc/3M, Ann Arbor, MI USA) reproducing physiological conditions. The procedure was performed in a hybrid operating room (GE Allia IGS 7). Each participant performed the cannulation task with fluoroscopy, standard IOPS guidance with flat screen display (IOPS-FS), and the investigational IOPS with augmented reality headset (IOPS-AR), in a randomly assigned order. All participants used the same sensorized guidewire and steerable 6Fr catheter during their three cannulation tasks. A total of 26 participants were classified in three groups of experience: Group 1 (endovascular naïve; n = 13), Group 2 (surgeon in training; n = 12) and Group 3 (one expert surgeon). Primary endpoints included cannulation time and technical success, which was defined as the advancement of the catheter over the guidewire within the main body of the aortic stent graft within a maximum 15 minutes cutoff time for each trial. ResultsIn group 1, the mean cannulation time was shorter using IOPS-AR versus fluoroscopy (4.3 ±4.4 min vs 7.1±4.9 min; P=.04), but not statistically different when comparing IOPS-FS and fluoroscopy (6.3±4.5 min vs 7.1±4.9 min; P=.63). In group 1, technical success was 77% with fluoroscopy and 92% with both IOPS-FS and IOPS-AR (P=.59). In group 2, although there was no significant difference between cannulation time amongst the three different endovascular approaches, there was a trend towards shorter cannulation times with IOPS-FS or IOPS-AR as compared to fluoroscopy (mean time of 2.5±0.9, 4.4±4.0 and 5.2±4.5 min, respectively). In group 2, technical success was 92% with fluoroscopy and 100% with both IOPS-FS and IOPS-AR (P>.99). The expert vascular surgeon repeated the cannulation task four times for each endovascular approach, with 100% technical success and no difference in mean cannulation time between the imaging modalities (P=.89). ConclusionsAugmented reality allows for reducing the gate cannulation time as compared to fluoroscopy in participants with no previous exposure to any endovascular procedure. This suggests that augmented reality can be beneficial for individuals early in their career and can mitigate the learning curve. As individuals become experts, their ability to adapt to different endovascular modalities increases, eliminating the learning curve altogether.

Integration of Augmented Reality in Temporal Bone and Skull Base Surgeries.

Augmented reality technologies provide transformative solutions in various surgical fields. Our research focuses on the use of an advanced augmented reality system that projects 3D holographic images directly into surgical footage, potentially improving the surgeon's orientation to the surgical field and lowering the cognitive load. We created a novel system that combines exoscopic surgical footage from the "ORBEYE" and displays both the surgical field and 3D holograms on a single screen. This setup enables surgeons to use the system without using head-mounted displays, instead viewing the integrated images on a 3D monitor. Thirteen surgeons and surgical assistants completed tasks with 2D and 3D graphical surgical guides. The NASA Task Load Index was used to assess mental, physical, and temporal demands. The use of 3D graphical surgical guides significantly improved performance metrics in cochlear implant surgeries by lowering mental, physical, temporal, and frustration levels. However, for Bonebridge implantation, the 2D graphical surgical guide performed better overall (p = 0.045). Participants found the augmented reality system's video latency to be imperceptible, measuring 0.13 ± 0.01 s. This advanced augmented reality system significantly improves the efficiency and precision of cochlear implant surgeries by lowering cognitive load and improving spatial orientation.

Bibliometric analysis of augmented reality in chemistry education over the past decade

Numerous studies have delved into the application of augmented reality (AR) in chemistry education, focusing on specific topics, equipment requirements, and advantages. However, there remains a notable dearth of research examining the evolutionary characteristics of AR in this context. This study, employing bibliometric analysis on 66 articles spanning from 2012 to 2022, reveals that research primarily revolves around pedagogical approaches and AR technology development, particularly in kindergarten to 12th grade (K-12) education. Despite the United States exhibiting the highest publication frequency, there is a significant absence of studies addressing emotions, cognition, and physiological changes. Shedding light on these research gaps, this study underscores the need for further exploration into the cognitive, emotional, and physiological aspects of AR integration in chemistry education. Ultimately, the insights gleaned from this study offer valuable guidance for researchers, educators, and practitioners alike, facilitating the advancement and effective application of augmented reality in chemistry education (ARCE).

Expressive 3D Facial Animation Generation Based on Local-to-Global Latent Diffusion.

3D Facial animations, crucial to augmented and mixed reality digital media, have evolved from mere aesthetic elements to potent storytelling media. Despite considerable progress in facial animation of neutral emotions, existing methods still struggle to capture the authenticity of emotions. This paper introduces a novel approach to capture fine facial expressions and generate facial animations using audio synchronization. Our method consists of two key components: First, the Local-to-global Latent Diffusion Model (LG-LDM) tailored for authentic facial expressions, which can integrate audio, time step, facial expressions, and other conditions towards possible encoding of emotionally rich yet latent features in response to possibly noisy raw audio signals. The core of LG-LDM is our carefully designed Facial Denoiser Model (FDM) for aligning the local-to-global animation feature with audio. Second, we redesign an Emotion-centric Vector Quantized-Variational AutoEncoder framework (EVQ-VAE) to finely decode the subtle differences under different emotions and reconstruct the final 3D facial geometry. Our work significantly contributes to the key challenges of emotionally realistic 3D facial animation for audio synchronization and enhances the immersive experience and emotional depth in augmented and mixed reality applications. We provide a reproducibility kit including our code, dataset, and detailed instructions for running the experiments. This kit is available at https://github.com/wangxuanx/Face-Diffusion-Model.

Efficient and Accurate Semi-Automatic Neuron Tracing with Extended Reality.

Neuron tracing, alternately referred to as neuron reconstruction, is the procedure for extracting the digital representation of the three-dimensional neuronal morphology from stacks of microscopic images. Achieving accurate neuron tracing is critical for profiling the neuroanatomical structure at single-cell level and analyzing the neuronal circuits and projections at whole-brain scale. However, the process often demands substantial human involvement and represents a nontrivial task. Conventional solutions towards neuron tracing often contend with challenges such as non-intuitive user interactions, suboptimal data generation throughput, and ambiguous visualization. In this paper, we introduce a novel method that leverages the power of extended reality (XR) for intuitive and progressive semi-automatic neuron tracing in real time. In our method, we have defined a set of interactors for controllable and efficient interactions for neuron tracing in an immersive environment. We have also developed a GPU-accelerated automatic tracing algorithm that can generate updated neuron reconstruction in real time. In addition, we have built a visualizer for fast and improved visual experience, particularly when working with both volumetric images and 3D objects. Our method has been successfully implemented with one virtual reality (VR) headset and one augmented reality (AR) headset with satisfying results achieved. We also conducted two user studies and proved the effectiveness of the interactors and the efficiency of our method in comparison with other approaches for neuron tracing.

Teaching students with autism spectrum disorder how to use an augmented reality application and the vocabulary of the natural sciences

Technology-based interventions are considered evidence-based for teaching students with Autism Spectrum Disorder (ASD). The main purpose of this study is to teach students with ASD the use of augmented reality (AR) mobile applications through systematic instruction and to test the effectiveness of the presented AR mobile application in teaching science concepts. The secondary aim of this study was to examine the students’ and teachers’ views on the acceptability of the AR mobile application in teaching science vocabulary. Three primary and one secondary school students with ASD, aged 10-13 years, participated in the study. A multiple probe design across participants was used to evaluate the effects of the intervention package. The findings showed that systematic instruction using an AR mobile application is effective in science vocabulary for students with ASD. All students reported using an AR mobile application to learn science vocabulary words was socially acceptable. These results suggest that the AR mobile applications may be a promising instruction instrument that is effective, efficient, and cost-effective. Implications for future research and practice are then discussed.

Ten years of immersive education: Overview of a virtual and augmented reality course at postgraduate level

In recent years, the market has seen the emergence of numerous affordable sensors, interaction devices, and displays, which have greatly facilitated the adoption of Virtual and Augmented Reality (VR/AR) across various applications. However, developing these applications requires a solid understanding of the field and specific technical skills, which are often lacking in current Computer Science and Engineering education programs. This work details an extended version from a Eurographics 2024 Education Paper, reporting a post-graduate-level course that has been taught for the past ten years to almost 200 students, across several Master’s programs. The course introduces students to the fundamental principles, methods, and tools of VR/AR. Its primary objective is to equip students with the knowledge necessary to understand, create, implement, and evaluate applications using these technologies. The paper provides insights into the course structure, key topics covered, assessment methods, as well as the devices and infrastructure utilized. It also includes an overview of various practical projects completed over the years. Among other reflections, we discuss the challenges of teaching this course, particularly due to the rapid evolution of the field, which necessitates constant updates to the curriculum. Finally, future perspectives for the course are outlined.

Gesture2Text: A Generalizable Decoder for Word-Gesture Keyboards in XR Through Trajectory Coarse Discretization and Pre-Training.

Text entry with word-gesture keyboards (WGK) is emerging as a popular method and becoming a key interaction for Extended Reality (XR). However, the diversity of interaction modes, keyboard sizes, and visual feedback in these environments introduces divergent word-gesture trajectory data patterns, thus leading to complexity in decoding trajectories into text. Template-matching decoding methods, such as SHARK2 [32], are commonly used for these WGK systems because they are easy to implement and configure. However, these methods are susceptible to decoding inaccuracies for noisy trajectories. While conventional neural-network-based decoders (neural decoders) trained on word-gesture trajectory data have been proposed to improve accuracy, they have their own limitations: they require extensive data for training and deep-learning expertise for implementation. To address these challenges, we propose a novel solution that combines ease of implementation with high decoding accuracy: a generalizable neural decoder enabled by pre-training on large-scale coarsely discretized word-gesture trajectories. This approach produces a ready-to-use WGK decoder that is generalizable across mid-air and on-surface WGK systems in augmented reality (AR) and virtual reality (VR), which is evident by a robust average Top-4 accuracy of 90.4% on four diverse datasets. It significantly outperforms SHARK2 with a 37.2% enhancement and surpasses the conventional neural decoder by 7.4%. Moreover, the Pre-trained Neural Decoder's size is only 4 MB after quantization, without sacrificing accuracy, and it can operate in real-time, executing in just 97 milliseconds on Quest 3.

CGLight: An effective indoor illumination estimation method based on improved convmixer and GauGAN

Illumination consistency is a key factor for seamlessly integrating virtual objects with real scenes in augmented reality (AR) systems. High dynamic range (HDR) panoramic images are widely used to estimate scene lighting accurately. However, generating environment maps requires complex deep network architectures, which cannot operate on devices with limited memory space. To address this issue, this paper proposes CGLight, an effective illumination estimation method that predicts HDR panoramic environment maps from a single limited field-of-view (LFOV) image. We first design a CMAtten encoder to extract features from input images, which learns the spherical harmonic (SH) lighting representation with fewer model parameters. Guided by the lighting parameters, we train a generative adversarial network (GAN) to generate HDR environment maps. In addition, to enrich lighting details and reduce training time, we specifically introduce the color consistency loss and independent discriminator, considering the impact of color properties on the lighting estimation task while improving computational efficiency. Furthermore, the effectiveness of CGLight is verified by relighting virtual objects using the predicted environment maps, and the root mean square error and angular error are 0.0494 and 4.0607 in the gray diffuse sphere, respectively. Extensive experiments and analyses demonstrate that CGLight achieves a balance between indoor illumination estimation accuracy and resource efficiency, attaining higher accuracy with nearly 4 times fewer model parameters than the ViT-B16 model.

Tasks Reflected in the Eyes: Egocentric Gaze-Aware Visual Task Type Recognition in Virtual Reality.

With eye tracking finding widespread utility in augmented reality and virtual reality headsets, eye gaze has the potential to recognize users' visual tasks and adaptively adjust virtual content displays, thereby enhancing the intelligence of these headsets. However, current studies on visual task recognition often focus on scene-specific tasks, like copying tasks for office environments, which lack applicability to new scenarios, e.g., museums. In this paper, we propose four scene-agnostic task types for facilitating task type recognition across a broader range of scenarios. We present a new dataset that includes eye and head movement data recorded from 20 participants while they engaged in four task types across 15 360-degree VR videos. Using this dataset, we propose an egocentric gaze-aware task type recognition method, TRCLP, which achieves promising results. Additionally, we illustrate the practical applications of task type recognition with three examples. Our work offers valuable insights for content developers in designing task-aware intelligent applications. Our dataset and source code are available at zhimin-wang.github.io/TaskTypeRecognition.html.

Should I Evaluate My Augmented Reality System in an Industrial Environment? Investigating the Effects of Classroom and Shop Floor Settings on Guided Assembly.

Numerous prior studies have investigated real-time assembly instructions using Augmented Reality (AR). However, most such experiments were conducted in laboratory settings with simplistic assembly tasks, failing to represent real-world industrial conditions. To ascertain to what extent results obtained in a laboratory environment may differ from studies in actual industrial environments, we carried out a user study with 32 manufacturing apprentices. We compared assembly task execution results in two settings, a classroom and an industrial workshop environment. To facilitate the experiments, we developed AR-guided manual assembly systems for simple and more complex assets. Our findings reveal a significantly improved task performance in the industrial workshop, reflected in faster task completion times, fewer errors, and subjectively perceived higher flow. This contradicted participants' subjective ratings, as they expected to perform better in the classroom environment. Our results suggest that the actual manufacturing environment is critical in evaluating AR systems for real-world industrial applications.

Utilization of learning media based on augmented reality on design material network topology

Utilization of learning media based on augmented reality on design material network topology

Searching Across Realities: Investigating ERPs and Eye-Tracking Correlates of Visual Search in Mixed Reality.

Mixed Reality allows us to integrate virtual and physical content into users' environments seamlessly. Yet, how this fusion affects perceptual and cognitive resources and our ability to find virtual or physical objects remains uncertain. Displaying virtual and physical information simultaneously might lead to divided attention and increased visual complexity, impacting users' visual processing, performance, and workload. In a visual search task, we asked participants to locate virtual and physical objects in Augmented Reality and Augmented Virtuality to understand the effects on performance. We evaluated search efficiency and attention allocation for virtual and physical objects using event-related potentials, fixation and saccade metrics, and behavioral measures. We found that users were more efficient in identifying objects in Augmented Virtuality, while virtual objects gained saliency in Augmented Virtuality. This suggests that visual fidelity might increase the perceptual load of the scene. Reduced amplitude in distractor positivity ERP, and fixation patterns supported improved distractor suppression and search efficiency in Augmented Virtuality. We discuss design implications for mixed reality adaptive systems based on physiological inputs for interaction.

Exploring the Effect of Viewing Attributes of Mobile AR Interfaces on Remote Collaborative and Competitive Tasks.

Mobile devices have the potential to facilitate remote tasks through Augmented Reality (AR) solutions by integrating digital information into the real world. Although prior studies have explored Mobile Augmented Reality (MAR) for co-located collaboration, none have investigated the impact of various viewing attributes that can influence remote task performance, such as target object viewing angles, synchronization styles, or having a secondary small screen showing other users current view in the MAR environment. In this paper, we explore five techniques considering these attributes, specifically designed for two modes of remote tasks: collaborative and competitive. We conducted a user study employing various combinations of those attributes for both tasks. In both instances, results indicate users' optimal performance and preference for the technique that allows asynchronous viewing of object manipulations on the small screen. Overall, this paper contributes novel techniques for remote tasks in MAR, addressing aspects such as viewing angle and synchronization in object manipulation alongside secondary small-screen interfaces. Additionally, it presents the results of a user study evaluating the effectiveness, usability, and user preference of these techniques in remote settings and offers a set of recommendations for designing and implementing MAR solutions to enhance remote activities.

Augmented Reality Control based Energy Management System for Residence

Introduction: The roots of virtual reality (VR) go back to the work of Ivan Sutherland in the 1960s. At the time, virtual reality showed promise in universities and labs, peaking in the early 1990s at a stable level. The latest form of IoT is merging with the Internet of Everything (IOE). Using this technology, users can access any device, device, and machine in the environment through any middleware application or system. In recent years, smart speakers have become a common commercial method for controlling household appliances, solving some of the problems of remote control. For example, speech recognition is difficult in noisy environments and is minimized using noise cancellation. It also takes a long time to determine the current state of the device as the user need to request or issue a command to the smart speaker and need to wait for a response. The evaluation of prototype system found, using a device with gestures and interacting with a virtual 3D device instead of a real device was acceptable in terms of usability. In this article, the Unity and Vuforia are used to create an AR-based IoT virtual switch that can control any device connected to a LAN controller via HTTP requests. Methods: The system is made experimental with the Augmented Reality (AR), the virtual communication system is designed instead of physical mode. It provide the real time based image for active control of Energy management. It consist of virtual switches, it supports the AR based power control system. Results: The proposed system are made evaluated and tested under various load conditions. The virtual image consist of virtual switches to make real time control of power equipment. The proposed system achieve the latency of about 0.06 sec. Conclusion: The advancement in day today activities required with AR based energy management system with proper control measures. The designed system is supportive for such mechanism. The further system is proposed to make more energy saving concept for enhancement in proposed work.

Moving objects detection based on histogram of oriented gradient algorithm chip for hazy environment

<span>The most important aspects of computer vision are moving object detection (MOD) and tracking. Many signal-processing applications use regional image statistics. Compute-intensive video and image processing with low latency and high throughput is done with field programmable gate array (FPGA) image processing. Local image statistics are used for edge identification and filtering. The histogram of oriented gradients (HoG) algorithm extracts local shape characteristics by equalizing histograms. The objective of the work is to design the hardware chip of the algorithm and perform the simulation in the Xilinx ISE 14.7 simulation environment. The performance of the chip is evaluated in Modelsim 10.0 simulation software to check its feasibility. The performance of the chip design is estimated on Viretx-5 FPGA and compared with the MATLAB-2020 image processing tool-based response time. This form of tracking typically deals with identifying, anchoring, and tracking images and videos. A mask made from a cut-out of the object can then determine the plane's coordinates depending on its position. This type of object tracking is frequently utilized in the field of augmented reality (AR). The algorithm is most suited for object detection using hardware controllers in haze and foggy environments.</span>

Investigating Object Translation in Room-scale, Handheld Virtual Reality.

Handheld devices have become an inclusive alternative to head-mounted displays in virtual reality (VR) environments, enhancing accessibility and allowing cross-device collaboration. Object manipulation techniques in 3D space with handheld devices, such as those in handheld augmented reality (AR), have been typically evaluated in table-top scale and we currently lack an understanding of how these techniques perform in larger scale environments. We conducted two studies, each with 30 participants, to investigate how different techniques impact usability and performance for room-scale handheld VR object translations. We compared three translation techniques that are similar to commonly studied techniques in handheld AR: 3DSlide, VirtualGrasp, and Joystick. We also examined the effects of target size, target distance, and user mobility conditions (stationary vs. moving). Results indicated that the Joystick technique, which allowed translation in relation to the user's perspective, was the fastest and most preferred, without difference in precision. Our findings provide insights for designing room-scale handheld VR systems, with potential implications for mixed reality systems involving handheld devices.

RingGesture: A Ring-Based Mid-Air Gesture Typing System Powered by a Deep-Learning Word Prediction Framework.

Text entry is a critical capability for any modern computing experience, with lightweight augmented reality (AR) glasses being no exception. Designed for all-day wearability, a limitation of lightweight AR glass is the restriction to the inclusion of multiple cameras for extensive field of view in hand tracking. This constraint underscores the need for an additional input device. We propose a system to address this gap: a ring-based mid-air gesture typing technique, RingGesture, utilizing electrodes to mark the start and end of gesture trajectories and inertial measurement units (IMU) sensors for hand tracking. This method offers an intuitive experience similar to raycast-based mid-air gesture typing found in VR headsets, allowing for a seamless translation of hand movements into cursor navigation. To enhance both accuracy and input speed, we propose a novel deep-learning word prediction framework, Score Fusion, comprised of three key components: a) a word-gesture decoding model, b) a spatial spelling correction model, and c) a lightweight contextual language model. In contrast, this framework fuses the scores from the three models to predict the most likely words with higher precision. We conduct comparative and longitudinal studies to demonstrate two key findings: firstly, the overall effectiveness of RingGesture, which achieves an average text entry speed of 27.3 words per minute (WPM) and a peak performance of 47.9 WPM. Secondly, we highlight the superior performance of the Score Fusion framework, which offers a 28.2% improvement in uncorrected Character Error Rate over a conventional word prediction framework, Naive Correction, leading to a 55.2% improvement in text entry speed for RingGesture. Additionally, RingGesture received a System Usability Score of 83 signifying its excellent usability.

Augmented and Virtual Reality Based Segmentation Algorithm for Human Pose Detection in Wearable Cameras

Pose graph optimisation is a crucial method that helps reduce cumulative errors while estimating visual trajectories for wearable cameras. However, when the posture graph's size increases with each additional camera movement, the optimization's efficiency diminishes. In terms of ongoing sensitive applications, such as extended reality and computer-generated reality, direction assessment is a major test. This research proposes an incremental pose graph segmentation technique that accounts for camera orientation variations as a solution to this challenge. The computation only improves the cameras that have seen large direction changes by breaking the posture chart during these instances. As a result, pose graph optimisation is essentially slowed down and optimised more quickly. For every camera that hasn't been optimised using a pose graph, the algorithm employs the wearable cameras at the start and end of each camera's trajectory segment. The final camera in attendance is then determined by weighted average the various postures evaluated with these wearable cameras; this eliminates the need for lengthy nonlinear enhancement computations, reduces disturbance, and achieves excellent accuracy. Experiments on the EuRoC, TUM, and KITTI datasets demonstrate that pose graph optimisation scope is reduced while maintaining camera trajectories accuracy.