Augmented Reality Support Research Articles

The effect of augmented reality storybooks on the story comprehension and retelling of preschool children.

This study aimed to compare the retelling and story comprehension performance of two groups of preschool children-an experimental and a control group-who experienced printed and augmented reality storybooks. The participant group consisted of 90 participants, with 45 in the experimental group (22 girls, 23 boys) and 45 in the control group (21 girls, 24 boys). The average age of the children was 54.2 months. In the study, the researcher evaluated children's story-retelling performance using the rubric and used a Story Comprehension Test to measure their story comprehension performance. The researchers utilized the ROAR application to incorporate augmented reality content into the books. In the pre-test phase, the teachers read the designated texts to the children in the experimental and control groups. After the eight-week break, the control group experienced the same texts with printed books, while the experimental group experienced them with augmented reality support. Each child participated individually in the reading process with the teacher. In the study, teachers asked the children to retell the story and asked the questions from the Story Comprehension Test at the end of each book reading section for pre-test and post-test measurements. The pre-test results revealed no significant difference in the two groups' story-retelling and Story Comprehension Test scores. The post-test results indicated a significant difference in the story-retelling performance and Story Comprehension Test scores between the experimental and control groups, favoring the experimental group. Based on these findings, the augmented reality content can potentially enhance children's retelling and story comprehension performances.

Augmented Reality in Extratemporal Lobe Epilepsy Surgery.

Background: Epilepsy surgery for extratemporal lobe epilepsy (ETLE) is challenging, particularly when MRI findings are non-lesional and seizure patterns are complex. Invasive diagnostic techniques are crucial for accurately identifying the epileptogenic zone and its relationship with surrounding functional tissue. Microscope-based augmented reality (AR) support, combined with navigation, may enhance intraoperative orientation, particularly in cases involving subtle or indistinct lesions, thereby improving patient outcomes and safety (e.g., seizure freedom and preservation of neuronal integrity). Therefore, this study was conducted to prove the clinical advantages of microscope-based AR support in ETLE surgery. Methods: We retrospectively analyzed data from ten patients with pharmacoresistant ETLE who underwent invasive diagnostics with depth and/or subdural grid electrodes, followed by resective surgery. AR support was provided via the head-up displays of the operative microscope, with navigation based on automatic intraoperative computed tomography (iCT)-based registration. The surgical plan included the suspected epileptogenic lesion, electrode positions, and relevant surrounding functional structures, all of which were visualized intraoperatively. Results: Six patients reported complete seizure freedom following surgery (ILAE 1), one patient was seizure-free at the 2-year follow-up, and one patient experienced only auras (ILAE 2). Two patients developed transient neurological deficits that resolved shortly after surgery. Conclusions: Microscope-based AR support enhanced intraoperative orientation in all cases, contributing to improved patient outcomes and safety. It was highly valued by experienced surgeons and as a training tool for less experienced practitioners.

Teaching three-phase half-wave power electronic rectifier with gamified augmented reality support

This paper presents a support tool designed for instructors in technical education within engineering programs, focusing specifically on power electronics and laboratory procedures related to the three-phase half-wave rectifier. The Augmented Reality (AR) application was developed for Android devices using Unity+Vuforia, employing a gamification-based approach. The application allows students to evaluate their knowledge and skills in real-time and offers an interactive learning environment. It provides learning diagnostics and sends each student's evaluation to a database stored on Google-Forms that is accessible by the instructor or assessor in question. It's important to note that this APP is freely accessible through the Android APP store. The learning strategy is around a game that tests players on their knowledge of measuring devices and theoretical concepts. Additionally, there is a puzzle task that encourages learners to correctly arrange the waveforms in a rectifier. Students can also connect the half-wave rectifier's essential connections and the transformer's delta-star connections through touch interaction. A comparison study between traditional teaching methods and the innovative AR-based approach has been conducted. This educational proposal itself is an innovative solution that will be especially helpful for laboratories lacking useful equipment in Mexican colleges. To test the effect of AR application on student success, an experimental study was conducted on two groups of students (a control group [non-using AR] and a test group [using AR]). To determine significant difference in acquired learning, an ANOVA statistical analysis was conducted using independent sample tests. According to tests result, there is a significant difference in the acquired knowledge for those who used the designed tool. The probability value (p = 1e-5 <0.05) for the use of AR is found. They increase their initial average grade from 27.08% to 72.3%, in contrast to the control group with no significant increasing learning, reporting an initial average grade of 28.75–43.33%. These results show that the use of AR in electronic practice makes a significant difference in student success.

Effect of Augmented Reality Support on Quality Inspection of Welded Structures

With the increasing complexity of production systems and manufactured products, operators face high demands for professional expertise and long-term concentration. Augmented reality (AR) can support users in their work by displaying relevant virtual data in their field of view. In contrast to the extensive research on AR assistance in assembly, maintenance, and training, AR support in quality inspection has received less attention in the industrial context. Quality inspection is an essential part of industrial processes; thus, it is important to verify whether AR assistance can support users in these tasks. This work proposes an AR-based approach for quality inspection. For this, pilot AR software was designed and developed. The proposed AR approach was tested with end users. The task efficiency, the error rate, the perceived mental workload, and the usability of the AR approach were analysed and compared to the conventional paper-based support. The field research confirmed the positive effect on user efficiency during quality inspection while decreasing the perceived mental workload. This work extends the research on the potential of AR assistance in industrial applications and provides experimental validation to confirm the benefits of AR support on user performance during quality inspection tasks.

Augmented Reality Support for Anterior Decompression and Fusion Using Floating Method for Cervical Ossification of the Posterior Longitudinal Ligament.

Anterior decompression and fusion (ADF) using the floating method for cervical ossification of the posterior longitudinal ligament (OPLL) is an ideal surgical technique, but it has a specific risk of insufficient decompression caused by the impingement of residual ossification. Augmented reality (AR) support is a novel technology that enables the superimposition of images onto the view of a surgical field. AR technology was applied to ADF for cervical OPLL to facilitate intraoperative anatomical orientation and OPLL identification. In total, 14 patients with cervical OPLL underwent ADF with microscopic AR support. The outline of the OPLL and the bilateral vertebral arteries was marked after intraoperative CT, and the reconstructed 3D image data were transferred and linked to the microscope. The AR microscopic view enabled us to visualize the ossification outline, which could not be seen directly in the surgical field, and allowed sufficient decompression of the ossification. Neurological disturbances were improved in all patients. No cases of serious complications, such as major intraoperative bleeding or reoperation due to the postoperative impingement of the floating OPLL, were registered. To our knowledge, this is the first report of the introduction of microscopic AR into ADF using the floating method for cervical OPLL with favorable clinical results.

Designing interactive glazing through an engineering psychology approach: Six augmented reality scenarios that envision future car human-machine interface

BackgroundWith an increasing number of vehicles becoming autonomous, intelligent, and connected, paying attention to the future usage of car human-machine interface with these vehicles should become more relevant. Several studies have addressed car HMI but were less attentive to designing and implementing interactive glazing for every day (autonomous) driving contexts. MethodsReflecting on the literature, we describe an engineering psychology practice and the design of six novel future user scenarios, which envision the application of a specific set of augmented reality (AR) support user interactions. Additionally, we conduct evaluations on specific scenarios and experiential prototypes, which reveal that these AR scenarios aid the target user groups in experiencing a new type of interaction. The overall evaluation is positive with valuable assessment results and suggestions. ConclusionsThis study can interest applied psychology educators who aspire to teach how AR can be operationalized in a human-centered design process to students with minimal pre-existing expertise or minimal scientific knowledge in engineering psychology.

General Software Platform and Content Description Format for Assembly and Maintenance Task Based on Augmented Reality

Augmented reality (AR) support systems have proven to be effective in supporting various tasks, but are not yet widely used in real-world applications due to their high development costs. Although several AR authoring tools have been proposed for content development to solve this problem, most AR systems developed with these tools are not compatible with industrial environments and task types, and have inflexible visualization styles. This study provides a systematic solution of combining the AR task support software platform and the general contents description format by expanding the degree of freedom of the visualization methods and improving the design process. The proposed solution adapts to different activities and environments, and optimizes content implementation and user experience. The user study was conducted to evaluate the task performance including processing time and errors and the user experience including the perceived cognitive load and usability of our solution by comparing with a conventional system. The results show that the proposed software platform could improve the compatibility with industrial environments and tasks, and reduce the workload and cognitive effort, although the task performance was the same as the conventional AR system.

Head-Mounted Augmented Reality Support for Assemblers of Wooden Trusses

Wooden-house assembly is an area where still a big part of the work is done manually. In this case study, pairs of operators compose large wooden pieces together based on paper-print instructions complemented by visual guidance in the shape of laser marks projected from lasers mounted in the ceiling, based on Computer-aided design (CAD) data. Augmented Reality (AR) head-mounted displays (HMD) offer a unique platform for providing instructions and additional information superimposed in the work environment and thus can provide guidance in a cognitively ergonomic way. A particular advantage compared to other computing platforms is that the operators have free hands and can perform the manual work and follow guidance simultaneously. We present an evaluation of a prototype that dynamically transforms a CAD data file with design and measurements of wooden trusses to be manufactured, into an AR-based guidance system developed in Unity for Microsoft HoloLens 2 devices. We used an iterative participatory design process for prototyping and think-aloud protocol combined with observations for evaluation, involving professional assemblers in different stages of the process. Participants found the solution to potentially save time in their everyday work and simplify the task by offering increased visibility of the marks compared to the existing laser projection. Large-scale deployment of the system is still facing design challenges of which some are also discussed in the paper.

Augmented reality support to employ tacit knowledge in non-conforming operations

Organisations often rely, implicitly, on the tacit knowledge of their employees, especially in non-conforming operations. These activities include (dis)assembly tasks, manual machining, troubleshooting, and tailored adjustments. While a traditional master-apprentice approach is a proven way to transfer tacit knowledge between employees, this approach is not feasible in current industrial situations. This research proposes a contemporary master-apprentice approach to support and improve non-conforming operations in situations that cannot rely on all parties being present at the same time, in the same location, while co-operating on the same task. The approach demonstrates the use of augmented reality as an implicit and discreet tool that purposefully addresses both capturing tacit knowledge, as well as employing it during operations - without disruptive interference in the primary processes.

Innovation in engineering education using AR technology on the example of disciplines “Descriptive Geometry” and “Engineering Graphics”

Research relevance. With the use of computer-electronic-digital equipment, the achievements of scientific and technological progress become part of education. This obliges to improve the field of education: to change and create the forms and content of education, methods and approaches to the educational process, to increase the volume and quality of knowledge. Augmented reality favors autonomous learning by using mobile devices, simplifies the understanding and interpretation of geometric shapes and contributes to the development of students' spatial thinking. The research purpose is to form new knowledge, skills and abilities for the convergence of the real and digital worlds based on the possibility of using augmented reality technology, to determine the reasons that make it difficult for users to accept information technology as an educational tool. Materials and methods. Scientific papers published between 2013 and 2022 in the scientometric databases Scopus, Web of Science, Google Scholar, E-library were considered, pedagogical experience on the research topic was studied; various types of diagnostics of students were taken into account with summing up the outcomes and the results of theoretical and practical research were summarized. A practical study of the problem was carried out by creating a 3D model of octants and projections of points. Research results. An augmented reality application was created on the Unity 2018.4.12f1 platform with additional modules – Vuforia Augmented Reality Support and Android Build Support. When developing the application, the authors created 3D models of octants, projections of points in the SketchUp program. A survey on conscious understanding and mastering the descriptive geometry course showed that the majority of students showed positive dynamics in mastering the descriptive geometry course. Conclusion. As a result of using a specially developed application based on the augmented reality platform, it is possible to move from standardized educational trajectories to individual ones, which significantly increases the effectiveness of modes of study and restructuring of the educational process.

The HORSE framework: A reference architecture for cyber-physical systems in hybrid smart manufacturing

In the Industry 4.0 era, manufacturers strive to remain competitive by using advanced technologies such as collaborative robots, automated guided vehicles, augmented reality support and smart devices. However, only if these technological advancements are integrated into their system context in a seamless way, they can deliver their full potential to a manufacturing organization. This integration requires a system architecture as a blueprint for positioning and interconnection of the technologies. For this purpose, the HORSE framework, resulting from the HORSE EU H2020 project, has been developed to act as a reference architecture of a cyber-physical system to integrate various Industry 4.0 technologies and support hybrid manufacturing processes, i.e., processes in which human and robotic workers collaborate. The architecture has been created using design science research, based on well-known software engineering frameworks, established manufacturing domain standards and practical industry requirements. The value of a reference architecture is mainly established by application in practice. For this purpose, this paper presents the application and evaluation of the HORSE framework in 10 manufacturing plants across Europe, each with its own characteristics. Through the physical deployment and demonstration, the framework proved its goal to be basis for the well-structured design of an operational smart manufacturing cyber-physical system that provides horizontal, cross-functional management of manufacturing processes and vertical control of heterogeneous technologies in work cells. We report on valuable insights on the difficulties to realize such systems in specific situations. The experiences form the basis for improved adoption, further improvement and extension of the framework. In sum, this paper shows how a reference architecture framework supports the structured application of Industry 4.0 technologies in manufacturing environments that so far have relied on more traditional digital technology.

Augmented reality support for self-service automated systems

Augmented Reality (AR) is a way of looking at the real environment in which the computer-generated environment is located. The use of augmented reality changes the way we interact with the real environment. In the past few years, with the development of technology, augmented reality is gaining in importance and becoming closer to everyday users. Many companies such as Google, Sony, HTC and Samsung are investing heavily in the development of this field. Augmented reality is finding usage in various areas such as entertainment, marketing, navigation, surveillance, system maintenance, and user training. The purpose of this paper is to present the possibilities of using augmented reality systems for the needs of self-service automated systems.

Spine Surgery Supported by Augmented Reality.

Study Design:A prospective, case-based, observational study.Objectives:To investigate how microscope-based augmented reality (AR) support can be utilized in various types of spine surgery.Methods:In 42 spinal procedures (12 intra- and 8 extradural tumors, 7 other intradural lesions, 11 degenerative cases, 2 infections, and 2 deformities) AR was implemented using operating microscope head-up displays (HUDs). Intraoperative low-dose computed tomography was used for automatic registration. Nonlinear image registration was applied to integrate multimodality preoperative images. Target and risk structures displayed by AR were defined in preoperative images by automatic anatomical mapping and additional manual segmentation.Results:AR could be successfully applied in all 42 cases. Low-dose protocols ensured a low radiation exposure for registration scanning (effective dose cervical 0.29 ± 0.17 mSv, thoracic 3.40 ± 2.38 mSv, lumbar 3.05 ± 0.89 mSv). A low registration error (0.87 ± 0.28 mm) resulted in a reliable AR representation with a close matching of visualized objects and reality, distinctly supporting anatomical orientation in the surgical field. Flexible AR visualization applying either the microscope HUD or video superimposition, including the ability to selectively activate objects of interest, as well as different display modes allowed a smooth integration in the surgical workflow, without disturbing the actual procedure. On average, 7.1 ± 4.6 objects were displayed visualizing target and risk structures reliably.Conclusions:Microscope-based AR can be applied successfully to various kinds of spinal procedures. AR improves anatomical orientation in the surgical field supporting the surgeon, as well as it offers a potential tool for education.

Olfactory-Based Augmented Reality Support for Industrial Maintenance

Augmented reality (AR) applications have opened innovative ways for performance improvement in the IoT industry. It can enhance user perception of the real-world by providing valuable information about an industrial environment and provide visual virtual information onto a head-mounted device (HMD). Such information is important for maintainers to quickly detect abnormalities, reduce nugatory routines and facilitate preventive maintenance activities. Since odors are made up of volatile compounds at low concentration, they can be used for olfactory-based identification. The article outlines the development of an olfactory-based AR system to help with the identification of maintenance issues using smell. The prototype comprises of three components: an electronic nose, a database and an AR application integrated with Microsoft HoloLens. After diagnosing an odor, the data is sent wirelessly through a local network to the HMD worn by the user. To validate the technology, four odors have been used, including engine oil, sun lotion, medical alcohol and perfume, to record behaviors and demonstrate the repeatability of the process. The presented technology incorporates sampling methods, cleaning processes and statistical analysis that can be further scrutinized to allow better smell augmentation for diagnosis.

HoloInjection: augmented reality support for CT-guided spinal needle injections.

The correct placement of needles is decisive for the success of many minimally-invasive interventions and therapies. These needle insertions are usually only guided by radiological imaging and can benefit from additional navigation support. Augmented reality (AR) is a promising tool to conveniently provide needed information and may thus overcome the limitations of existing approaches. To this end, a prototypical AR application was developed to guide the insertion of needles to spinal targets using the mixed reality glasses Microsoft HoloLens. The system's registration accuracy was attempted to measure and three guidance visualisation concepts were evaluated concerning achievable in-plane and out-of-plane needle orientation errors in a comparison study. Results suggested high registration accuracy and showed that the AR prototype is suitable for reducing out-of-plane orientation errors. Limitations, like comparatively high in-plane orientation errors, effects of the viewing position and missing image slices indicate potential for improvement that needs to be addressed before transferring the application to clinical trials.

Microscope-Based Augmented Reality in Degenerative Spine Surgery: Initial Experience

To establish microscope-based augmented reality (AR) support for degenerative spine surgery. Head-up displays of operating microscopes were used to establish AR in a series of 10 patients. Segmentation of the vertebra and additional target structures, which were visualized by AR, was based on preoperative magnetic resonance and computed tomography (CT) images, that were nonrigidly fused to low-dose intraoperative CT (iCT) data. AR registration was achieved by automatic registration applying iCT and microscope calibration. AR support could be smoothly implemented in the surgical workflow. AR allowed to visualize the target structures reliably in the surgical field, facilitating surgical orientation. Flexible placement of the reference array enabled AR implementation for anterior, lateral, posterior median, and posterior paramedian approaches. Identification of bony and artificial landmarks allowed validating registration accuracy; the measured target registration error was 1.11 ± 0.42 mm (mean ± standard deviation). The effective dose for registration scanning ranged from 0.52 to 8.71 mSv, which is on average about one-third of a standard diagnostic spine scan. This depended mainly on the scan length (mean scan length cervical/thoracic/lumbar: 99/218/118 mm). Longest scan ranges were in the mid-thoracic region to ensure unambiguous vertebra assignment as prerequisite for reliable nonlinear registration (mean cervical/thoracic/lumbar effective dose: 0.52/6.14/2.99 mSv). Reliable microscope-based AR support is possible because of automatic registration based on intraoperative imaging. Application of AR in degenerative spine surgery has a big potential; it might be especially helpful in complex anatomical situations and resident education.

Implementation of augmented reality support in spine surgery.

To implement a straightforward workflow that allows to establish augmented reality (AR) support in spine surgery. Intraoperative computed tomography (iCT) applying a 32-slice movable scanner was used for navigation registration in a series of 10 patients who underwent surgery for extra- or intradural spinal lesions. Preoperative multimodal image data were integrated by nonlinear registration with the iCT images. Automatic segmentation was used to delineate the 3-dimensional (3-D) outline of the vertebra, and in addition, the tumor extent, as well as implants, was segmented and visualized. Automatic patient registration without user interaction resulted in high navigation accuracy with a mean registration error of only about 1mm. Moreover, the workflow for establishing AR was straightforward and could be easily integrated in the normal surgical procedure. Low-dose iCT protocols resulted in a radiation exposure of 0.35-0.98mSv for cervical, 2.16-6.92mSv for thoracic, and 3.55-4.20mSv for lumbar surgeries, which is a reduction in the effective radiation dose by 70%. The segmented structures were intuitively visualized in the surgical field using the heads-up display of the operating microscope. In parallel, the microscope video was superimposed with the segmented 3-D structures, which were visualized in a semitransparent manner along with various display modes of the image data. A microscope-based AR environment was successfully implemented for spinal surgery. The application of iCT for registration imaging ensures high navigational accuracy. AR greatly supports the surgeon in understanding the 3-D anatomy thereby facilitating surgery. These slides can be retrieved under Electronic Supplementary Material.

Practical Insights On Augmented Reality Support for Shop-Floor Tasks

The use of Augmented Reality (AR) to improve processes in production has a long history and there are various established use cases in industry and research. Nevertheless, barriers to its seamless integration in everyday shop-floor processes still remain. A holistic approach grounded on socio-technical systems theory and a practical use case in a real-world production environment serve to identify the barriers and to provide recommendations for implementing AR in shop-floor processes. As a starting point, an overview of use cases in industry and research is provided focusing on how AR is used to support shop-floor tasks. Based on this, a practical use case, where maintenance engineers were guided by a head-mounted AR device (Microsoft HoloLens), was implemented to gain insights into the current AR-technology. An experiment was carried out consisting of twelve test runs with six maintenance engineers in a real-world production setting in the automotive industry. Semi-structured interviews with the test persons and video recordings of the tests serve as a basis for data analysis. The results consist primarily of specific use case feedback from test persons and video-based observation findings. Based on these findings, we derived recommendations to seamlessly integrate AR into production processes. These results are structured around enterprise IT infrastructure, AR-technology, shop-floor process related and human-centered aspects.

THE SECURITY AND PRIVACY PROBLEMS OF AUGMENTED REALITY TECHNOLOGIES

The analysis of security and confidential problems, connected with augmented reality systems and support technologies have been conducted. These problems have been analyzed using 2 categories: system volume and functionality. In the first category the augmented reality systems (AR-systems) of growing scale: single applications, multiple applications within a single AR platform, and multiple communicating AR-systems have been reviewed. For this category the problems connected with input, output and data access have been analyzed. Based on the described problems of information security in AR-technologies, the main directions of their solutions have been indicated.

On-the-fly augmented reality for orthopedic surgery using a multimodal fiducial.

Fluoroscopic x-ray guidance is a cornerstone for percutaneous orthopedic surgical procedures. However, two-dimensional (2-D) observations of the three-dimensional (3-D) anatomy suffer from the effects of projective simplification. Consequently, many x-ray images from various orientations need to be acquired for the surgeon to accurately assess the spatial relations between the patient's anatomy and the surgical tools. We present an on-the-fly surgical support system that provides guidance using augmented reality and can be used in quasiunprepared operating rooms. The proposed system builds upon a multimodality marker and simultaneous localization and mapping technique to cocalibrate an optical see-through head mounted display to a C-arm fluoroscopy system. Then, annotations on the 2-D x-ray images can be rendered as virtual objects in 3-D providing surgical guidance. We quantitatively evaluate the components of the proposed system and, finally, design a feasibility study on a semianthropomorphic phantom. The accuracy of our system was comparable to the traditional image-guided technique while substantially reducing the number of acquired x-ray images as well as procedure time. Our promising results encourage further research on the interaction between virtual and real objects that we believe will directly benefit the proposed method. Further, we would like to explore the capabilities of our on-the-fly augmented reality support system in a larger study directed toward common orthopedic interventions.