Augmented Reality Software Research Articles

The Instantiation of Narrative in Picturebook Forms

Narratives have been instantiated across an array of materials, media, and modalities, including oral storytelling, film, novels, tapestries, comics, theatre, and children’s picturebooks for centuries. Narrative picturebooks come in many different forms, contain a variety of modalities, primarily written and oral language, visual images, and design elements, and are produced using an array of material and production technologies. Like the continuing evolution of narratives in general, narrative picturebooks are part of an extended history of visual and multimodal communication and have been instantiated as printed codices, moveable or pop-up books, volvelles, electronic books, and as digital and augmented reality software applications. This article focuses on the semiotic, material, and ideological aspects of the instantiation of narrative in picturebook form and examines how narrative picturebooks are embedded in an array of social, cultural, and literacy practices. Additionally, the article proposes a framework from which to consider the syntactical, modal, compositional, mediational, material, technological, historical, and sociocultural aspects of the instantiation of narrative in picturebook forms.

Augmented Reality Software in Presentation of Cultural and Historical Heritage

This paper presents the advent in the presentation of cultural and historical heritage by using augmented reality (AR) technology. The use cases are given on the examples of mobile applications realized as electronic multimedia guides for various cultural institutions in Serbia. Then, a classification of the AR software is proposed based on their usage in the given examples.

Resilience Informatics: Role of Informatics in Enabling and Promoting Public Health Resilience to Pandemics, Climate Change, and Other Stressors.

Climate change, local epidemics, future pandemics, and forced displacements pose significant public health threats worldwide. To cope successfully, people and communities are faced with the challenging task of developing resilience to these stressors. Our viewpoint is that the powerful capabilities of modern informatics technologies including artificial intelligence, biomedical and environmental sensors, augmented or virtual reality, data science, and other digital hardware or software, have great potential to promote, sustain, and support resilience in people and communities. However, there is no "one size fits all" solution for resilience. Solutions must match the specific effects of the stressor, cultural dimensions, social determinants of health, technology infrastructure, and many other factors.

Augmented Reality (AR) in Surgery in Low- and Middle-Income Countries (LMICs): A Scoping Review.

Surgical disparities persist in low- and middle-income countries (LMICs). Insufficient access to surgical care places a large burden on these regions, with high mortality rates for otherwise standard procedures performed in high-income countries (HICs). Augmented Reality (AR) and Virtual Reality (VR) now provide us with a platform to improve the delivery of surgical access and training to LMICs. The use of AR technologies to provide additional training to surgeons and residents globally can help bridge the gap and reduce health disparities in LMICs. The goal of this scoping review is to evaluate whether surgical trainees and surgeons from LMICs have access to or use AR software in their training or practice. A systematic search was conducted on seven databases. Inclusion criteria included populations in LMICs with access to AR-based training. Articles using VR software, or those conducted in HICs were excluded from the review. From the 428 records screened, 58 reports were assessed for eligibility, and of these, a total of six studies were included in the review. Five of the six studies used mentors from an HIC, including the United States (US) and the United Kingdom (UK), whereas one study had mentorship from another LMIC. Three surgical specialties were explored: neurosurgery, plastic surgery, and urology. Although the integration of AR in surgical training is promising, the six studies evaluated in this review emphasize that costs and connection issues are major challenges that can set back these technologies in the operating room. Despite these revelations, with certain improvements, AR training programs are promising as they can help to reduce the global disparity in surgical proficiency.

Enhancing Online Furniture Shopping for Seniors and Physically Challenged Individuals through AR

Abstract: “Augmented reality (AR) technology has potential to revolutionize the way we interact with and visualize the world around us. One area where this technology has great potential is in the field of furniture design and home decor. The goal of the Augmented Reality Furniture project is to develop a platform which allows users to virtually preview and interact with furniture in their own home environments using AR technology. Using a smartphone or other AR-enabled device, users will be able to see how different furniture pieces will look and fit in their own living spaces before making a purchase. The project will involve the use of 3D models of a variety of furniture items, aswell as the development of AR software that can accurately place and scale these models in the user's environment. The platform will also include a user-friendly interface for browsing and selecting different furniture items, as well as tools for customizing and arranging the items in the virtual space.

The concept of an intelligent mobile application with elements of augmented reality and PLM software for value stream mapping

AbstractGrowing demands for variability, reducing losses and increasing production efficiency bring difficult tasks for business management. In order to cope with increasingly demanding tasks, a good knowledge of effective management methods is required, as well as orientation in a wide range of application programs. One of the approaches dealing with reducing losses and streamlining selected processes is the introduction of the Lean philosophy. This philosophy is firmly embedded in the concept of Industry 4.0 and its principles are proving to be highly effective. In the presented contribution, we will focus on the possibilities of reducing losses by implementing selected Lean principles in conjunction with selected program resources and elements of mixed reality. The goal is to create an intelligent mobile application that, based on a simulation model, would enable an in-depth analysis of selected processes from the aspect of material value stream mapping analysis. The model created in this way could serve as a relevant basis for top management in the search for solutions to optimize processes. In the article, we evaluate selected Lean methods, create a workplace simulation model using the PLM simulation tool Tecnomatix Plant Simulation, and the obtained data are presented in an intelligent mobile application based on augmented reality. After the online update, the created application conveys all required relevant information to managers and simplifies their decision-making process.

The development of an augmented reality application for exercise prescription within paediatric oncology: App design and protocol of a pilot study.

Background: Children and young people with cancer face barriers when engaging with exercise, such as treatment-related side effects, psychosocial burdens and lack of individualised provisions. Digital health tools, such as smartphone applications, have emerged as a promising driver to support healthcare provisions in exercise prescription among patients. It is vital to explore how such technologies can be developed more effectively in order to strengthen the evidence supporting their use and for more appropriate implementation within healthcare. This study aims to explore user experiences, preferences and suggested improvements from healthy children and young people aged 9-21 years. Methods: An augmented reality (AR) application was specifically developed for children and young people aged 9-21 years undergoing cancer treatment and a protocol for a pilot study was designed. The target sample of this pilot study is 90 healthy children and young people aged 9-21 years. Practical 30-min workshops will be conducted encouraging participants to engage with the smartphone app. Focus groups will explore participant experiences, preferences, and suggested improvements. Data will be analysed deductively with apriori themes derived from the semi-structured interviews. Discussion: Obtaining user experiences, preferences and suggested improvements is especially important for the development of novel apps, such as those prescribing exercise and using algorithms and augmented reality software. Results from this study will directly influence the development of an augmented reality application, which will also be applied within a long-term trial in paediatric oncology.

Augmented reality-based surgical navigation of pelvic screw placement: an ex-vivo experimental feasibility study

BackgroundMinimally invasive surgical treatment of pelvic trauma requires a significant level of surgical training and technical expertise. Novel imaging and navigation technologies have always driven surgical technique, and with head-mounted displays being commercially available nowadays, the assessment of such Augmented Reality (AR) devices in a specific surgical setting is appropriate.MethodsIn this ex-vivo feasibility study, an AR-based surgical navigation system was assessed in a specific clinical scenario with standard pelvic and acetabular screw pathways. The system has the following components: an optical-see-through Head Mounted Display, a specifically designed modular AR software, and surgical tool tracking using pose estimation with synthetic square markers.ResultsThe success rate for entry point navigation was 93.8%, the overall translational deviation of drill pathways was 3.99 ± 1.77 mm, and the overall rotational deviation of drill pathways was 4.3 ± 1.8°. There was no relevant theoretic screw perforation, as shown by 88.7% Grade 0–1 and 100% Grade 0–2 rating in our pelvic screw perforation score. Regarding screw length, 103 ± 8% of the planned pathway length could be realized successfully.ConclusionThe novel innovative system assessed in this experimental study provided proof-of-concept for the feasibility of percutaneous screw placement in the pelvis and, thus, could easily be adapted to a specific clinical scenario. The system showed comparable performance with other computer-aided solutions while providing specific advantages such as true 3D vision without intraoperative radiation; however, it needs further improvement and must still undergo regulatory body approval. Future endeavors include intraoperative registration and optimized tool tracking.

Analysis of the impact of energy consumption data visualization using augmented reality on energy consumption and indoor environment quality

This study proposes augmented reality (AR) as one of the methods to effectively reduce energy consumption in smart homes while maintaining an acceptable level of indoor environmental quality (IEQ). The experiment was conducted for a total of 90 min in an environment chamber where IEQ could be controlled, and participants were able to adjust the IEQ as desired throughout the duration of the experiment. Targeting a total of 21 participants, data on energy consumption, IEQ, and psychological response were collected under two conditions: (i) without providing energy consumption data through AR software, and (ii) with providing energy consumption data. The results show that AR software can positively impact energy consumption reduction. Also, reducing energy consumption through AR software did not significantly affect the psychological response to the subjects’ IEQ, suggesting that AR software can be used as an effective tool for reducing energy consumption without compromising occupant comfort concerning IEQ. In other words, when occupants are satisfied with indoor environments and are aware of their energy consumption, it means that energy consumption can be reduced for economic benefits. The results of this study suggest potential economic and social benefits from reducing building energy consumption.

Augmented reality-based software (Echo-QR) for guiding the echographic probe toward the acoustic window: a pilot study.

With current technology, ultrasound imaging in remote conditions, such as the International Space Station, is performed with vocal guidance or using a teleoperated echograph controlled by an expert. Both methods require real-time communications between the novice operator and expert to locate the probe over the appropriate acoustic windows (AW). The current study presents the development and testing of a new augmented reality software (Echo-QR) that would allow novice operators (with no medical imaging background) to correctly locate the ultrasound probe over the AW of interest without expert assistance. On the first day of the study, the positions of the probe over the AWs were identified for each organ by an expert sonographer and saved in the Echo-QR software. On the second day, the novices independently performed the ultrasound investigation using the Echo-QR software to correctly position the probe over each organ's AW. Using the Echo-QR software, novice operators found the AW in 73 (92%) of the 79 organs. The 2D images acquired by the novices "2D direct image" were acceptable for medical evaluation in 41% of the cases. However, when the "2D direct image" did not show the entire organ, a 3D capture of the volume below the probe was also performed, which allowed for the extraction of the appropriate 2D image "2D/3D image" for medical evaluation in 85% of the cases. Therefore, in the absence of real-time communication between an isolated participant and an expert sonographer, novel software (Echo-QR) and automated 3D volume capture can be used to obtain images usable for ultrasound diagnostics.

Impact of 3D imaging techniques and virtual patients on the accuracy of planning and surgical placement of dental implants: A systematic review.

The integration of advanced technologies, including three-dimensional (3D) imaging modalities and virtual simulations, has significantly influenced contemporary approaches to preoperative planning in implant dentistry. Through a meticulous analysis of relevant studies, this review synthesizes findings related to accuracy outcomes in implant placement facilitated by 3D imaging in virtual patients. A comprehensive literature search was conducted across relevant databases to identify relevant studies published to date. The inclusion criteria were studies utilizing 3D imaging techniques, virtual patients, and those focusing on the accuracy of dental implant planning and surgical placement. The selected studies were critically appraised for their methodological quality. After a rigorous analysis, 21 relevant articles were included out of 3021 articles. This study demonstrates the versatility and applicability of these technologies in both in vitro and in vivo settings. Integrating Computer-Aided Design/Computer-Aided Manufacturing (CAD/CAM), cone bean computed tomography (CBCT), and advanced 3D reconstruction methodologies showcases a trend toward enhanced precision in implant planning and placement. Notably, the evaluation parameters varied, encompassing distances, discrepancies, and deviations in the implant placement. The ongoing integration of systems such as dynamic navigation systems, augmented reality, and sophisticated software platforms shows a promising trajectory for the continued refinement of virtual reality applications in dental implantology, providing valuable insights for future research and clinical implementation. Moreover, using stereolithographic surgical guides, virtual planning with CBCT data, and 3D-printed templates consistently demonstrates enhanced precision in dental implant placement compared to traditional methods. The synthesis of the available evidence underscores the substantial positive impact of 3D imaging techniques and virtual patients on dental implant planning and surgical placement accuracy. Utilizing these technologies contributes to a more personalized and precise approach that enhances overall treatment outcomes. Future research directions and potential refinements to the application of these technologies in clinical practice should be discussed.

Targeted augmented reality-guided transperineal prostate biopsies study: initial experience.

Transperineal biopsy of magnetic resonance imaging (MRI)-detected prostate lesions is now the established technique used in prostate cancer (CaP) diagnostics. Virtual Surgery Intelligence (VSI) Holomedicine by Apoqlar (Hamburg, Germany) is a mixed reality (MR)/augmented reality (AR) software platform that runs on the HoloLens II system (Microsoft, Redford, USA). Multiparametric prostate MRI images were converted into 3D holograms and added into a MR space, enabling visualization of a 3D hologram and image-assisted prostate biopsy. The Targeted Augmented Reality-GuidEd Transperineal (TARGET) study investigated the feasibility of performing AR-guided prostate biopsies in a MR framework, using the VSI platform in patients with MRI-detected prostate lesions. Ten patients with a clinical suspicion of CaP on MRI (Prostate Imaging-Reporting and Data System, PI-RADS 4/5) were uploaded to the VSI HoloLens system. Two MR/AR-guided prostate biopsies were then acquired using the PrecisionPoint Freehand transperineal biopsy system. Cognitive fusion biopsies were performed as standard of care following the MR/AR-guided prostate biopsies. All 10 patients successfully underwent MR/AR-guided prostate biopsy after 3D MR images were overlaid on the patient's body. Prostatic tissue was obtained in all MR/AR-guided specimens. Seven patients (70%) had matching histology in both the standard and MR/AR-guided biopsies. The remaining three had ISUP (International Society of Urological Pathology) Grade 2 CaP. There were no immediate complications. We believe this is a world first. The initial feasibility data from the TARGET study demonstrated that an MR/AR-guided prostate biopsy utilizing the VSI Holomedicine system is a viable option in CaP diagnostics. The next stage in development is to combine AR images with real-time needle insertion and to provide further data to formally appraise the sensitivity and specificity of the technique.

Evaluation of a novel augmented reality educational tool and its effects on patient experience: A randomized controlled trial.

Patient education is an essential element of the treatment pathway. Augmented reality (AR), with disease simulations and three-dimensional visuals, offers a developing approach to patient education. We aim to determine whether this tool can increase patient understanding of their disease and post-visit satisfaction in comparison to current standard of care (SOC) educational practices in a randomized control study. Our single-site study consisted of 100 patients with initial diagnoses of kidney masses or stones randomly enrolled in the AR or SOC arm. In the AR arm, a physician used AR software on a tablet to educate the patient. SOC patients were educated through traditional discussion, imaging, and hand-drawn illustrations. Participants completed pre- and post-physician encounter surveys adapted from the Press Ganey® patient questionnaire to assess understanding and satisfaction. Their responses were evaluated in the Readability Studio® and analyzed to quantify rates of improvement in self-reported understanding and satisfaction scores. There was no significant difference in participant education level (P = 0.828) or visit length (27.6 vs. 25.0 min, P = 0.065) between cohorts. Our data indicate that the rate of change in pre- to post-visit self-reported understanding was similar in each arm (P ≥ 0.106 for all responses). The AR arm, however, had significantly higher patient satisfaction scores concerning the educational effectiveness and understanding of images used during the consultation (P < 0.05). While AR did not significantly increase self-reported patient understanding of their disease compared to SOC, this study suggests AR as a potential avenue to increase patient satisfaction with educational tools used during consultations.

An interactive augmented reality software for facial reconstructive surgeries

Background and ObjectiveSurgical trainees need a lot of training and practice before being able to operate independently. The current approach of surgical education mainly involves didactic teaching and psychomotor training through physical models or real tissue. Due to the unavailability of physical resources and lack of objective ways of evaluation, there is a demand for developing alternative training methods for surgeons. In this paper, we present an application that provides additional training opportunities to surgical trainees in the field of facial reconstructive surgeries. MethodsWe built a mobile augmented reality application that helps the user to visualize important concepts and experiment with different surgical plans for facial reconstructive surgeries. The application can overlay relaxed skin tension lines on a live video input or a patient's photo, which serve as bases for aligning a skin flap. A surgical trainee can interactively compare different skin flap design choices with estimated final scars on a photo of a patient. Data collection capability is also added to the application, and we performed a Monte Carlo experiment with simulated users (five classes of 100 users each) as an example of objectively measuring user performance. ResultsThe application can overlay relaxed skin tension lines on a patient's face in real time on a modern mobile device. Accurate overlays were achieved in over 91% as well as 84% and 88% out of 263 generated face images, depending on the method. Visual comparisons of the three overlay methods are presented on sample faces from different population groups. From the Monte Carlo experiment, we see that user actions in each class follow a normal distribution with a distinct set of parameters. ConclusionsThis application can serve as a basis for teaching surgical trainees the fundamentals of different facial reconstructive procedures, especially concepts related to relaxed skin tension lines and skin flaps. It can objectively evaluate the performance of surgical trainees in a course. This setup focuses on illustrating the relationship between the orientation of skin flaps and relaxed skin tension lines, which is a prerequisite of minimizing scarring in patients in addition to good surgical techniques.

Hybrid learning impact with augmented reality to improve higher order thinking skills of students

The purpose of this study was to determine the effectiveness of using hybrid learning with augmented reality to improve the high-level thinking skills of junior high school students in the area of geometry. In addition, the effects of hybrid learning with augmented reality on students' motivation to study geometry are also examined. This research technique uses a sequential mixed methods approach, combining quantitative and qualitative methods in sequence. Quantitative approaches were used in the first phase to collect quantifiable data, and qualitative methods were used in the second phase to explore the results of the first phase. A two-group pretest-posttest design is the quantitative research method used. In order to establish an experimental class and a control class, the research sample was selected using the cluster random sampling method. In addition, while the control class uses hybrid learning without augmented reality, the experimental class takes advantage of it. Interviews and observation sheets are used in the qualitative design dimensions. The quantitative part of the data shows that students who receive hybrid learning with augmented reality have higher order thinking skills in terms of ideas in geometry than students who receive hybrid learning without augmented reality. The HOTS N-Gain scores from hybrid learning with augmented reality are 0.62, while the HOTS N-Gain scores from hybrid learning without augmented reality are 0.43, both of which support this conclusion. In addition, according to the findings in the qualitative dimension, students are more motivated and engaged in learning when using augmented reality applications than when they are not. These qualitative findings suggest that augmented reality software can be successfully used as a teaching tool for geometry topics.

Effect of Augmented Reality Support on Quality Inspection of Welded Structures

With the increasing complexity of production systems and manufactured products, operators face high demands for professional expertise and long-term concentration. Augmented reality (AR) can support users in their work by displaying relevant virtual data in their field of view. In contrast to the extensive research on AR assistance in assembly, maintenance, and training, AR support in quality inspection has received less attention in the industrial context. Quality inspection is an essential part of industrial processes; thus, it is important to verify whether AR assistance can support users in these tasks. This work proposes an AR-based approach for quality inspection. For this, pilot AR software was designed and developed. The proposed AR approach was tested with end users. The task efficiency, the error rate, the perceived mental workload, and the usability of the AR approach were analysed and compared to the conventional paper-based support. The field research confirmed the positive effect on user efficiency during quality inspection while decreasing the perceived mental workload. This work extends the research on the potential of AR assistance in industrial applications and provides experimental validation to confirm the benefits of AR support on user performance during quality inspection tasks.

Augmented Reality-Assisted Surgical Exposure of an Impacted Tooth: A Pilot Study

Three-dimensional radiological evaluation through cone beam computer tomography is essential in diagnosing and establishing proper surgical management in impacted teeth. Through Augmented Reality (AR), clinicians have the opportunity to use three-dimensional computer-generated radiologic information to visualise the patient and simultaneously the superimposition of his internal structures. Here, we describe a digital workflow to assist the oral surgeon in pre-orthodontic exposure of a vestibular impacted canine using AR. The AR hardware consists of a camera and a traditional stand-up monitor. The registration and tracking are video-based and marker-free, with an automatic pose estimation obtained through VisLab 20.10.1AR software algorithm’s object recognition and tracking approach. A 3D model is created by combining the anterior teeth taken from the intraoral scan with the same teeth plus the included tooth taken from the CBCT segmentation. The 3D file is uploaded into the AR software. Model tracking is straightforward to set up without prior registration of targets or surroundings. The AR information is used successfully to define the surgical access to perform flap and osteotomy. The accuracy of model tracking matching was calculated constantly by the software. During the tracking, the process recorded an inlier ratio of 0.39:0.48. Further studies and clinical trials will evaluate the value of this novel technology in the management of impacted teeth.

Visualization of Volcanic Landform with Augmented Reality

Visualization of volcanic landforms with digital elevation model (DEM) data on the GIS desktop computer has limitations regarding access to the software display function and its use outside the room. This obstacle can be overcome by combining GIS and mobile augmented Reality. However, using digital elevation model data and satellite imagery in augmented reality software has not been widely explored. This study shows a practical way of integrating geospatial data with mobile augmented reality software. This research aims to create an augmented reality prototype to visualize virtual volcano objects. DEM data and satellite imagery create 3-dimensional (3D) objects and provide texture characteristics of volcanic landform objects. Visualization of volcanic landforms is done through an augmented reality (AR) application prototype on an Android smartphone. The experimental results show that augmented reality technology effortlessly combines DEM data and land features from satellite imagery. Evaluation of a smartphone device's manufacturing and installation stages shows results with a "successful" status. Through follow-up testing nine times, the AR prototype showed positive performance in all aspects of distance, angle, and lighting. The final test scenario in the form of responses regarding the application from other users obtained an average value of the "Good" category of 92.8%. In conclusion, these results indicate that the system built can be used in learning. However, further research requires more diverse test scenarios to see how far this application prototype can be used in various conditions. Keywords: Digital Elevation Model, GIS, Augmented Realit, 3D Visualization

Experience with augmented reality technology in the surgical treatment of a patient with encapsulated metal foreign bodies of lower extremities

ABSTRACT. This paper presents the experience of using augmented reality (AR) technology in the surgical treatment of a patient with encapsulated foreign bodies in the lower extremities. The study aims to test a previously developed algorithm for implementing AR technology in surgery and assess its effectiveness in treating patients with encapsulated foreign bodies. The study was conducted by a multidisciplinary team, comprising the Department of Naval Surgery of the S.M. Kirov Military Medical Academy and the Technical University named after Peter the Great. The medical and technical aspects involved an AR hardware and software package, including a personal computer and Microsoft Hololens II AR glasses. An invasive fixation system was also developed, incorporating a threaded pin used as an X-ray contrast mark with a seat for attaching special markers. The clinical part of the study involved observing and subsequently removing encapsulated foreign bodies in a patient who received inpatient treatment at the clinic of the Department of Naval Surgery in October 2022. Overall, AR technology demonstrated the potential for performing minimally invasive removal of encapsulated foreign bodies in limbs. The detailed visualization provided by AR allowed for determining optimal operative access, the volume of the upcoming operation, and the exact position, skeletotopy, and syntopia of the foreign body during the preoperative stage. Consequently, various intervention options could be digitally simulated. The use of AR technology facilitated intraoperative navigation, enhancing the safety and efficiency of the operation. Thus, AR technology has proven to be a minimally invasive, safe, and effective tool in surgical treatment. However, several unresolved issues require further research.

Design of a location-based augmented reality game for the development of key 21st century competences in primary education

The use of augmented reality games (ARGs) in education has gained increased attention from curriculum developers, teachers, and researchers in the past decade. Research findings show that ARGs can promote meaningful learning environments that foster key competences for the 21st century. This paper presents the design process of “EcoAegean”, an ARG for mobile devices, which was implemented in primary classroom environments to support the development of students’ key competences in the context of sustainability. The game was created using an open augmented reality software platform and its design was based on contemporary theoretical underpinnings regarding the use of such games in educational contexts. In the first section of the paper, we describe the design procedures of the learning scenario as well as the game itself. In the last section of the paper, we offer a set of critical insights on the design and implementation of mobile augmented reality games for the purpose of supporting students’ development of key 21st century competences.