Interactive Augmented Reality System Research Articles

A Fast and Interactive Augmented Reality System for PET/CT-guided Intervention of Neuroblastoma.

Neuroblastoma is the most common type of extracranial solid tumor in children and can often result in death if not treated. High-intensity focused ultrasound (HIFU) is a non-invasive technique for treating tissue that is deep within the body. It avoids the use of ionizing radiation, avoiding long-term side-effects of these treatments. The goal of this project was to develop the rendering component of an augmented reality (AR) system with potential applications for image-guided HIFU treatment of neuroblastoma. Our project focuses on taking 3D models of neuroblastoma lesions obtained from PET/CT and displaying them in our AR system in near real-time for use by physicians. We used volume ray casting with raster graphics as our preferred rendering method, as it allows for the real-time editing of our 3D radiologic data. Some unique features of our AR system include intuitive hand gestures and virtual user interfaces that allow the user to interact with the rendered data and process PET/CT images for optimal visualization. We implemented the feature to set a custom transfer function, set custom intensity cutoff points, and region-of-interest extraction via cutting planes. In the future, we hope to incorporate this work as part of a complete system for focused ultrasound treatment by adding ultrasound simulation, visualization, and deformable registration.

Interactive Augmented Reality System for Learning Phonetics Using Artificial Intelligence

Interactive Augmented Reality System for Learning Phonetics Using Artificial Intelligence

An Interactive Augmented and Virtual Reality System for Managing Dental Anxiety among Young Patients: A Pilot Study

Dental anxiety is a common health problem among children. It creates major issues for patients, parents, and dental professionals. Children who cancel or otherwise miss their dental appointments generally do so due to fear of the unknown and lack of understanding of what they can expect from the environment and treatment when they arrive there. Some distraction interventions are already used by dental professionals, such as using clown doctors, watching cartoons, and utilizing the tell–show–do (TSD) technique. Still, the problem is common, and the fail to attend (FTA) rates at clinics are high. Familiarizing children with the dental setting and procedures in advance may help to manage their anxiety. This paper aims to help in managing children’s dental anxiety in a simple, attractive, and age-appropriate way through the use of augmented reality (AR) and virtual reality (VR) technologies. The developed system is named “Dr. Barea”. It targets Arabic-speaking children aged from 7 to 10 years old. It uses model–view–control (MVC) as its architectural design pattern. The proposed solution consists of three main sections: a 360° VR video that simulates a dental clinic environment, an educational description on dental tools using AR technology, and interactive educational stories that educate children about dental hygiene. The system performance was evaluated using unit, integration, performance, and user acceptance testing. The results demonstrate that the proposed solution, which performed reasonably, achieved the usability requirements and was engaging for learning information about dental hygiene. A feasibility study with 16 children was conducted to evaluate the effectiveness of the proposed system. The Child Fear Survey Schedule—Dental Subscale (CFSS-DS) was used to measure children’s dental anxiety level. The T test was used to evaluate the differences between groups, and Fisher’s exact test was used to compare the distributions of gender and age between the groups. The CFSS-DS index in the VR group decreased after dental consultation (35.04 ± 9.14 before consultation and 32.32 ± 8.32 after consultation, p = 0.041). The implications of this study shall be beneficial to patients, parents, and dental professionals.

Affordable Projector-Based AR Experimental Platform with Fitting Simulation Asset for Exploring Thermal Management

Augmented reality (AR) applied in education provides learners a possible way for better understanding and thorough learning. Although the traditional projector is used to integrate the augmented information with real objects without wearing AR glasses, the projector-based AR system is unlikely to be adopted widely in education due to the cost, heavy weight, and space issues. In this paper, an alternative projector-camera AR platform, utilizing a digital light processing (DLP) module matched with a Beaglebone Black (BB) controller, is proposed for AR physical experiments. After describing the DLP-based AR learning design method, the algorithm of pre-deforming projection content with simulation-based poly fitting is presented to keep the virtual asset consistent with the user action; and then a prototype with the content regarding the thermal management of power devices is illustrated to validate the performance of the AR experimental platform. The result shows that the DLP-based AR platform is an accurate and interactive AR system with a response time of 1 s, and a registration deviation of 3 mm. It is also an affordable AR learning design tool with a bill of materials of about $200, and thus casts light on creating AR-based physical experiments to explore more physical phenomena.

Designing and evaluating a high interactive augmented reality system for programming learning

Programming plays a pivotal role in Science, Technology, Engineering, and Mathematics (STEM) education. However, programming concepts and syntax are extremely abstract. We developed an augmented reality (AR)-based learning system to make programming concepts tangible. Specifically, we adopted a high interactive AR mode that allows students to freely assemble puzzle cards to compare various 3D animated programming results. This study adopted a quasi-experimental design and recruited 98 fifth graders as participants. Three groups, named the high interactive AR, low interactive AR, and the traditional learning (control) groups, were formed. The results revealed that the AR-based instruction, especially the high interactive AR mode, helped the students improve their programming achievements, gain higher learning motivation, reduce their cognitive load, and acquire a high degree of technology acceptance. The interview data mainly showed that the students in the high interactive group attained more positive experiences of assembling puzzle cards to learn programming. Our study suggests that using AR technology alone did not facilitate learning motivation, although it did produce better programming achievements compared to the traditional learning. A high interactive design such as “puzzle cards” is vital for arousing higher motivation, which can in turn facilitate a more in-depth understanding of programming logic.

Interaction With Gaze, Gesture, and Speech in a Flexibly Configurable Augmented Reality System

Multimodal interaction has become a recent research focus since it offers better user experience in augmented reality (AR) systems. However, most existing works only combine two modalities at a time, e.g., gesture and speech. Multimodal interactive system integrating gaze cue has rarely been investigated. In this article, we propose a multimodal interactive system that integrates gaze, gesture, and speech in a flexibly configurable AR system. Our lightweight head-mounted device supports accurate gaze tracking, hand gesture recognition, and speech recognition simultaneously. The system can be easily configured into various modality combinations, which enables us to investigate the effects of different interaction techniques. We evaluate the efficiency of these modalities using two tasks: the lamp brightness adjustment task and the cube manipulation task. We also collect subjective feedback when using such systems. The experimental results demonstrate that the <i>Gaze+Gesture+Speech</i> modality is superior in terms of efficiency, and the <i>Gesture+Speech</i> modality is more preferred by users. Our system opens the pathway toward a multimodal interactive AR system that enables flexible configuration.

Effectiveness of Interactive Augmented Reality and Electrical Neuromodulation System in Persons With Stroke

Effectiveness of Interactive Augmented Reality and Electrical Neuromodulation System in Persons With Stroke

Three-Dimensional Augmented Reality System for Balance and Mobility Rehabilitation in the Elderly: A Randomized Controlled Trial.

We attempted to evaluate the clinical efficiency of a novel three-dimensional interactive augmented reality system (3D-ARS) for balance and mobility rehabilitation. This system enables participant training with a realistic 3D interactive balance exercise and assessing movement parameters and joint angles by using a kinetic sensor system. We performed a randomized controlled trial in a general hospital. Thirty-six participants (age, 56-76 years) who could independently walk and stand on one leg were recruited. The participants were randomly assigned to either group. The control group (n = 18) underwent a conventional physical fitness program such as lower-extremity strengthening and balance training thrice per week for 1 month. The experimental group (n = 18) experienced 3D-ARS training thrice per week (1 session = 30 minutes) for 4 weeks. Training comprised a balloon game for hip exercise, cave game for knee exercise, and rhythm game for one-leg balance exercise. Lower-extremity clinical scale scores, fall index, and automatic balance score were measured by using Tetrax® posturography before, during, and after training. Significant group (3D-ARS vs. control) × time (before and after exercise) interaction effect was observed for Berg balance scale (BBS) scores (p = 0.04) and timed-up-and-go (TUG; p < 0.001). Overall improvements occurred in stability index, weight distribution index, fall risk index, and Fourier transformations index of posturography for both groups. However, score changes were significantly greater in the 3D-ARS group. Significant group × time interaction effect was observed for the fall risk index. This demonstrates that the 3D-ARS system can improve balance in the elderly more effectively.

Augmented reality supported mobile self-guided system for enhancing authentic learning activities

Traditional teaching models can often only provide a one-way transmission of knowledge in real-world situations. These methods are rarely effective, as students require much more interaction with the instructor to gain and retain knowledge from the curriculum. For these reasons, this study supports the use of an interactive augmented reality (AR) system that combines AR and QR code technologies for teaching purposes. The AR system includes two subsystems: the mobile AR system and the AR materials remote server. Users can easily build their own learning environments and include information and materials relevant to their needs. We hope that this study will promote the use of AR systems for educational purposes and provide students with virtual content linked to real-world objects to create an interactive method of learning new information. Our final goal is to encourage the widespread use of AR technology.

Augmented reality supported mobile self-guided system for enhancing authentic learning activities

Traditional teaching models can often only provide a one-way transmission of knowledge in real-world situations. These methods are rarely effective, as students require much more interaction with the instructor to gain and retain knowledge from the curriculum. For these reasons, this study supports the use of an interactive augmented reality (AR) system that combines AR and QR code technologies for teaching purposes. The AR system includes two subsystems: the mobile AR system and the AR materials remote server. Users can easily build their own learning environments and include information and materials relevant to their needs. We hope that this study will promote the use of AR systems for educational purposes and provide students with virtual content linked to real-world objects to create an interactive method of learning new information. Our final goal is to encourage the widespread use of AR technology.

An Human-Computer Interactive Augmented Reality System for Coronary Artery Diagnosis Planning and Training.

In order to let the doctor carry on the coronary artery diagnosis and preoperative planning in a more intuitive and more natural way, and to improve the training effect for interns, an augmented reality system for coronary artery diagnosis planning and training (ARS-CADPT) is designed and realized in this paper. At first, a 3D reconstruction algorithm based on computed tomographic (CT) images is proposed to model the coronary artery vessels (CAV). Secondly, the algorithms of static gesture recognition and dynamic gesture spotting and recognition are presented to realize the real-time and friendly human-computer interaction (HCI), which is the characteristic of ARS-CADPT. Thirdly, a Sort-First parallel rendering and splicing display subsystem is developed, which greatly expands the capacity of student users. The experimental results show that, with the use of ARS-CADPT, the reconstruction accuracy of CAV model is high, the HCI is natural and fluent, and the visual effect is good. In a word, the system fully meets the application requirement.

The constrained SLAM framework for non-instrumented augmented reality

This paper addresses the challenging issue of marker less tracking for Augmented Reality. It proposes a real-time camera localization in a partially known environment, i.e. for which a geometric 3D model of one static object in the scene is available. We propose to take benefit from this geometric model to improve the localization of keyframe-based SLAM by constraining the local bundle adjustment process with this additional information. We demonstrate the advantages of this solution, called contrained SLAM, on both synthetic and real data and present very convincing augmentation of 3D objects in real-time. Using this tracker, we also propose an interactive augmented reality system for training application. This system, based on a Optical See-Through Head Mounted Display, allows to augment the users vision field with virtual information accurately co-registered with the real world. To keep greatly benefit of the potential of this hand free device, the system combines the tracker module with a simple user-interaction vision-based module to provide overlaid information in response to user requests.

Interactive Medical Augmented Reality System for Remote Surgical Assistance

Medical surgery is considered as a difficult technique and ne eds to be performed by well-trained and experienced medical personnel to complete the procedure. During a complicated surgery, an experienced surgeon at distant can provide sophisticated instruction or guidance to help the operating surgeon when it is necessary. In this study, we have designed an interactive augmented reality system for remote surgical assistance. The surgeon at the remote site can indicate or contour a region of interes t (ROI) by manually drawing on streaming images which were captured by the camera at the local site. The coordinate of the ROI in the 3-D physical space can be estimated by using the CT images of the patient. Therefore, the ROI and the anatomical information of the CT images can be rendered on the captured images, overlaid on the patient and projected onto a head-mounted display of the local operating surgeon, providing an across-space augmented reality visualization.

Utility of a Three-Dimensional Interactive Augmented Reality Program for Balance and Mobility Rehabilitation in the Elderly: A Feasibility Study.

ObjectiveTo improve lower extremity function and balance in elderly persons, we developed a novel, three-dimensional interactive augmented reality system (3D ARS). In this feasibility study, we assessed clinical and kinematic improvements, user participation, and the side effects of our system.MethodsEighteen participants (age, 56-76 years) capable of walking independently and standing on one leg were recruited. The participants received 3D ARS training during 10 sessions (30-minute duration each) for 4 weeks. Berg Balance Scale (BBS) and the Timed Up and Go (TUG) scores were obtained before and after the exercises. Outcome performance variables, including response time and success rate, and kinematic variables, such as hip and knee joint angle, were evaluated after each session.ResultsParticipants exhibited significant clinical improvements in lower extremity balance and mobility following the intervention, as shown by improved BBS and TUG scores (p<0.001). Consistent kinematic improvements in the maximum joint angles of the hip and knee were observed across sessions. Outcome performance variables, such as success rate and response time, improved gradually across sessions, for each exercise. The level of participant interest also increased across sessions (p<0.001). All participants completed the program without experiencing any adverse effects.ConclusionSubstantial clinical and kinematic improvements were observed after applying a novel 3D ARS training program, suggesting that this system can enhance lower extremity function and facilitate assessments of lower extremity kinematic capacity.

Developing an interactive augmented reality system as a complement to plant education and comparing its effectiveness with video learning

The learning of plants has garnered considerable attention in recent years, but students often lack the motivation to learn about the process of plant growth. Also, students are not able to apply what they have learned in class in the form of observation, since plant growth takes a long time. In this study, we use augmented reality (AR) technology to develop the ARFlora system, which can assist students in observing the changes in plant growth while in the classroom. More specifically, students are able to use AR markers to manipulate various virtual objects (e.g. sunlight) and observe the changes they have on plant growth. Meanwhile, a quasi-experimental evaluation is in place to substantiate the effectiveness of ARFlora in the learning of plants and to compare it with digital video learning. In the quasi-experimental design, 55 elementary-school students participated in the study. The participants are divided into two groups, an “experimental group” and a “control group.” The experimental group was taught using the ARFlora system, while the control group was taught by employing the digital video. Results show that (1) ARFlora and digital video have the same effectiveness on student's learning outcomes; (2) ARFlora is more effective in helping students retain learned knowledge; and (3) ARFlora is comparatively more useful in motivating students to learn about plants.

Augmented Reality System for the Visualization and Interaction with 3D Digital Models in a Wide Environment

ABSTRACTThis paper proposes a new interactive Augmented Reality (AR) system, which has been conceived to allow a user to freely interact with virtual objects integrated in a real environment without the need to wear cumbersome equipment. The AR system has been developed by integrating the Fog Screen display technology, stereoscopic visualization and the Microsoft Kinect. The user can select and manage the position of virtual objects visualized on the Fog Screen display by using directly his/her hands. A specific software application has been developed to perform evaluation testing sessions with users. The aim of the testing sessions was to verify the influence of issues related to tracking, visualization and interaction modalities on the overall usability of the AR ‘system. The collected experimental results demonstrate the easiness of use and the effectiveness of the new interactive AR system and highlight the features preferred by the users.

Interactive augmented reality system for enhancing library instruction in elementary schools

Due to limited budgets and manpower, most elementary schools in Taiwan do not plan or provide library instruction for students. Although students can use libraries, they typically lack the knowledge needed to use library resources effectively. Consequently, students have difficulty finding the books they need and can easily become overwhelmed by the massive amount of information in libraries. Computer-assisted instruction for teaching basic library skills to large numbers of students is an appealing method. Particularly, developing augmented reality (AR) technologies for learning have garnered considerable attention in education research. Many researchers and scholars believe that integrating teaching and AR enhances student learning performance and motivation. This work develops an educational AR system based on situated learning theory, and applies innovative augmented reality interactive technology to a library’s learning environment. Student library knowledge can be enhanced via the proposed augmented reality library instruction system (ARLIS). Experimental results demonstrate that student learning performance is improved significantly by using the proposed ARLIS. Moreover, this work demonstrates that using the proposed ARLIS for library instruction results in the same learning performance as conventional librarian instruction and there is no gender difference on learning performance between the proposed ARLIS and conventional librarian instruction. Moreover, the proposed library instruction system overcomes shortcomings of personal teaching skills of librarians that may adversely affect student learning performance by conveying the same learning content to all students. Additionally, the proposed system results in better learning performance for learners with the field-dependent cognitive style than learners with the field-independent cognitive style. Further, the proposed system provides more benefits in terms of library skills of application and comprehension than conventional librarian instruction. Moreover, the learning performance of students is not affected by their gaming skills. Therefore, student gaming skills do not need to be considered when adopting the proposed system in library instruction programs.

Development of Online Computer Supported Collaborative Work with Augmented Reality for Machine Assembly Training

This paper proposes an interactive augmented reality (AR) architecture for online computer supported collaborative work (CSCW). Current CSCW applications for AR are focused on multiple users met together to collaborate on the same physical environment. The proposed architecture can provide collaboration through computer network without the need of face-to-face communication. System architecture and operation scenario are firstly constructed. AR applications for machine assembly trainingis studied. The proposed interactive AR system architecture can provide new experiences for users and both the effectiveness and efficiency for online training applications can be significantly improved.