Augmented Reality Glasses Research Articles

User Satisfaction in Augmented Reality-Based Training Using Microsoft HoloLens

With the recent developments in augmented reality (AR) technologies comes an increased interest in the use of smart glasses for hands-on training. Whether this interest is turned into market success depends at the least on whether the interaction with smart AR glasses satisfies users, an aspect of AR use that so far has received little attention. With this contribution, we seek to change this. The objective of the article, therefore, is to investigate user satisfaction in AR applied to three cases of practical use. User satisfaction of AR can be broken down into satisfaction with the interaction and satisfaction with the delivery device. A total of 142 participants from three different industrial sectors contributed to this study, namely, aeronautics, medicine, and astronautics. In our analysis, we investigated the influence of different factors, such as age, gender, level of education, level of Internet knowledge, and the roles of the participants in the different sectors. Even though users were not familiar with the smart glasses, results show that general computer knowledge has a positive effect on user satisfaction. Further analysis using two-factor interactions showed that there is no significant interaction between the different factors and user satisfaction. The results of the study affirm that the questionnaires developed for user satisfaction of smart glasses and the AR application performed well, but leave room for improvement.

Didactic Concept for Increasing Acceptance of Consistent Data Standards Using the Example of Assistance Systems in Assembly

Abstract In times of increasing variant diversity, there are ever higher demands on production. Particularly in assembly, activities can be supported by assistance systems such as lightweight robots or augmented reality glasses. However, a wide range of assembly variants requires the automatic configuration of the assistance systems since manual configuration is not economical. Nonetheless, in order to achieve this automatic configuration, data consistency must be ensured within the company so that data can be exchanged both horizontally and vertically. As especially small and medium-sized enterprises have no experience with consistent data standards, this paper presents a concept to convey these contents with the help of a learning factory.

Reliability Testing of Augmented Reality Glasses Technology: Establishing the Evidence Base for Telewound Care.

To test the fidelity and feasibility of using augmented reality glass (ARG) telehealth technology for comprehensive wound, ostomy, and continence (WOC) service delivery to underserved rural practice settings. Intrarater reliability design focusing on treatment integrity. Adult patients in a tertiary care hospital and a rural long-term care setting who were scheduled for routine wound care consults. Intra- and interrater reliability were assessed using a documentation-based wound assessment tool comprising 6 discrete assessment points. The wound teleassessment was first conducted by the telehealth "hub" WOC nurse remotely in collaboration with a bedside nurse wearing the ARG. The same hub WOC nurse then conducted an assessment at the bedside. Initial assessment points and treatment plans were compared to establish assessment and treatment intrarater reliability. A different WOC nurse, blinded to the first assessment, also conducted a bedside assessment and provided treatment recommendations, which were then compared to the hub WOC nurse's initial remotely ARG-generated treatment plan to establish treatment interrater reliability. Sixteen patients with 21 wounds were assessed. Six wound assessment components were included, yielding a total of 126 total observation points. Intrarater assessment reliability was 98%. Treatment plan interrater reliability was 100%. Results support further evaluation of ARG technology as a tool to enhance the delivery of wound care services in remote underserved settings. Implementation and evaluation of this technology on clinical and financial outcomes in multiple wound care delivery environments should be determined moving forward. Successful implementation should serve as a template to expand evidence-based WOC nursing care across the globe.

A Novel Natural Mobile Human-Machine Interaction Method With Augmented Reality

This paper proposes a novel teleoperation method that allows users to guide robots hand by hand along with speech. In this method, the virtual robot modeled according to the remote real robot is projected into the real local environment to form a 3D operation interface. In this case, users can directly interact with virtual objects by their hands. Furthermore, since the Leap Motion is attached to the augmented reality (AR) glasses, the operation space is greatly extended. Therefore, users can observe the virtual robot from an arbitrary angle without blind angle in such a mobile pattern, which enhances the users' interactive immersion and provides more natural human-machine interaction. To improve the accuracy of the measurement, an unscented Kalman filter (UKF) and an improved particle filter (IPF) are used to estimate the position and orientation of the hand, respectively. Furthermore, Term Frequency-Inverse Document Frequency (TF-IDF) and maximum entropy model are adopted to recognize the speech and gestures instructions of the user. The proposed method is compared with the three human-machine methods on various experiments. The results verified the effectiveness of the proposed method.

An Application of 3D Model Reconstruction and Augmented Reality for Real-Time Monitoring of Additive Manufacturing

This paper presents a novel scan-based method for the real-time monitoring of additive manufacturing processes. Most traditional scanning techniques used for generating 3D models capture the only outer shape of the object after completion of the printing process. The method proposed in this paper differs as it relies on a layer-by-layer scanning of the 3D object directly during the printing process. This strategy has been successfully implemented with a fused filament 3D printer (PRUSA i3 MK3). Furthermore, in order to offer an increased interaction between the obtained 3D model and the user, a virtual environment has been developed for the augmented reality glasses HoloLens. The novelty of this method lies in the layer-by-layer 3D model reconstruction of both the outer shape and the inner layers of the printed part. It enables the user, directly during the printing process, to view and detect potential defects, not only at the surface but also in the inner layers of the printed object. Therefore, it can provide detailed information about the build quality and can be used as the basis of a decision-making tool.

Human intention estimation based on hidden Markov model motion validation for safe flexible robotized warehouses

With the substantial growth of logistics businesses the need for larger warehouses and their automation arises, thus using robots as assistants to human workers is becoming a priority. In order to operate efficiently and safely, robot assistants or the supervising system should recognize human intentions in real-time. Theory of mind (ToM) is an intuitive human conception of other humans' mental state, i.e., beliefs and desires, and how they cause behavior. In this paper we propose a ToM based human intention estimation algorithm for flexible robotized warehouses. We observe human's, i.e., worker's motion and validate it with respect to the goal locations using generalized Voronoi diagram based path planning. These observations are then processed by the proposed hidden Markov model framework which estimates worker intentions in an online manner, capable of handling changing environments. To test the proposed intention estimation we ran experiments in a real-world laboratory warehouse with a worker wearing Microsoft Hololens augmented reality glasses. Furthermore, in order to demonstrate the scalability of the approach to larger warehouses, we propose to use virtual reality digital warehouse twins in order to realistically simulate worker behavior. We conducted intention estimation experiments in the larger warehouse digital twin with up to 24 running robots. We demonstrate that the proposed framework estimates warehouse worker intentions precisely and in the end we discuss the experimental results.

Evaluation of a Visual–Tactile Multimodal Display for Surface Obstacle Avoidance During Walking

The future of human space exploration will involve extra-vehicular activities (EVAs) on foreign planetary surfaces (i.e., Mars), an activity that will have significantly different characteristics than exploration scenarios on Earth. These activities become challenging due to restricted vision and limitations placed on sensory feedback from altered gravity and the space suit. The use of a bulky, pressurized EVA suit perceptually disconnects human explorers from the hostile environment, increasing navigation workload and the risk of collision associated with traversing through unfamiliar terrain. Multimodal displays are considered as a means to provide information to the astronaut as cues to multiple sensory modalities enhance cognitive processing through taking advantage of multiple sensory resources, and are believed to communicate the risk more efficiently than unimodal cues. This paper presents a wearable multimodal interface system to examine human performance when visual, vibratory, and visual–vibratory cues are provided to aid in ground obstacle avoidance. The wearable system applies vibrotactile cues to the feet and visual cues through augmented reality glasses to convey obstacle location and proximity. Participants ( n = 16) performed an overground obstacle avoidance task with the multimodal device. Performance metrics included completion time, subjective workload, head-down time, collisions, as well as gait parameters. Results indicate that information displays enhance task performance, with the visual-only display promoting the least head-down time over tactile-only or visual–tactile displays. Head-down time was the highest for trials without a display. Results provide implications for presenting information during physically active tasks such as suited obstacle avoidance.

Development of skills and tools for micro opto-electrical integration on wafer level

Abstract New devices for a digital lifestyle like augmented reality (AR) glasses may find their wide spread use only if consumer relevant pricing can be achieved. Low cost RGB-light sources are a key element for AR. assembly productivity for these micro opto-electrical RGB-light sources and a further reduction in their package dimensions. We present our new approach to build miniature RGB-light sources on structured silicon wafers and elaborate the required skill set and our future process infrastructure based on our existing high volume capable optical packaging platform. The platform provides an optical bench with integrated heat spreader and emission windows enabling miniature hermetic housing for laser diodes assemblies on 8″ wafers.

Seamless human robot collaborative assembly – An automotive case study

This paper presents the implementation of a robotic system for advanced human robot collaboration assembly and discusses all the technological approach that has been implemented for facilitating the interaction and support of human operators. Unlike current industrial practice where the assembly is performed by operators, the proposed approach aims at combining the benefits of high payload industrial robots with human capabilities under a fenceless environment, by assigning to them each task based on their capabilities. Enabling technologies involve manual guidance techniques and new wearables devices allowing for multi – modal interaction as well as robot safety control functionalities. Wearable devices such as Augmented Reality glasses and smartwatches are used for closing the communication loop between operators and robots under a service-oriented architecture. The complete system is validated in a case study from the automotive industry under the ROBO-PARTNER project. A detailed safety analysis of the scenario has been performed supported by a Risk Assessment, safety concept and Safety Related Parts/Control Systems (SRC/CS) design. The findings support the concept that humans’ and robots’ destiny is collaboration rather than competition.

New Way of Monitoring of the Production Environment with Application of Augmented Reality and Artificial Intelligence

Abstract Augmented reality technology (AR), also known as mixed reality, allows to enhance the seen reality with additional, virtual objects, displayed right in front of the user’s eyes. The article presents the concept of using Augmented Reality glasses, to support the user with pre-defined knowledge and instructions, giving the ability to do the job without almost any idea of what should be done. The operator simply follows the instructions shown in front of the eyes and communicate with the system with simple gesture and voice. Additionally, the article touches the topic of Artificial Intelligence (AI) which will allow the system in the future to have not only the pre-defined, hard coded instructions, but also the flexibility to give different hints to the user based on a different situation observed.

3D separable convolutional neural network for dynamic hand gesture recognition

Abstract Dynamic hand gesture recognition, as an essential part of Human–Computer Interaction, and especially an important way to realize Augmented Reality, has been attracting attention from many scholars and yet presenting many more challenges. Recently, being aware of deep convolutional neural network's excellent performance, many scholars began to apply it to gesture recognition, and obtained promising results. However, no enough attention has been paid to the number of parameters in the network and the amount of computer calculation needed until now. In this paper, a 3D separable convolutional neural network is proposed for dynamic gesture recognition. This study aims to make the model less complex without compromising its high recognition accuracy, such that it can be deployed to augmented reality glasses more easily in the future. By the application of skip connection and layer-wise learning rate, the undesired gradient dispersion due to the separation operation is solved and the performance of the network is improved. The fusion of feature information is further promoted by shuffle operation. In addition, a dynamic hand gesture library is built through HoloLens, which thus proves the feasibility of the proposed method.

The Helping Hand: An Assistive Manipulation Framework Using Augmented Reality and Tongue-Drive Interfaces.

A human-in-the-loop system is proposed to enable collaborative manipulation tasks for person with physical disabilities. Studies show that the cognitive burden of subject reduces with increased autonomy of assistive system. Our framework obtains high-level intent from the user to specify manipulation tasks. The system processes sensor input to interpret the user's environment. Augmented reality glasses provide ego-centric visual feedback of the interpretation and summarize robot affordances on a menu. A tongue drive system serves as the input modality for triggering a robotic arm to execute the tasks. Assistance experiments compare the system to Cartesian control and to state-of-the-art approaches. Our system achieves competitive results with faster completion time by simplifying manipulation tasks.

Could Prolonged Usage of GPS Navigation Implemented in Augmented Reality Smart Glasses Affect Hippocampal Functional Connectivity?

Background Augmented reality (AR) glasses with GPS navigation represent the rapidly evolving technology which spares (and externalizes) navigational capacities. Regarding the expected everyday usage of this device, its impact on neuroplastic brain changes and navigation abilities should be evaluated. Aims This study aimed to assess possible changes in functional connectivity (FC) of hippocampus and other brain regions involved in spatial navigation. Methods Thirty-three healthy participants completed two resting state functional magnetic resonance imaging (rsfMRI) measurements at the baseline and after 3 months. For this period, the experimental group (n = 17) has had used AR device (Vuzix M100) with incorporated GPS guidance system during navigation in real world. Participants from the control group (n = 16) have not used any GPS device while navigating during walking. The rsfMRI FC of right and left hippocampi was analyzed using a seed-driven approach. Virtual city task was used to test navigational abilities both before and after the usage of AR device. Results We identified strong functional coupling of right and left hippocampi at the baseline (p < 0.05, FDR corrected). Mild changes in bilateral hippocampal FC (p < 0.05, FDR uncorrected) were observed in both assessed groups mainly between the bilateral hippocampi and between each hippocampus and temporal regions and cerebellum. However, the experimental group showed FC decrease after three months of using GPS navigation implemented in AR glasses in contrast to FC increase in the control group without such intervention. Importantly, no effect of intervention on navigational abilities was observed. Discussion Our observation supports the assumption that externalization of spatial navigation to technological device (GPS in AR glasses) can decrease the functional coupling between hippocampus and associated brain regions. Considering some limitations of the present study, further studies should elucidate the mechanism of the observed changes and their impact on cognitive abilities.

The use of augmented reality glasses in central line simulation: "see one, simulate many, do one competently, and teach everyone".

ObjectiveThe aim of this study was to investigate the feasibility of using augmented reality (AR) glasses in central line simulation by novice operators and compare its efficacy to standard central line simulation/teaching.DesignThis was a prospective randomized controlled study enrolling 32 novice operators. Subjects were randomized on a 1:1 basis to either simulation using the augmented virtual reality glasses or simulation using conventional instruction.SettingThe study was conducted in tertiary-care urban teaching hospital.SubjectsA total of 32 adult novice central line operators with no visual or auditory impairments were enrolled. Medical doctors, respiratory therapists, and sleep technicians were recruited from the medical field.Measurements and main resultsThe mean time for AR placement in the AR group was 71±43 s, and the time to internal jugular (IJ) cannulation was 316±112 s. There was no significant difference in median (minimum, maximum) time (seconds) to IJ cannulation for those who were in the AR group and those who were not (339 [130, 550] vs 287 [35, 475], p=0.09), respectively. There was also no significant difference between the two groups in median total procedure time (524 [329, 792] vs 469 [198, 781], p=0.29), respectively. There was a significant difference in the adherence level between the two groups favoring the AR group (p=0.003).ConclusionAR simulation of central venous catheters in manikins is feasible and efficacious in novice operators as an educational tool. Future studies are recommended in this area as it is a promising area of medical education.

P‐95: Study of AR Optical Design with Lightweight Optical Waveguide

Augmented Reality (AR) is a technique that uses computer generated additional information to augment or expand the real world view, and the optical device is one of the most core components. This paper introduces our research about AR optical system that applied in lightweight waveguide with low thickness and light weight structure, and discussed the stray light phenomenon, as well as our AR glasses can be implemented in only one FPGA chip with distortion correction.

17‐3: Distinguished Paper: A Plastic Holographic Waveguide Combiner for Light‐weight and Highly‐transparent Augmented Reality Glasses

We have developed a unique production process of a full‐color plastic holographic waveguide combiner with a light‐weight and see‐through capability. The novel plastic waveguide technology enables us to increase design flexibility in the eyewear and to expand the market for augmented reality (AR). This paper presents the approach to production.

79‐4: A Plastic Electrochromic Dimming Device for Augmented Reality Glasses

We have developed an electrochromic dimming device on a plastic substrate with high transparency modulation from 70% to 10% and bending radius below 30 mm. It works more than endurance 10,000 cycles and high‐temperature‐humidity conditions. Combination of the device and AR glass enables the clear image visibility in various environments.

A plastic holographic waveguide combiner for light‐weight and highly‐transparent augmented reality glasses

AbstractThere is a high demand for light‐weight, stylishly designed augmented reality (AR) glasses with natural see‐through capabilities for the wide‐spread distribution of novel wearable device to general consumers. We have successfully developed a unique production process of a holographic waveguide combiner that enables us to laminate holographic optical elements (HOEs) onto a plastic substrate with optical grade quality. The plastic substrate waveguide combiner has a number of advantages over conventional glass substrate combiners; the plastic substrate makes AR glasses lighter in weight and unbreakable. With the lamination process of HOEs, we can apply them to a various designs to satisfy general customers' wide range of preferences for the style. We also potentially made it possible for the holographic waveguide combiner to be produced in larger volumes at lower costs by using our novel roll‐to‐roll hologram recording and laminating process. In this paper, we present our approach of the plastic substrate HOE production process for AR glasses.

Using Augmented Reality Glasses in Central Venous Line Placement Simulations

Central venous lines are used to administer medicines, nutrients, and blood products to the heart via the large internal jugular vein. This technology is used when medications cannot be administered orally or injected through smaller veins. The procedure involves a high level of precision and there is little room for error. The goal of this research is to investigate the feasibility of using augmented reality (AR) glasses in central line simulation by novice operators and comparing its efficacy to standard central line simulation/teaching. With the Brother AiRScouter augmented reality glasses, the learner controls the timing of each step and receives real‐time information from the AR glasses. A grader documented the time to AR glasses placement, time to venipuncture, number of attempts, and completion of a pre‐determined checklist. This study used a prospective randomized enrollment of 32 novice operators. Subjects were randomized on a 1:1 basis to either a simulation using the augmented reality glasses or a simulation using only conventional instruction. There was a total of 32 subjects participating in this study. Mean age of participants was 39.8±7.8 years; 5 Females and 27 Males; 11 Physicians, 19 Respiratory Therapists, and 2 Sleep Technologists. The mean time for AR placement in the AR group was 71 ± 43 secs and time to IJ cannulation was 316 ± 112 sec. There was no significant difference in median (min, max) time (seconds) to IJ cannulation for those who were in the AR group and those who were not (339 (130, 550) vs 287 (35, 475), p=0.09), respectively. There was also no significant median difference between the two groups in total procedure time (524 (329, 792 vs 469 (198, 781), p=0.29), respectively. Our results are consistent with the idea that AR‐simulation of central venous catheters is feasible and efficacious for novice operators. AR technology is likely to be generally applicable to both procedural and fundamental medical education.Support or Funding InformationBrother Augmented Reality glasses were provided by Brother‐USA. Tom &amp; Vi Zapara School of Business at La Sierra University supported this study.This abstract is from the Experimental Biology 2018 Meeting. There is no full text article associated with this abstract published in The FASEB Journal.

Calling while Driving Using Augmented Reality: Blessing or Curse?

Abstract We report on an experiment on the distracting effects of in-car conversations through augmented-reality glasses. Previous research showed that in-car phone conversations can be distracting, but that the distraction might be reduced if the remote caller receives visual information about the driving context. However, what happens if such video sharing becomes bidirectional? The recent introduction of commercial augmented-reality glasses in particular might allow drivers to engage in video-supported conversations while driving. We investigate how distracting such video-based conversations are in an experiment. Our participants operated a simulated vehicle, while also playing a conversational game (Taboo) with a remote conversant. The driver either only heard the remote conversant (speech-only condition), or was also able to see the remote person in a virtual window that was presented through augmented reality (video call condition). Results show that our participants did not spend time looking at the video of the remote conversant. We hypothesize that this was due to the fact that in our experiment participants had to turn their head to get a full view of the virtual window. Our results imply that we need further studies on the effects of augmented reality on the visual attention of the driver, before the technology is used on the road.