Augmented Reality Glasses Research Articles

Augmented Reality Driven Steady-State Visual Evoked Potentials for Wheelchair Navigation.

Medically oriented Brain Computer Interfaces (BCIs) have been proposed as a promising approach addressed to individuals suffering from severe paralysis. Steady-State Visual Evoked Potentials (SSVEPs) in particular have been proven successful in many different applications, achieving high information throughput with short or even no training. However, efficient electric wheelchair navigation combining high accuracy and comfort is still not demonstrated. In this paper, we propose the use of an SSVEP-based universal control system featuring augmented reality (AR) glasses in an attempt to increase ease of use and patient acceptability without making compromises on BCI performance. The system received positive user-feedback, reaching a mean accuracy of 90%. Merits and pitfalls of the system proposed are also addressed.

A Natural Bare-Hand Interaction Method With Augmented Reality for Constraint-Based Virtual Assembly

Traditional virtual assembly methods have a high requirement in hand-eye coordination because of the separation of feedback and operation regions. These methods are inconsistent with human interaction habits and lack of naturalness because of limited interaction space and indirect interaction mode. Therefore, we propose a natural bare-hand interaction method for virtual assembly, enabling operators to interact with virtual objects by using natural gestures even while in motion. The Leap Motion controller (LMC) fixed on the Augmented Reality (AR) glasses is used to track hands of the operator and the mobility of the interactive device relieves the location limitation of assembly processes. Furthermore, AR allows operators to perform bare-hand assembly in a realistic situation. The interval Kalman filter (IKF) is applied to estimate hands’ positions to improve the accuracy of measured gesture data. Moreover, constraint assisted technology is introduced to aid operators in learning and completing assembly tasks quickly and accurately, and the assembly sequence is generated to assist the decision-making process during the interaction. Experimental results show that the proposed method can be used by non-professional operators for assembly tasks and can potentially improve assembly efficiency. Based on operator ratings about assembly experience and significant difference analysis, the proposed method performed better at providing better interactive experiences.

Application of the Augmented Reality Goggle for a Muting Procedure with Convolutional Neural Networks Algorithm

The use of augmented reality glasses is one of many elements of the standard known as Industry 4.0. This article shows the use of Microsoft HoloLens headset to support the work of the semi-automatic machine operator. As the quality of collecting cloud points for these glasses is not enough, it was decided to implement a neural algorithm that was used to filter the online image obtained from augmented reality headset.

Surgical Applications of Lymphatic Vessel Visualization Using Photoacoustic Imaging and Augmented Reality.

Lymphaticovenular anastomosis (LVA) is a widely performed surgical procedure for the treatment of lymphedema. For good LVA outcomes, identifying lymphatic vessels and venules is crucial. Photoacoustic lymphangiography (PAL) is a new technology for visualizing lymphatic vessels. It can depict lymphatic vessels at high resolution; therefore, this study focused on how to apply PAL for lymphatic surgery. To visualize lymphatic vessels, indocyanine green was injected as a color agent. PAI-05 was used as the photoacoustic imaging device. Lymphatic vessels and veins were visualized at 797- and 835-nm wavelengths. First, it was confirmed whether the branching of the vasculature as depicted by the PAL was consistent with the actual branching of the vasculature as confirmed intraoperatively. Second, to use PAL images for surgical planning, preoperative photoacoustic images were superimposed onto the patient limb through augmented reality (AR) glasses (MOVERIO Smart Glass BT-30E). Lymphatics and venule markings drawn using AR glasses were consistent with the actual intraoperative images obtained during LVA. To anastomose multiple lymphatic vessels, a site with abundant venous branching was selected as the incision site; and selecting the incision site became easier. The anatomical morphology obtained by PAL matched the surgical field. AR-based marking could be very useful in future LVA.

Augmenting the City: The Photo-Realistic Animation of a Historic Building and its influence on spatial perception and meaning

Augmented Reality (AR), defined as the holographic overlay of physical space with virtual objects in real time (Azuma, 1997), can be considered a prime example of mediatization. This development is particularly evident in the public space of the “mediatised city” (Hepp, Simon & Sowinska, 2018), being a focal point of the latest media technologies already overlaid with a multitude of AR content. But how does AR change the perception and meaning of urban space? And how can researchers capture methodically the appropriation of complex, large-scaled AR content experienced via high-tech AR glasses? To answer these questions, a historical building, that had been destroyed during the Second World War, was reconstructed as a holographic animation on a public city square. In order to resurrect this building in AR, old photographs, paintings and postcards were evaluated and used to create a virtual model in the original size and place it at its original location. The test subjects were then able to view the hologram from various different angles using AR glasses (Microsoft HoloLens 2), move freely around the square and even enter it. Combining quantitative, before-and-after questionnaires and qualitative thinking-aloud protocols, our results show that the holographic animation of a historical building can influence both the sensual-aesthetic perception and the personal meaning of a public square for city dwellers. Specifically, our test subjects perceived differences in its accessibility, coherence and aesthetics, simplicity, atmosphere and legibility. The meaning of the square was altered with regard to personal memories (= the self), typical groups of people (= others) and certain opportunities (= environment) associated with it by city dwellers.

Fostering Performance in Hands-On Laboratory Work with the Use of Mobile Augmented Reality (AR) Glasses

The learning of laboratory skills is essential in science education, but students often get too little individual guidance in this area. Augmented reality (AR) technologies are a promising tool to tackle this challenge and promote students’ high-level learning and performance in science laboratories. Thus, the purpose of this study was (1) to design an AR-assisted learning environment to support individual knowledge construction, (2) to investigate students’ learning processes and learning outcomes and (3) to examine the usability of the system. Pharmacy students (n = 16) were assigned to experimental (n = 10) and control (n = 6) groups and performed the same laboratory work together with pre- and post-tests. The experimental group worked with AR glasses that provided additional support and timely guidance during the work with additional info-screens, questions related to choosing correct reagents and laboratory tools and think-aloud questions, whereas the control group worked in a traditional laboratory context. The results showed that AR was more effective in fostering performance in the science laboratory compared to traditional laboratory instruction and prevented most of the mistakes. The AR group considered the guidance and feedback provided by AR to be beneficial for their learning. However, no apparent differences were found in tasks measuring students’ understanding of the content knowledge. Thus, an AR environment embedded with supportive tools could partly replace the teacher in science teaching laboratories by providing individual and timely guidance for the students.

Augmented reality in vocational training: A systematic review of research and applications

Augmented reality (AR) technology is widely used in various fields. However, there are few systematic reviews on the application of AR in vocational training. To fill this research gap, the current study reviewed the application of AR technology in the training of various industries over a 20 year period (2000–2021). Through cross-referencing and abstract reading, 80 relevant studies were selected for the final analysis from two perspectives: the improvement of vocational skills (including application area, target audience, training objectives, and effects) and AR training technology (including AR application, AR training system, and device). Furthermore, CiteSpaceV was employed to analyze the research hotspots and trends of AR vocational training. The results indicated that AR training is frequently applied in the industry, vocational education and medical fields. Among these industries, AR has been most frequently used in medical training, industrial maintenance, and assembly. Furthermore, AR glasses, simulators, the Unity3D game engine, 360° panorama, AR systems and apps are becoming widely used for training tasks. The benefits of these systems have been identified. We also explored the impact of AR on vocational training results. Based on 17 empirical studies, this study summarized the results and advantages of AR vocational training. This verified that AR has a high promotion effect on vocational training when the meta-analysis method is used. Future researchers can study how vocational skills education can be combined with these new intelligent technologies to design more mature teaching practice cases.

A Methodology to Produce Augmented-Reality Guided Tours in Museums for Mixed-Reality Headsets

In recent years, the use of technology in the museum context has changed radically. It has switched from the display of information to offering emotive, immersive, and rich experiences with heritage. Virtual interactive media have the potential to put museums back into a relevant place in our increasingly digital society. The emergence of augmented-reality glasses offers the possibility to test and implement new methodologies compatible with this aim. However, most of the first examples developed in recent years did not take advantage of the possibilities of this new medium. This paper presents a novel methodology for producing mixed-reality applications for museums and heritage sites, with an intuitive, immersive, and natural way of operating. An experimental prototype designed for the archaeological museum of the Almoina is shown in the paper to demonstrate the benefits of the proposed system and methodology of production. In addition, the paper shows the results of several tests.

Canadian Conference for the Advancement of Surgical Education (C-CASE) 2021: Post-Pandemic and Beyond Virtual Conference AbstractsBlended learning using augmented reality glasses during the COVID-19 pandemic: the present and the futureActivating emotions enhance surgical simulation performance: a cluster analysisTraining in soft-tissue resection using real-time visual computer navigation feedback from the Surgery Tutor: a randomized controlled trialSonoGames: delivering a point of care

# Blended learning using augmented reality glasses during the COVID-19 pandemic: the present and the future {#article-title-2} The COVID-19 pandemic has caused major disruption to a normal way of life. It has also affected medical education, and traditional methods of delivering learning have not

Past Forward: Electricity’s Perilous Narrative Arc

Summary form only given, as follows. In L. Frank Baum’s 1901 classic "The Master Key: An Electrical Fairy Tale", Rob Josyln, a young boy who is wild about all things electrical, accidentally summons the Demon of Electricity to his attic workshop. Similarly to the genie in Aladdin’s lamp, the demon offers Rob three gifts for each of the next three weeks to demonstrate the extraordinary potential of electricity. The fantastical inventions Rob receives foreshadow modern-day Tasers, streaming video, and augmentedreality glasses. Adventure and near disaster ensue. Eventually, Rob decides to avoid further mishap and politely refuses any more gifts. In his sci-fi morality tale, Baum seems to recognize that the world of 120 years ago was not prepared for this futuristic technology. Are we ready now?

A Low-cost Mobile System with Multi-AR Guidance for Brain Surgery Assistance.

Surgical operation especially brain surgery requires comprehensive understanding on the surrounding area of the surgical path. Augmented Reality (AR) technology provided an effective way to increase the surgeon's perception on the plan. However, current applications were hindered by the expensive hardware and limited guidance information. In this paper, an AR system especially designed for brain surgery was proposed, which featured in low-cost system components and multi-AR guidance. A light-weight AR glasses was utilized together with normal mobile phone to provide mobile AR to the surgeon. A web-based application was implemented for compatibility of various mobile devices. Multi-AR information was designed for surgical guidance, including planned operation path, dangerous areas, and three quantitative guidance metrics. Patient's specific 3D model was reconstructed based on CT images, and the phantom was utilized to evaluate the effectiveness of the system. The experimental results indicated that the assistance of the multi-AR guidance outperformed the results of with no AR guidance at all and with virtual path guidance only. As a result, our system could help the operator to perform the operation tasks easier.Clinical Relevance- This proposed method provided a potential way for brain surgery with multiple AR guidance information with the assistance of a low-cost AR system, which may improve the surgeon's cognition on the surgical site.

Robust speech processing for voice interfaces and video content understanding at Facebook

As voice interfaces to devices and digital assistants have increased in popularity, so too, have the challenging environments in which they are expected to perform. In this talk, we’ll present an overview of the signal processing and speech recognition AI modeling techniques that we have developed at Facebook to enable robust voice interaction on Portal video calling devices and Oculus VR headsets. We will also describe progress in captioning and understanding the wide variety of video content shared on Facebook apps, where the acoustic conditions are diverse and challenging and the audio is typically captured on commodity mobile phones. While such systems have been historically developed to run on powerful servers in the cloud, there is increasing interest in speech models that can run locally on the client device. We will describe the challenges of on-device processing and our recent progress in creating efficient, low-footprint speech models. Finally, we will present the challenges and future directions we are exploring to enable rich voice interactions on the next generation of computing devices, including augmented reality glasses.

Next-Generation Imaging Techniques: Functional and Miniaturized Optical Lenses Based on Metamaterials and Metasurfaces.

A variety of applications using miniaturized optical lenses can be found among rapidly evolving technologies. From smartphones and cameras in our daily life to augmented and virtual reality glasses for the recent trends of the untact era, miniaturization of optical lenses permits the development of many types of compact devices. Here, we highlight the importance of ultrasmall and ultrathin lens technologies based on metamaterials and metasurfaces. Focusing on hyperlenses and metalenses that can replace or be combined with the existing conventional lenses, we review the state-of-art of research trends and discuss their limitations. We also cover applications that use miniaturized imaging devices. The miniaturized imaging devices are expected to be an essential foundation for next-generation imaging techniques.

Low‐Temperature Solution‐Processed Transparent QLED Using Inorganic Metal Oxide Carrier Transport Layers

AbstractQuantum dot light‐emitting diodes (QLEDs) represent an exciting new technology that has many desirable attributes when compared to existing organic LEDs (OLEDs) including increased brightness, contrast, and response time. Solution‐based fabrication approaches have the advantage of being able to produce large‐area electronic systems at reduced costs and critical in applications such as large display fabrication and electronics on curved surfaces including low‐profile augmented reality glasses. In this paper, for the first time, a fully solution‐processed transparent inorganic QLED is described. Traditional QLED fabrication methodologies require the use of air‐sensitive materials that make fabrication of these devices challenging and expensive. Instead of using air‐sensitive organic materials, in the approach, nickel oxide (NiO) is used as the hole transport layer and is deposited using a sol‐gel method. Copper doping of the NiO to reduce the turn‐on voltage of the QLED device is investigated. Importantly, the post‐annealing temperature of the sol‐gel process is below 275 °C, which permits the fabrication of QLEDs on a wide range of substrates. The experimental results are concordant with the COMSOL simulation data and demonstrate the feasibility of fabricating fully transparent inorganic QLED devices using a solution‐based process.

Technologically empowered? perception and acceptance of AR glasses and 3D printers in new generations of consumers.

This research investigates the perception, acceptance, and economic valuation of new generations of consumers concerning disruptive technologies (DT) using the modified Technology Acceptance Model (TAM) as a reference. Specifically, it analyzes a group of university students' responses after experimenting with augmented reality (AR) glasses and 3D printers. The results show an eminent interest in technology on the part of young people and a frequent acceptance of its inclusion in their daily lives, and a clear intention to acquire this type of technology someday. Likewise, social projection and self-realization should be considered by marketing managers for the positioning and support of DT's future sales among young people. Based on the existing literature and the research results, the role of technology in generations' daily lives Y and Z is discussed.

Using Cost Efficient Augmented Reality Glasses in Anatomical Identification

Augmented Reality (AR) is an emerging technology capable of integrating visual technology with the physical world in novel ways. AR uses may range from education delivery to the operating room. This study sought to evaluate cost-effective AR glasses compared to a traditional computer screen when identifying anatomical structures in a chest radiograph (X-ray). The purpose of this experiment was to determine if the Vuzix Blade AR technology is feasible for use in the medical field to identify anatomical structures. Novice undergraduate and graduate students (n = 14) were recruited to participate in this feasibility study. Radio-graphic images were compiled, and subjects were asked to identify 12 anatomical structures on each image. Images were randomly assigned to identify structures using either the AR glasses or a traditional computer screen for each set. Images were viewed randomly by each subject using a crossover design to compare traditional computer screen vs. AR screen. Subjects identified statistically fewer anatomical structures using the AR glasses compared to using the traditional computer screen (p<0.01). However, the glasses did not require significantly more subjective workload (NASA-TLX) on the user to use compared to the computer screen. When subjects used the cost-effective AR glasses, subjects were unable to differentiate as many anatomical structures as possible (10 vs. 9 structures at p < 0.01), but the glasses were not more demanding overall. Thus, AR glasses may soon be an integral part of the operating room, provided adequate resolution to visualize the necessary details.

56.2: Invited Paper: Evaluation of Optical See‐through Real Scene Image Quality in AR Eyewear Displays

Augmented Reality (AR) has become a hot topic. It seems AR is finally becoming ready for the generals: Public securities have been using AR glasses to assist their public patrol activities, the museum offers the AR glasses to enhance the visitor experience, the oil evaporation workers wear AR headset to help their evaporation operation, etc. However, the one component that is probably most limiting today's popularization is the display's image quality. This paper addressed the image quality of see‐through real scene for AR glasses. Michelson contrast or namely luminance modulation, is measured to evaluate the quality of an see‐through optics. In this paper, we studied the Michelson contrast at different locations of the optics of AR displays. Two types of AR displays were selected for the evaluation and measurement. This paper discussed the two favorable AR type technologies: waveguide and combiner (prism‐combiner).

Augmented reality head-mounted display-based incision planning in cranial neurosurgery: a prospective pilot study.

Monitor and wand-based neuronavigation stations (MWBNSs) for frameless intraoperative neuronavigation are routinely used in cranial neurosurgery. However, they are temporally and spatially cumbersome; the OR must be arranged around the MWBNS, at least one hand must be used to manipulate the MWBNS wand (interrupting a bimanual surgical technique), and the surgical workflow is interrupted as the surgeon stops to "check the navigation" on a remote monitor. Thus, there is need for continuous, real-time, hands-free, neuronavigation solutions. Augmented reality (AR) is poised to streamline these issues. The authors present the first reported prospective pilot study investigating the feasibility of using the OpenSight application with an AR head-mounted display to map out the borders of tumors in patients undergoing elective craniotomy for tumor resection, and to compare the degree of correspondence with MWBNS tracing. Eleven consecutive patients undergoing elective craniotomy for brain tumor resection were prospectively identified and underwent circumferential tumor border tracing at the time of incision planning by a surgeon wearing HoloLens AR glasses running the commercially available OpenSight application registered to the patient and preoperative MRI. Then, the same patient underwent circumferential tumor border tracing using the StealthStation S8 MWBNS. Postoperatively, both tumor border tracings were compared by two blinded board-certified neurosurgeons and rated as having an excellent, adequate, or poor correspondence degree based on a subjective sense of the overlap. Objective overlap area measurements were also determined. Eleven patients undergoing craniotomy were included in the study. Five patient procedures were rated as having an excellent correspondence degree, 5 had an adequate correspondence degree, and 1 had poor correspondence. Both raters agreed on the rating in all cases. AR tracing was possible in all cases. In this small pilot study, the authors found that AR was implementable in the workflow of a neurosurgery OR, and was a feasible method of preoperative tumor border identification for incision planning. Future studies are needed to identify strategies to improve and optimize AR accuracy.

Research on problems and countermeasures in the application of substation intelligent inspection system

Substation plays an important role in the power system as the pivot of the power grid. With the continuous expansion of the power grid, the intelligent inspection system based on robots is widely used in the substation industry. However, the existing inspection robot has some drawbacks, such as low battery capacity, weak obstacle performance, slow movement speed, and blind areas during the inspection. This paper proposed some measurements aimed at these drawbacks, such as optimizing the robot performance, adding the image acquisition PTZ (Pan, Tilt, and Zoom camera) in fixed point, using the AR (Augmented Reality) glasses assistance. These approaches improve the efficiency and quality of the existing intelligent inspection system in the substation.

Extended-Reality Technologies: An Overview of Emerging Applications in Medical Education and Clinical Care.

Extended-Reality Technologies: An Overview of Emerging Applications in Medical Education and Clinical Care.