Optical Head-mounted Display Research Articles

Surgical Training Using Augmented Reality-Based Optical Head-Mounted Displays: A Pilot Study.

While the educational benefits of the modern techniques such as virtual reality (VR) or augmented reality (AR) have been suggested, there is still a lack of reports on actual surgeons' experiences. In this study, we evaluated the effectiveness of a holographic AR-based surgical training in tonsillectomy. Two otolaryngologists, 1 trainee and 1 instructor, performed 5 tonsillectomies using an AR headset (HoloLens 2, Microsoft, USA). The trainee wore the AR headset to share the surgical view through front camera while the instructor remotely accessed the device using the Microsoft Teams program and provided real-time guidance. The AR-based surgical training offered several advantages, including direct real-time guidance for the trainee and clear instructions without disturbing the surgical process. However, there were also drawbacks, such as the front camera not always matching the trainee's view and some difficulty with focusing, depending on the depth of the oral cavity. Our study suggests that AR devices are a feasible and alternative method for surgical training. With the ability to provide clear guidance, even from a distance, this technology has the potential to revolutionize surgical training in the future.

VidAdapter

Video lectures are increasingly being used by learners in a ubiquitous manner. However, existing video designs are not optimised for ubiquitous use, creating the need to adapt the style of these videos to meet the constraints of the learning platform and context of use. Our formative study with experienced video editing users, however, found that performing these adaptations using traditional video editors can be a challenging and time-consuming task. We developed VidAdapter, a tool that facilitates lecture video adaptation by allowing direct manipulation of the video content. For this, VidAdapter automatically extracts meaningful elements from the video, enables spatial and temporal reorganisation of the elements, and streamlines the modification of an element's visual appearance. We demonstrate the capabilities and specific use cases of VidAdapter within the domain of adapting existing blackboard lecture videos for on-the-go learning on Optical Head-Mounted Displays. Our evaluation of the tool with experienced video editing users revealed that VidAdapter was strongly preferred over traditional approaches and can improve the efficiency of the adaptation process by over 53% on average.

Inter-Subject EEG Synchronization during a Cooperative Motor Task in a Shared Mixed-Reality Environment

Mixed-reality (MR) environments, in which virtual objects are overlaid on the real environment and shared with peers by wearing a transparent optical head-mounted display, are considered to be well suited for collaborative work. However, no studies have been conducted to provide neuroscientific evidence of its effectiveness. In contrast, inter-brain synchronization has been repeatedly observed in cooperative tasks and can be used as an index of the quality of cooperation. In this study, we used electroencephalography (EEG) to simultaneously measure the brain activity of pairs of participants, a technique known as hyperscanning, during a cooperative motor task to investigate whether inter-brain synchronization would be also observed in a shared MR environment. The participants were presented with virtual building blocks to grasp and build up an object cooperatively with a partner or individually. We found that inter-brain synchronization in the cooperative condition was stronger than that in the individual condition (F(1, 15) = 4.70, p < 0.05). In addition, there was a significant correlation between task performance and inter-brain synchronization in the cooperative condition (rs = 0.523, p < 0.05). Therefore, the shared MR environment was sufficiently effective to evoke inter-brain synchronization, which reflects the quality of cooperation. This study offers a promising neuroscientific method to objectively measure the effectiveness of MR technology.

Effects of the number of strokes, components and usage frequency on the legibility of Chinese characters for optical head-mounted display

ABSTRACT This study investigates the visual angle threshold value for the recognition of Chinese characters displayed on the optical head-mounted display. As optical head-mounted displays are becoming increasingly popular in various fields, character legibility has become an important topic for research. In this study, a legibility test was conducted by randomly displaying characters on the optical head-mounted display. The characteristics that were tested included the number of strokes, components and usage frequency rankings. The visual angle threshold value of each character was recorded and analysed. The results indicate that Chinese characters with more strokes, more components and lower usage frequency have a larger visual angle threshold value than characters with fewer strokes, fewer components and higher frequency. The number of strokes and usage frequency rankings were inputted into a regression model to predict the visual angle threshold value for character recognition (R 2 = 0.72; Adj-R2 = 0.71). The regression analyses revealed that the foregoing inputs increased the visual angle threshold value. These findings provide a practical reference for the display of Chinese characters on optical head-mounted displays.

Determining Work-Rest Schedules for Visual Tasks That Use Optical Head-Mounted Displays Based on Visual Fatigue and Visually Induced Motion Sickness Recovery.

This study aimed to determine work-rest schedules for visual tasks of different lengths by evaluating visual fatigue and visually induced motion sickness (VIMS) using an optical head-mounted display (OHMD). Thirty participants were recruited to perform 15 and 30 min visual tasks using an OHMD. After completing each visual task, participants executed six levels of rest time. Critical flicker fusion frequency (CFF) values, relative electroencephalography indices, and Simulator Sickness Questionnaire (SSQ) scores were collected and analyzed. Results indicated that after completing the 15 and 30 min visual tasks, participants experienced visual fatigue and VIMS. There was no significant difference between baseline CFF values, four electroencephalography relative power index values, and SSQ scores when participants completed a 15 min visual task followed by a 20 min rest and a 30 min visual task followed by a 30 min rest. Based on our results, a 20 min rest for visual fatigue and VIMS recovery after a 15 min visual task on an OHMD and a 25 min rest for visual fatigue and VIMS recovery after a 30 min visual task on an OHMD are recommended. This study suggests a work-rest schedule for OHMDs that can be used as a reference for OHMD user guidelines to reduce visual fatigue and visually induced motion sickness.

Virtual Reality versus Augmented Reality

Technology is developing quickly, making a lot of things that were before impossible feasible. Virtual reality (VR) and augmented reality (AR) are two examples of the cutting-edge technology that, only a few years ago, were considered science fiction. However, they are now an essential component of everyday life. The distinction between augmented reality & virtual reality is the one that causes the most misunderstanding in the world. While Virtual Reality is entirely immersive, augmented realities is a synthetic, computer-simulated reality or reproduction of a real-time setting that allows a user to interact with the reproduced actual settings. Optical head-mounted displays (OHMDs), which combine virtual reality (VR) or augmented reality (AR), are close to becoming common gear that consumers may purchase and use to do 3D tasks. Front-facing cameras are a feature of certain OHMDs that enable augmented reality (AR) or virtual reality (VR) capability. Seeing the actual world may alter interaction with virtual items in addition to preventing collisions with the surroundings. It is unknown if virtual reality (VR) or augmented reality (AR) provides any advantages over one another for almost all jobs. One of the most crucial areas of research in the present-day IT field is the development of augmented reality as well as virtual reality technologies. These two technologies have been used in a variety of fields, including education, healthcare, construction, military operations, and entertainment, and research on their technical aspects is progressing concurrently with research on their evaluation and the development of user-centered optimisation programmes. The starting place for this article was the real estate show. The authors selected the appropriate assessment index as well as quantification technique through comparisons with virtual reality (VR) and augmented reality (AR) in the physical display field, compared the evaluation results of each technology, and then created a user experience evaluation model for each.

Virtual Reality versus Augmented Reality

Technology is developing quickly, making a lot of things that were before impossible feasible. Virtual reality (VR) and augmented reality (AR) are two examples of the cutting-edge technology that, only a few years ago, were considered science fiction. However, they are now an essential component of everyday life. The distinction between augmented reality & virtual reality is the one that causes the most misunderstanding in the world. While Virtual Reality is entirely immersive, augmented realities is a synthetic, computer-simulated reality or reproduction of a real-time setting that allows a user to interact with the reproduced actual settings. Optical head-mounted displays (OHMDs), which combine virtual reality (VR) or augmented reality (AR), are close to becoming common gear that consumers may purchase and use to do 3D tasks. Front-facing cameras are a feature of certain OHMDs that enable augmented reality (AR) or virtual reality (VR) capability. Seeing the actual world may alter interaction with virtual items in addition to preventing collisions with the surroundings. It is unknown if virtual reality (VR) or augmented reality (AR) provides any advantages over one another for almost all jobs. One of the most crucial areas of research in the present-day IT field is the development of augmented reality as well as virtual reality technologies. These two technologies have been used in a variety of fields, including education, healthcare, construction, military operations, and entertainment, and research on their technical aspects is progressing concurrently with research on their evaluation and the development of user-centered optimisation programmes. The starting place for this article was the real estate show. The authors selected the appropriate assessment index as well as quantification technique through comparisons with virtual reality (VR) and augmented reality (AR) in the physical display field, compared the evaluation results of each technology, and then created a user experience evaluation model for each.

Application of Smart Glasses for Field Workers Performing Soil Contamination Surveys with Portable Equipment

Currently, portable X-ray fluorescence (PXRF) analysis is widely used as an auxiliary method for the preliminary investigation of soil heavy metal contamination. In this study, a smart glasses-based application (app) was developed to support field workers performing soil contamination surveys with a PXRF analyzer. The app was developed using the MIT App Inventor and runs on smart glasses based on an optical head-mounted display that provides both the original function of glasses to see the objects in front of the wearer, and the function of a computer at the same time. Using the app, a field worker wearing smart glasses can move to soil sampling points while checking the satellite image, survey plan, and real-time locations of other field workers through the smart glasses. At a sampling point, the worker can use both hands to collect and pretreat soil samples, and then measure the content of elements using a PXRF analyzer. The measurement results can be entered into the app using a wearable keyboard and shared in real-time with other field workers. The demonstration at the Ilgwang mine in Korea revealed that the app could effectively support field workers and shorten the working time compared to a previous study that was performed under the same conditions. The subjective workload was evaluated using the NASA task load index on ten subjects, and most of workload factors were evaluated as low.

Extended Reality-Assisted Surgery as a Surgical Training Tool: Pilot Study Presenting First HoloLens-Assisted Complex Penile Revision Surgery

Extended reality-assisted urologic surgery (XRAS) is a novel technology that superimposes a computer-generated image on the physician's field to integrate common elements of the surgical process in more advanced detail. An extended reality (XR) interface is generated using optical head-mounted display (OHMD) devices. To present the first case of HoloLens-assisted complex penile revision surgery. We describe our pilot study of HoloLens-assisted penile revision surgery and present a thorough review of the literature regarding XRAS technology and innovative OHMD devices. The ability of XRAS technology to superimpose a computer-generated image of the patient and integrate common elements of the surgical planning process with long-distance experts. XRAS is a feasible technology for application in complex penile surgical planning processes. XRAS and OHMD devices are novel technologies applicable to urological surgical training and planning. Evidence suggests that the potential use of OHMD devices is safe and beneficial for surgeons. We intend to pioneer HoloLens technology in the surgical planning process of a malfunctioning penile implant due to herniation of the cylinder. This novel technology has not been used in prosthetic surgery, and current data about XRAS are limited. OHMD devices are effective in the operative setting. Herein, we successfully demonstrated the integration of Microsoft HoloLens 2 into a penile surgical planning process for the first time. Further development and studies for this technology are necessary to better characterize the XRAS as a training and surgical planning tool. Quesada-Olarte J, Carrion RE, Fernandez-Crespo R, et al. Extended Reality-Assisted Surgery as a Surgical Training Tool: Pilot Study Presenting First HoloLens-Assisted Complex Penile Revision Surgery. J Sex Med 2022;19:1580-1586.

A Survey on Smart Glass Technology

The smart glass technology aims to improve wearable computing devices in today’s world. The werable compter glasses can change the optical properties at runtime. Its technology combines humans and machines with the help of new emerging technologies. This is made up of an optical head-mou nted display or embedded wireless glass with a transparent heads-up display or augumented reality (AR) overlay[7]. This technology has gained interest but it hasn't received any commercial breakthroughs as it is still a young and undeveloped technology. In this paper, you can study the complete overview of smart glasses companies that are currently available on the market in 2021 and 2022[10]. You can study the paradigms which are augmented reality, virtual reality, and extended reality. In this paper, you can also find the features, types of glass, application of smart glasses, and conclusion at the end.

Augmented Reality-Based Surgery on the Human Cadaver Using a New Generation of Optical Head-Mounted Displays: Development and Feasibility Study.

BackgroundAlthough nearly one-third of the world’s disease burden requires surgical care, only a small proportion of digital health applications are directly used in the surgical field. In the coming decades, the application of augmented reality (AR) with a new generation of optical-see-through head-mounted displays (OST-HMDs) like the HoloLens (Microsoft Corp) has the potential to bring digital health into the surgical field. However, for the application to be performed on a living person, proof of performance must first be provided due to regulatory requirements. In this regard, cadaver studies could provide initial evidence.ObjectiveThe goal of the research was to develop an open-source system for AR-based surgery on human cadavers using freely available technologies.MethodsWe tested our system using an easy-to-understand scenario in which fractured zygomatic arches of the face had to be repositioned with visual and auditory feedback to the investigators using a HoloLens. Results were verified with postoperative imaging and assessed in a blinded fashion by 2 investigators. The developed system and scenario were qualitatively evaluated by consensus interview and individual questionnaires.ResultsThe development and implementation of our system was feasible and could be realized in the course of a cadaver study. The AR system was found helpful by the investigators for spatial perception in addition to the combination of visual as well as auditory feedback. The surgical end point could be determined metrically as well as by assessment.ConclusionsThe development and application of an AR-based surgical system using freely available technologies to perform OST-HMD–guided surgical procedures in cadavers is feasible. Cadaver studies are suitable for OST-HMD–guided interventions to measure a surgical end point and provide an initial data foundation for future clinical trials. The availability of free systems for researchers could be helpful for a possible translation process from digital health to AR-based surgery using OST-HMDs in the operating theater via cadaver studies.

Is the use of augmented reality-assisted surgery beneficial in urological education? A systematic review.

Background:Google Glass is an optical head-mounted display that has been used in multiple medical and surgical settings to enhance delivery of education and training. This systematic review focuses solely on the use of this technology in urology operating theaters for the purpose of surgical education.Materials and methods:A systematic search strategy was employed using EMBASE (1996–2019), Medline (1946–2019) and PubMed. Search terms included optical head-mounted displays, Google Glass and urological surgical training. Use of this technology in a nonurological setting, nonteaching sessions, case reports, reviews, editorials, abstracts, and articles not in English were rejected. Three studies were identified following the exclusion criteria.Results:All 3 studies received positive feedback from trainees regarding this technology in relation to enhanced surgical education. In addition, in all studies the trainees felt the technology had a place for educational training in the future. All studies described disadvantages to the technology as well including battery life, comfort, and cost.Conclusions:Studies describe a big potential for Google Glass and similar head-mounted devices for the role of surgical training in urology, however, larger studies looking at more varied operations can help reinforce this viewpoint.

LSVP

The ubiquity of mobile phones allows video content to be watched on the go. However, users' current on-the-go video learning experience on phones is encumbered by issues of toggling and managing attention between the video and surroundings, as informed by our initial qualitative study. To alleviate this, we explore how combining the emergent smart glasses (Optical Head-Mounted Display or OHMD) platform with a redesigned video presentation style can better distribute users' attention between learning and walking tasks. We evaluated three presentation techniques: highlighting, sequentiality, and data persistence to find that combining sequentiality and data persistence is highly effective, yielding a 56% higher immediate recall score compared to a static video presentation. We also compared the OHMD against smartphones to delineate the advantages of either platform for on-the-go video learning in the context of everyday mobility tasks. We found that OHMDs improved users' 7-day delayed recall scores by 17% while still allowing 5.6% faster walking speed, especially during complex mobility tasks. Based on the findings, we introduce Layered Serial Visual Presentation (LSVP) style, which incorporates sequentiality, strict data persistence, and transparent background, among other properties, for future OHMD-based on-the-go video learning.

Assessing augmented reality in production: remote-assisted maintenance with HoloLens

Digitalisation provides many opportunities for process improvements by using smart devices such as Augmented-reality (AR) displays in production. Nevertheless, in-depth studies about the effects of these applications on key performance process indicators are sparse. The goal of this study is to evaluate AR-based remote maintenance processes supported by cutting edge optical head-mounted display technology. A total of 12 test runs conducted with real-world maintenance engineers in a real-world industry setting were analysed based on video recordings and semi-structured interviews. The study identified both case-specific and general issues, and the inherent potential of the technology.

Usability evaluation of monocular optical head-mounted displays on reading tasks

This study explores the factors affecting reading time, error rate, visual fatigue, and mental workload of subjects using a monocular optical head-mounted display (OHMD). Forty healthy participants were involved in this study using four factors: gender, reading conditions (walking and sitting), font sizes (12-, 16-, 20-, 24-, 28-, and 32-point), and contrast ratios (3.0:1, 7.5:1, 12.0:1, 16.5:1, and 21.0:1). The reading time, error rate, critical fusion frequency, the National Aeronautics and Space Administration’s raw task load index, and visual fatigue scale were the dependent measures. Gender showed no significant effect on reading time or error rate. Reading conditions, font sizes, and contrast ratios had significant effects on reading time. Large font sizes produced the fastest reading times for both walking and sitting conditions. The walking condition produced significantly more visual fatigue and mental workload than the sitting condition. Font size and contrast ratio had significant effects on reading time and error rate. The fastest reading time was found at the 16.5:1 contrast ratio for both walking and sitting conditions. Combined with the 28-point font size, we found the fastest reading time and lowest error rate performance combination. Considering reading conditions, contrast ratios, and font size combinations, the 16.5:1 contrast ratio with 28-point font size in the sitting condition and the 16.5:1 contrast ratio with 32-point font size in the walking condition showed the best reading times and error rates. The results from this study can provide very useful reference information for OHMDs and related product interface designs.

Creating Mobile Augmented Reality Experiences in Unity

This lesson serves as an introduction to creating mobile augmented reality applications. Augmented reality (AR) can be defined as the overlaying of digital content (images, video, text, sound, etc.) onto physical objects or locations, and it is typically experienced by looking through the camera lens of an electronic device such as a smartphone, tablet, or optical head-mounted display.

Three-dimensional sound-field visualization system using head mounted display and stereo camera

Visualization of a sound field helps us to intuitively understand various acoustic phenomena in sound design and education. The most straightforward method is to overlap the measured data onto a photographic image. However, in order to understand an entire three-dimensional (3D) sound field by using a conventional two-dimensional screen, it is necessary to move a camera and measure repeatedly. On the other hand, the augmented reality (AR) techniques such as an optical head mounted display (OHMD) have been rapidly developed. In this study, we propose a sound field visualization system using an OHMD and a handheld four-point microphone. This system calculates sound intensity from the four sound signals in real time. Then, the sound intensity distribution is depicted as arrows in the 3D display. The position and angle of the microphones and user’s head are acquired via AR markers and a gyro sensor of the OHMD. The system realizes simple and effective visualization of 3D sound field information from the vario...

Ebsar: Indoor guidance for the visually impaired

The number of people experiencing vision loss has been increasing rapidly, leading to great interest in assistive technologies research. Together with the emergence of wearable devices equipped with cameras and sensors, these technologies integrate multiple hardware and software tools to help the blind. This paper describes Ebsar, an Arabic navigation aid with an optical head-mounted display that guides blind people indoors. The application constructs indoor maps by tracking the movements of a sighted person inside a building, generates and prints QR code location markers for points of interest, and then provides voice guidance in Arabic to blind users. A user acceptance study, in which 15 sighted people evaluated the map construction process and 9 blind users evaluated the guidance system's usability and effectiveness, demonstrated the ease of use of the Ebsar application.

Introduction to Mobile Augmented Reality Development in Unity

This lesson serves as an introduction to creating mobile augmented reality applications. Augmented reality (AR) can be defined as the overlaying of digital content (images, video, text, sound, etc.) onto physical objects or locations, and it is typically experienced by looking through the camera lens of an electronic device such as a smartphone, tablet, or optical head-mounted display.

Augmented reality assisted surgery: a urologic training tool.

Augmented reality is widely used in aeronautics and is a developing concept within surgery. In this pilot study, we developed an application for use on Google Glass® optical head-mounted display to train urology residents in how to place an inflatable penile prosthesis. We use the phrase Augmented Reality Assisted Surgery to describe this novel application of augmented reality in the setting of surgery. The application demonstrates the steps of the surgical procedure of inflatable penile prosthesis placement. It also contains software that allows for detection of interest points using a camera feed from the optical head-mounted display to enable faculty to interact with residents during placement of the penile prosthesis. Urology trainees and faculty who volunteered to take part in the study were given time to experience the technology in the operative or perioperative setting and asked to complete a feedback survey. From 30 total participants using a 10-point scale, educational usefulness was rated 8.6, ease of navigation was rated 7.6, likelihood to use was rated 7.4, and distraction in operating room was rated 4.9. When stratified between trainees and faculty, trainees found the technology more educationally useful, and less distracting. Overall, 81% of the participants want this technology in their residency program, and 93% see this technology in the operating room in the future. Further development of this technology is warranted before full release, and further studies are necessary to better characterize the effectiveness of Augmented Reality Assisted Surgery in urologic surgical training.