Head-up Display Research Articles

Swimming with a head-mounted display: dual-task costs

Head-up displays (HUDs) have the potential to change work in operation environments by providing hands-free information to wearers. However, these benefits may be accompanied by trade-offs, primarily by increasing cognitive load due to dividing attention. Previous studies have attempted to understand the trade-offs of HUD usage; however, all of which were focused on land-based tasks. A gap in understanding exists when examining HUD use in aquatic environments as immersion introduces unique environmental and physiological factors that could affect multitasking. In this study, we investigated multitasking performance associated with swimming with a HUD. Eighteen participants completed three tasks: swimming only, a HUD-administered word recall task, and a dual-task combining both tasks. Results revealed significant dual-task interference in both tasks, though possibly less pronounced than in land-based tasks. These findings enhance not only help characterise dual-task performance, but also offer valuable insights for HUD design for aquatic settings.

Evaluating Driver Preferences for In-Vehicle Displays during Distracted Driving Using Driving Simulators

Research on the impact of distracted driving on in-vehicle displays was conducted through a user preference experiment based on simulated driving. The development of automotive automation and multi-screen in-vehicle displays has improved the driving experience but also increased distracted driving. Typical scenarios and information needs were identified through user research, and a simulated driving platform was used to record user choices of different display modes and their reaction times after being distracted. Questionnaires were analyzed using the Analytic Hierarchy Process (AHP). Results showed that in the manual driving group, users preferred Head-Up Display (HUD), which had the highest efficiency in terms of reaction time. The study confirmed that as automation levels increase, distraction scenarios will change, and user focus will shift from safety to experience.

Comparing Three-Dimensional Digitally Enabled Intraoperative OCT With Conventional Microscope-Integrated OCT in Vitreoretinal Surgery: A Post Hoc Analysis of the DISCOVER Study.

This study compared the surgeon experience between conventional microscope-integrated intraoperative optical coherence tomography (iOCT) and digitally enabled microscope-integrated iOCT in vitreoretinal surgery. This is a post hoc case-control analysis of the DISCOVER study. Conventional microscope-integrated iOCT (Rescan 700, Zeiss) was compared with digitally enabled iOCT (Artevo 800, Zeiss). Compared variables included surgical field-based visualization (ie, ocular heads-up display in the conventional group; three-dimensional screen-based visualization in the digital iOCT group) and non-surgical field-based visualization (ie, review on the external two-dimensional monitor). A total of 200 patients were included. Surgical field-based visualization of iOCT was significantly higher in the digitally enabled group (P < 0.0001). Required endoillumination level was significantly lower in the digital iOCT group (P < 0.0001). Surgeons reported "significant" back discomfort and headache more frequently when using conventional iOCT (P = 0.003 and P = 0.001, respectively). Digitally enabled iOCT resulted in greater surgical visualization efficiency, appeared to require a lower illumination level, and may provide advantages for ergonomic-related discomfort. [Ophthalmic Surg Lasers Imaging Retina 2024;55:270-277.].

Automotive Augmented Reality Head-Up Displays.

As the next generation of in-vehicle intelligent platforms, the augmented reality heads-up display (AR-HUD) has a huge information interaction capacity, can provide drivers with auxiliary driving information, avoid the distractions caused by the lower head during the driving process, and greatly improve driving safety. However, AR-HUD systems still face great challenges in the realization of multi-plane full-color display, and they cannot truly achieve the integration of virtual information and real road conditions. To overcome these problems, many new devices and materials have been applied to AR-HUDs, and many novel systems have been developed. This study first reviews some key metrics of HUDs, investigates the structures of various picture generation units (PGUs), and finally focuses on the development status of AR-HUDs, analyzes the advantages and disadvantages of existing technologies, and points out the future research directions for AR-HUDs.

Evaluating the Effectiveness of Contact-Analog and Bounding Box Prototypes in Augmented Reality Head-Up Display Warning for Chinese Novice Drivers Under Various Collision Types and Traffic Density

Augmented Reality Head-Up Display (AR-HUD) is a promising solution to the current warning system distraction problem. However, how to effectively convey warnings through AR graphics is still unclear. This study examines the effectiveness of the contact-analog graphic compared to the bounding box graphic in various collision types and traffic densities. Forty-eight participants watched AR-augmented driving videos and were instructed to respond to critical events. Reaction time, response rate, and subjective evaluations were compared for rear-end and pedestrian collisions in different traffic densities under different warnings. Both bounding box and contact-analog warnings improved driving performance compared to the non-warning group. The contact-analog warning performed better for rear-end collisions, while the bounding box warning had a lower reaction time for pedestrian collisions, regardless of traffic density.

Head-Up Haptic Display Using Dual-Mode Haptic Actuator for Vehicles

Driving assistance modules are important components to improving driver safety in driving environments, which can create overloaded multisensory information. This paper introduces the concept of a head-up haptic display that assists drivers in keeping their visual attention focused, and then offers a haptic actuator that may be used as the main component in the head-up haptic display. We selected magneto-rheological elastomer (MRE) as the base material and adopted a porous structure to reduce its initial stiffness. Furthermore, we fabricate a haptic actuator based on an MRE having a porous structure (pMRE) and a solenoid. The current to the solenoid generates a magnetic field that causes the pMRE to stiffen, thereby increasing the resistive force. As soon as the current input is removed, the pMRE becomes soft again. We simulated the solenoid coil and the magnetic path to further maximize the magneto-motive force generated by the solenoid coil and experimentally optimized the pMRE. We showed that the proposed haptic actuator creates not only a sufficient resistive force but also vibration to generate various haptic sensations. Moreover, we constructed a head-up haptic display using four haptic actuators and demonstrated the possibility of utilizing a new user interface to increase driving safety.

The Development and Usability Assessment of an Augmented Reality Decision Support System to Address Burn Patient Management

Critical care injuries, such as burn trauma, require specialized skillsets and knowledge. A clinical decision support system to aid clinicians in providing burn patient management can increase proficiency and provide knowledge content for specific interventions. In austere environments, decision support tools can be used to aid in decision making and task guidance when skilled personnel or resources are limited. Therefore, we developed a novel software system that utilizes augmented reality (AR) capabilities to provide enhanced step-by-step instructions based on best practices for managing burn patients. To better understand how new technologies, such as AR, can be used for burn care management, we developed a burn care application for use on a heads-up display. We developed four sub-set applications for documenting and conducting burn wound mapping, fluid resuscitation, medication calculations, and an escharotomy. After development, we conducted a usability study utilizing the System Usability Scale, pre- and post- simulation surveys, and after-action reviews to evaluate the AR-based software application in a simulation scenario. Results of the study indicate that the decision support tool has generalized usability and subjects were able to use the software as intended. Here we present the first use case of a comprehensive burn management system utilizing augmented reality capabilities to deliver care.

Cognitive evaluation of HUD interface layout for intelligent automotive based on Bayesian BWM and Gray-TOPSIS

To reduce drivers’ cognitive load during the driving process, The present study concentrates on the cognitive evaluation and analysis of the Head-Up Display (HUD) interface layout, aiming to enhance human cognitive efficiency. Initially, a combination of eye-tracking technology and cognitive load theory is used to investigate users’ attention allocation and changes in eye movement indicators, followed by the conversion of these indicators. A comprehensive HUD interface layout evaluation system is established, considering structural layout esthetics, task efficiency, and cognitive load. To achieve this, an intelligent cognitive evaluation method for the automotive HUD interface layout is proposed, based on the Bayesian BWM and Gray-TOPSIS. Bayesian BWM is employed to determine the weights of evaluation indicators, followed by Gray-TOPSIS to assess and rank the layout candidate solutions. Experimental results indicate that in the optimal layout design, users exhibit fewer eye movements, shorter gaze durations, esthetically pleasing interface structures, and lower cognitive loads. Furthermore, comparative experiments validate the effectiveness and stability of the Bayesian BWM and Gray-TOPSIS methods. These findings offer guidance and reference for further optimizing the layout of intelligent automotive HUD interfaces.

Colorful multi-plane augmented reality display with dynamically tunable reflective Pancharatnam-Berry phase lens.

Reflective cholesteric liquid crystal (CLC) Pancharatnam-Berry phase lens (PBL) devices have attracted significant attention in augmented reality (AR) display due to their wide spectral and angular response bandwidths, high diffraction efficiency, and polarization selectivity. However, currently reported CLC reflective PBLs are either limited by monochrome display or suffers from complicated design for colorful display. Herein, we demonstrate a colorful multi-plane AR display system with dynamically tunable reflective PBL. The reflective PBL is fabricated by polymer-stabilized cholesteric liquid crystal (PSCLC) that provides dynamical and continuous tunability of color and focal length by direct current (DC) voltage. A proof-of-concept colorful multi-plane AR device is demonstrated, where over 90% diffraction efficiency at desired wavelength has been obtained. The proposed simple, compact, and light AR display system capable of color-imaging with multi-depth shows great application potential in the vehicle-mounted head-up display (HUD).

The Effect of Arrow Spacing and Blinking Speed Difference on Driver's Visual Perception and Cognitive Functioning in an Augmented Reality Information Delivery Method of Head-Up Display

The Effect of Arrow Spacing and Blinking Speed Difference on Driver's Visual Perception and Cognitive Functioning in an Augmented Reality Information Delivery Method of Head-Up Display

Compact pupil-expansion AR-HUD based on surface-relief grating.

Augmented reality head-up display (AR-HUD) using diffractive waveguide is a challenging research field. It can drastically reduce the system volume compared with AR-HUD based on freeform mirror. However, one of the remaining challenges that affects the performance of the diffractive waveguide is to expand the eye-box while maintaining the illuminance uniformity. In this paper, a one-dimensional pupil expansion diffractive optical waveguide system for AR-HUD is presented. The optimization of grating parameters is based on scalar diffraction theory and rigorous coupled wave analysis (RCWA). Then, the illuminance uniformity is optimized through non-sequential ray tracing. We simulate and construct a waveguide-based AR-HUD. The presented AR-HUD realized an exit pupil size of 80 mm × 15 mm and a field of view of 10° × 5° at the wavelength of 532 nm.

Artificial Intelligence, Digital Imaging, and Robotics Technologies for Surgical Vitreoretinal Diseases

ObjectiveTo review recent technological advancement in imaging, surgical visualization, robotics technology and the use of artificial intelligence in surgical vitreoretinal (VR) diseases. BackgroundTechnological advancements in imaging enhance both pre-operative and intra-operative management of surgical VR diseases. Widefield imaging in fundal photography and optical coherence tomography (OCT) can improve assessment of peripheral retinal disorders like retinal detachments, degeneration, and tumors. OCT angiography provides a rapid and non-invasive imaging of the retinal and choroidal vasculature. Surgical visualization has also improved with intraoperative OCT providing a detailed real-time assessment of retinal layers to guide surgical decisions. Heads-up display and head-mounted display utilize three-dimensional technology to provide surgeons with enhanced visual guidance and improved ergonomics during surgery. Intraocular robotics technology allows for greater surgical precision and is shown to be useful in retinal vein cannulation and subretinal drug delivery. In addition, deep learning techniques leverages on diverse data including widefield retinal photography and OCT for better predictive accuracy in classification, segmentation, and prognostication of many surgical VR diseases. ConclusionThis review article summarized the latest updates in these areas and highlights the importance of continuous innovation and improvement in technology within the field. These advancements have the potential to reshape management of surgical VR diseases in the very near future and to ultimately improve patient care.

Impact of three-dimensional heads-up display on surgical performance, ergonomics and teaching in vitreoretinal surgery: first African study

PurposeTo assess the impact of a Three-Dimensional (3D) Heads-Up Display (HUD) compared to a conventional microscope (COVM) on surgical performance, ergonomics, and teaching in vitreoretinal (VR) surgery. DesignA single-center, retrospective cohort study and cross-sectional survey. MethodsSurgical assistants (N = 12) and Vitreoretinal Surgeons (N = 2) were asked, in the survey arm, to complete a questionnaire regarding various parameters related to ergonomics, quality of visualization, quality of teaching, and overall satisfaction. Parameters were rated on a scale from 1 to 10 for both the 3DHUD and COVM visualization systems (VS). The scores of the two groups were compared using appropriate statistical tests. The retrospective arm of the study included 34 consecutive cases that underwent vitrectomy with the 3DHUD VS compared to 32 matched controls with the COVM. Demographics and preoperative, intraoperative, and postoperative clinical parameters were identified in both groups and were compared using appropriate statistical tests. ResultsSurvey scores were significantly higher in the 3DHUD VS group when compared to the COVM VS group for both Surgeons and Assistants regarding ergonomic parameters (overall comfort (p < 0.001), neck pain (p < 0.001), and lower back pain (p < 0.001)), visualization parameters (overall visibility (p < 0.001), visibility of the macula (p < 0.001), depth of field (p < 0.001), and quality of magnification (p < 0.001)) and teaching parameters (understanding of surgical steps by assistants (p < 0.001) and teaching to medical students and residents (p < 0.001)). Both demographics and clinical data of cases operated with the 3DHUD VS and COVM VS were comparable (p > 0.05). Anatomical and functional outcomes were similar in both groups (p > 0.05); nevertheless, mean surgical time was significantly longer in the 3DHUD group for tractional retinal detachments (p < 0.001), epiretinal membranes, and phacofragmentation (p = 0.03). Conclusion3DHUD greatly enhances ergonomics during vitreoretinal surgery and represents a significant innovation in the teaching armamentarium within the operating room. It provides the same surgical view for both the surgeons and assistants with an unparalleled image quality. A learning curve and an integrated ecosystem may be necessary to maximize its potential and reduce surgical time.

Holographic multiplane augmented reality head-up display with switchable display modes based on polymer dispersed liquid crystal.

The multiplane augmented reality (AR) head-up display (HUD) is important in improving driving safety and comfort. In this paper, we propose an AR-HUD with switchable display modes based on polymer dispersed liquid crystal (PDLC) and lens holographic optical elements (HOEs), which can provide two display modes: the dual-virtual-image mode and the virtual-real-image mode. The dual-virtual-image mode can produce two virtual images at different depths, which can provide a better sense of reality integration for the driver to improve driving safety and comfort. The virtual-real-image mode can produce one far virtual image and one near real image at different depths, and it provides a larger eye box (EB) for both driver and passengers in the car and a higher image contrast. The two display modes can be switched by an electronically controlled scattering module consisting of a pair of PDLC films. The proposed AR-HUD system is compact and equipped with multiplane display and mode-switching functions, and is expected to be applied in the future.

Optimizing graphics for understanding real-scene using 3D image registration and tracking and mapping

A satisfactory graphic design and good-looking 3D models and environments are the backbones of a positive user experience, especially in Augmented Reality (AR) / Virtual Reality (VR) app development. Where these technologies is seen as the an excellent realm of human-computer interaction. The purpose is to fool the viewer by the seamless incorporation of simulated features. Every AR system relies on true interaction and three-dimensional registration to function properly. In this research, we present a strategy for real-world 3D image registration and tracking. The primary foci of this study are the first three stages: initial registrations and matrix acquisitions, road scene feature extraction, and virtual information registration. At initial registration, a rough virtual plane is estimated onto which the objects will be projected. To this, we propose YoloV3 for transferring features from a virtual to a real-world setting. The projection process concludes with a guess at the camera’s posture matrix. This tech is used in the vehicle’s head-up display to augment reality. The average time required to register a virtual item is 43 seconds. The final step in making augmented reality content is to merge the computer-generated images of virtual objects with real-world photographs in full colour. Our results indicate that this method is effective and precise for 3D photo registration but has the potential to dramatically increase the verisimilitude of AR systems.

Broadband beam deflector for GaN based micro-LED with a phase-gradient metasurface in entire visible spectrum

Metasurfaces with subwavelength structures are good candidates for beam manipulation which are usually working at a narrow bandwidth with high coherence. Here, we demonstrate a broadband beam deflector for GaN based micro-LED, utilizing a phase-gradient metasurface in conjunction with a resonant cavity. The beam deflector works at the entire visible spectrum, achieving deflection angles from 9° to 18°, with beam efficiencies of approximately 20% to 35%. This miniature structure is well-suited for applications such as full-color display, naked-eye three-dimensional display, and head-up display. It will facilitate the development of GaN based micro-LEDs with complex optical functionalities.

A Single-Frame Deflectometry Method for Online Inspection of Light-Transmitting Components.

Transparent materials are widely used in industrial applications, such as construction, transportation, and optics. However, the complex optical properties of these materials make it difficult to achieve precise surface form measurements, especially for bulk surface form inspection in industrial environments. Traditional structured light-based measurement methods often struggle with suboptimal signal-to-noise ratios, making them ineffective. Currently, there is a lack of efficient techniques for real-time inspection of such components. This paper proposes a single-frame measurement technique based on deflectometry for large-size transparent surfaces. It utilizes the reflective characteristics of the measured surface, making it independent of the surface's diffuse reflection properties. This fundamentally solves the issues associated with signal-to-noise ratios. By discretizing the phase map, it separates the multiple surface reflection characteristics of transparent devices, enabling transparent device measurement. To meet the requirements of industrial dynamic measurement, this technique only needs a simple and low-cost system structure, which contains just two cameras for image capture. It does not require phase shifting to complete the measurement, making it independent of the screen and having the potential for larger surface measurement. The proposed method was used to measure a 400mm aperture automobile glass, and the results showed that it is able to achieve a measurement accuracy on the order of 10 μ m. The method proposed in this paper overcomes the influence of surface reflection on transparent objects and significantly improves the efficiency and accuracy of large-sized transparent surface measurements by using a single-frame image measurement. Moreover, this method shows promise for broader applications, including measurements of lenses and HUD (Heads-Up Display) components, showcasing significant potential for industrial applications.

IoT-Based Integrated Heads-Up Display for Motorcycle Helmet

IoT-Based Integrated Heads-Up Display for Motorcycle Helmet

Design and Optimization of a Head-Up Display System with Variable Projection Distance

Design and Optimization of a Head-Up Display System with Variable Projection Distance

Dynamic Distortion Assessment in Automobile Head-Up Displays with Subjective Methods

Dynamic Distortion Assessment in Automobile Head-Up Displays with Subjective Methods