Head-up Display Research Articles

Visions of Control: The Head-Up Display, Perceptual Labor, and a Lesson for Augmented Reality.

Traditional workplace studies focus on the bodily integration of familiar technologies like the desktop computer and smartphone. To better understand augmented reality (AR) technologies that provide new ways to perform daily tasks, this article turns to an overlooked yet critical forerunner of AR: the head-up display (HUD). Extensive archival research and oral histories show that the HUD caused pilots to depend on a single interface, curtailing their autonomy for integrating disparate information that could inform critical, often lethal decision-making processes. This history provides evidence that once a single device subsumes many functions of a work environment, adaptive articulation work can drive information workers to undertake increasingly insulated information practices. The HUD's isolating and insulating perceptual legacy suggests that AR will similarly train users in many contexts how to trust interactive data overlays in ways that preclude cross-checking alternative data sources that could inform decision-making processes.

Comparison of 3D heads-up display system with conventional surgical microscopy for minimally invasive glaucoma surgery on an artificial eye model

ObjectifComparer un nouveau dispositif de visualisation tête haute en 3 dimensions (3D) au microscope chirurgical classique dans la chirurgie du glaucome à effraction minimale (MIGS, pour minimally invasive glaucoma surgery) sur un modèle d’œil artificiel et évaluer l'opinion des chirurgiens ophtalmologistes à cet égard. Matériel et MéthodesAinsi, 21 ophtalmologistes qui participaient au congrès annuel de la Société canadienne d'ophtalmologie de 2021 à Halifax (Nouvelle-Écosse) ont pris part à une activité de transfert des compétences de 90 minutes sur la MIGS. À l'aide d'un modèle d’œil artificiel (SimulEYE iTrack; InsEYE LLC, Westlake Village, Californie), les participants ont réalisé des MIGS à l'aide d'un dispositif de visualisation tête haute en 3D (Zeiss Artevo 800; Carl Zeiss Meditec, Iéna, Allemagne) et d'un microscope chirurgical classique. Par la suite, les participants et les formateurs ont rempli un formulaire de 16 questions en vue de comparer les 2 systèmes (dispositif de visualisation tête haute en 3D vs microscope chirurgical classique). Les réponses ont été enregistrées sur une échelle Likert double en 9 points s’échelonnant comme suit : de fortement en faveur du dispositif de visualisation tête haute en 3D (1) à fortement en faveur du microscope chirurgical classique (9). Le test U de Mann–Whitney pour analyse non paramétrique a servi à comparer les réponses des formateurs à celles des participants, de même que les réponses des experts à celles des novices. RésultatsLes réponses favorisaient le dispositif de visualisation tête haute en 3D par rapport au microscope chirurgical classique; pour l'ensemble des questions, elles variaient de 1 (fortement en faveur du dispositif de visualisation tête haute en 3D) à 5 (équivalence des dispositifs). Le test U de Mann–Whitney n'a pas relevé de différence significative entre les réponses des formateurs et celles des participants, ni entre celles les experts et des novices. La plupart des évaluations favorisant le dispositif de visualisation tête haute en 3D portaient sur la configuration ergonomique du modèle chirurgical, la profondeur de champ et/ou le champ d'observation de même que sur son utilité pour entraîner des résidents à réaliser des MIGS. Les évaluations qui indiquaient une équivalence entre le dispositif de visualisation tête haute en 3D et le microscope chirurgical classique portaient sur les défaillances du système et les décalages de l'image pendant la chirurgie. ConclusionsTous les ophtalmologistes, sans égard à leur degré d'expérience, ont préféré le dispositif de visualisation tête haute en 3D au microscope chirurgical classique pour réaliser une MIGS simulée. D'après les résultats du sondage, le recours au dispositif de visualisation tête haute en 3D permet d'enseigner efficacement la MIGS sur un modèle d’œil artificiel, surtout depuis l'avènement de la formation axée sur les compétences en ophtalmologie.

Wavefront Characteristics of a Digital Holographic Optical Element.

In this study, a 50 × 50 mm holographic optical element (HOE) with the property of a spherical mirror was recorded digitally on a silver halide photoplate using a wavefront printing method. It consisted of 51 × 96 hologram spots with each spot measuring 0.98 × 0.52 mm. The wavefronts and optical performance of the HOE were compared with those of reconstructed images from a point hologram displayed on DMDs of different pixel structures. The same comparison was also performed with an analog-type HOE for a heads-up display and with a spherical mirror. A Shack-Hartmann wavefront sensor was used to measure the wavefronts of the diffracted beams from the digital HOE and the holograms as well as the reflected beam from the analog HOE and the mirror when a collimated beam was incident on them. These comparisons revealed that the digital HOE could perform as a spherical mirror, but they also revealed astigmatism-as in the reconstructed images from the holograms on DMDs-and that its focusability was worse than that of the analog HOE and the spherical mirror. A phase map, i.e., the polar coordinate-type presentation of the wavefront, could visualize the wavefront distortions more clearly than the reconstructed wavefronts obtained using Zernike polynomials. The phase map revealed that the wavefront of the digital HOE was more distorted than those of the analog HOE and the spherical mirror.

Visual Complexity of Head-Up Display in Automobiles Modulates Attentional Tunneling.

To investigate how the visual complexity of head-up displays (HUDs) influence the allocation of driver's attention in two separate visual domains (near and far domains). The types and amount of information displayed on automobile HUDs have increased. With limited human attention capacity, increased visual complexity in the near domain may lead to interference in the effective processing of information in the far domain. Near-domain and far-domain vision were separately tested using a dual-task paradigm. In a simulated road environment, 62 participants were to control the speed of the vehicle (SMT; near domain) and manually respond to probes (PDT; far domain) simultaneously. Five HUD complexity levels including a HUD-absent condition were presented block-wise. Near domain performance was not modulated by the HUD complexity levels. However, the far domain detection accuracies were impaired as the HUD complexity level increased, with greater accuracy differences observed between central and peripheral probes. Increased HUD visual complexity leads to a biased deployment of driver attention toward the central visual field. Therefore, the formulation of HUD designs must be preceded by an in-depth investigation of the dynamics of human cognition. To ensure driving safety, HUD designs should be rendered with minimal visual complexity by incorporating only essential information relevant to driving and removing driving-irrelevant or additional visual details.

Augmented-reality swim goggles accurately and reliably measure swim performance metrics in recreational swimmers.

Swimmers commonly access performance metrics such as lap splits, distance, and pacing information between work bouts while they rest. Recently, a new category of tracking devices for swimming was introduced with the FORM Smart Swim Goggles (FORM Goggles). The goggles have a built-in see-through display and are capable of tracking and displaying distance, time splits, stroke, and pace metrics in real time using machine learning and augmented reality through a heads-up display. The purpose of this study was to assess the validity and reliability of the FORM Goggles compared with video analysis for stroke type, pool length count, pool length time, stroke rate, and stroke count in recreational swimmers and triathletes. A total of 36 participants performed mixed swimming intervals in a 25-m pool across two identical 900-m swim sessions performed at comparable intensities with 1 week interval. The participants wore FORM Goggles during their swims, which detected the following five swim metrics: stroke type, pool length time, pool length count, stroke count, and stroke rate. Four video cameras were positioned on the pool edges to capture ground truth video footage, which was then manually labeled by three trained individuals. Mean (SD) differences between FORM Goggles and ground truth were calculated for the selected metrics for both sessions. The absolute mean difference and mean absolute percentage error were used to assess the differences of the FORM Goggles relative to ground truth. The test-retest reliability of the goggles was assessed using both relative and absolute reliability metrics. Compared with video analysis, the FORM Goggles identified the correct stroke type at a rate of 99.7% (N = 2,354 pool lengths, p < 0.001), pool length count accuracy of 99.8%, and mean differences (FORM Goggles-ground truth) for pool length time: -0.10 s (1.49); stroke count: -0.63 (1.82); and stroke rate: 0.19 strokes/min (3.23). The test-retest intra-class correlation coefficient (ICC) values between the two test days were 0.793 for pool length time, 0.797 for stroke count, and 0.883 for stroke rate. Overall, for pool length time, the residuals were within ±1.0s for 65.3% of the total pool lengths, for stroke count within ±1 stroke for 62.6% of the total pool lengths, and for stroke rate within ±2 strokes/min for 66.40% of the total pool lengths. The FORM Goggles were found valid and reliable for the tracking of pool length time, pool length count, stroke count, stroke rate, and stroke type during freestyle, backstroke, and breaststroke swimming in recreational swimmers and triathletes when compared with video analysis. This opens perspectives for receiving real-time information on performance metrics during swimming.

Implementation of flexible and customizable free-from mirror heads-up display

Implementation of flexible and customizable free-from mirror heads-up display

34‐2: A dual‐focal‐plane augmented reality head‐up display using polarization multiplexing

The combination of augmented reality (AR) and head‐up display (HUD) makes HUD more and more attractive in recent years. In this paper, we propose a dual‐focal‐plane windshield‐type AR‐HUD prototype using a single freeform mirror, then fold the light path by polarization‐related elements to realize a compact volume using a single picture generation unit (PGU). The designed system meets the design specifications with experimental verification.

34‐1: Invited Paper: Future AR‐HUD System

The automotive head‐up display (HUD) was introduced by General Motors in 1988 to prevent driver distraction. Augmented reality (AR) technology has made the HUD more attractive for ADAS visualization, with the ability to overlay driving and sensed information on the road. Early versions of AR‐HUD products have limited functionality due to conventional optic technology and a limited perception of the environment. Future AR‐HUD will require a large field of view, multiplane capability, real‐time sensor fusion and correction, and an embedded system implementation. This paper reviews the future enabling technologies and reference implementations for AR‐HUD.

THREE-DIMENSIONAL HEADS-UP VITRECTOMY VERSUS CONVENTIONAL MICROSCOPIC VITRECTOMY FOR PATIENTS WITH EPIRETINAL MEMBRANE.

To investigate the efficacy and safety of 3D heads-up display (3D-HUD) vitrectomy compared with conventional microscopy (CM) vitrectomy in epiretinal membrane (ERM) surgery. Epiretinal membrane removal with or without internal limiting membrane (ILM) peeling was performed using a 3D-HUD or CM system. The mean changes in best-corrected visual acuity (BCVA) and in central macular thickness (CMT) and postoperative complications were assessed. Baseline demographics were comparable except for the follow-up period. Both BCVA and CMT improved at the final visit (all P < 0.05). The ERM recurrence and dissociated optic nerve fiber layer (DONFL) rates were lower in the 3D group (both P < 0.05). conventional microscopic vitrectomy (odds ratio [OR] = 12.86, P = 0.02) and absence of ILM peeling (OR = 45.25, P < 0.05) were associated with ERM recurrence. In the DONFL, CM vitrectomy (OR = 1.98, <0.05) and combined phacovitrectomy (OR = 2.33, P = 0.03) were analyzed as risk factors for DONFL. The improvement in BCVA and CMT in ERM surgery using a 3D-HUD is comparable with that of CM vitrectomy, with a significantly low rate of ERM recurrence and DONFL occurrence. Therefore, 3D vitrectomy might have an advantage for ERM surgery.

Development and Evaluation of an Enhanced Virtual Reality Flight Simulation Tool for Airships

A real-time flight simulation tool is proposed using a virtual reality head-mounted display (VR-HMD) for remotely piloted airships operating in beyond-line-of-sight (BLOS) conditions. In particular, the VR-HMD was developed for stratospheric airships flying at low/high altitudes. The proposed flight simulation tool uses the corresponding aerodynamics characteristics of the airship, the buoyancy effect, mass balance, added mass, propulsion contributions and ground reactions in the FlightGear Flight Simulator (FGFS). The VR headset was connected to the FGFS along with the radio controller containing the real-time orientation/state of each button, which is also simulated to provide better situational awareness, and a head-up display (HUD) that was developed to provide the required flight data. In this work, a system was developed to connect the FGFS and the VR-capable graphics engine Unity to a PC and a wireless VR-HMD in real time with minimal lag between data transmission. A balance was found for FGFS to write to a CSV file at a period of 0.01 s. For Unity, the file was read every frame, which translates to around 0.0167 s (60 Hz). A test procedure was also conducted with a similar rating technique based on the NASA TLX questionnaire, which identifies the pilot’s available mental capacity when completing an assigned task to assure the comfortability of the proposed VR-HMD. Accordingly, a comparison was made for the aircraft control using the desktop simulator and the VR-HMD tool. The results showed that the current iteration of the system is ideal to train pilots on using similar systems in a safe and immersive environment. Furthermore, such an advanced portable system may even increase the situational awareness of pilots and allow them to complete a sizeable portion of actual flight tests with the same data transmission procedures in simulation. The VR-HMD flight simulator was also conceived to express the ground control station (GCS) concept and transmit flight information as well as the point of view (POV) visuals in real-time using the real environment broadcast using an onboard camera.

Investigation of Text-Background Lightness Combination on Visual Clarity Using a Head-Up Display under Various Surround Conditions and Different Age Groups

Up until now, head-up displays (HUDs) have been installed in front of the driver’s seat to provide drivers with auxiliary information. It may be used in various surroundings, ranging from very dark to very bright environments, such as daylight. A suitable text-background lightness combination can improve the driver’s visual clarity and efficiency when identifying displayed information to raise driving safety. Although many kinds of HUDs are designed to improve visual clarity by adjusting brightness, few studies have investigated the influence of the text-background lightness combination of a HUD on visibility, especially as the lighting level of the driving condition changes dramatically. In the study, 60 observers, comprising 20 young, 20 middle-aged, and 20 older participants, evaluated the visual clarity of 20 text-background lightness combinations on a HUD using paired comparison methods under dark and daylight surroundings (i.e., 300, 1500, and 9000 lx conditions). As a result, the combination of white text on a black background and black text on a white background presented the most significant preference and the best visual clarity under dark and daylight surroundings, respectively, improving visual safety when driving.

Microdisplays: Mini‐LED, Micro‐OLED, and Micro‐LED

AbstractThe field of next‐generation microdisplays is flourishing. Relevant display technologies, such as mini‐light emission diodes (mini‐LEDs), micro‐organic light emission diodes (micro‐OLEDs), and micro‐light emission diodes (micro‐LEDs) are thus in the urgent stage of development. From this perspective, comprehensive and systematical analyzes are conducted for the aforesaid microdisplay configurations. A holistic view of microdisplay technologies is developed with the corresponding performance metrics, providing a path for miscellaneous scenarios. Among these scenarios, the applications in augmented reality (AR), virtual reality (VR), wearable devices, and head‐up displays (HUD) are currently attracting considerable attention for deeper human‐digital interactions. However, there is a multiplicity of obstacles and challenges hindering such development. Nevertheless, recent advances in microdisplay technologies hold tremendous promise for the paradigms of these applications, taking a leap forward for next‐generation microdisplays. This review presents perspectives, relevant materials, and the technology landscape for such ongoing display technologies, offering guidance on the design of advanced microdisplays.

Advanced Study of Optical Imaging Systems for Virtual Reality Head-Mounted Displays

Driven by the rapid innovation of science and technology and industrial manufacturing technology, virtual reality display technology has developed rapidly. At present, the application of virtual reality display technology is expanding in many fields such as military, medical, aviation and education. This paper analyzes the imaging principle of the human vision system and the optical performance requirements of VR heads-up display, summarizes the current design scheme of VR heads-up optical imaging system, focuses on the principle and index parameters of each optical system, and compares the advantages and disadvantages of different schemes. The development prospects and directions of virtual reality headset displays are also prospected.

Effects of button colour and background on augmented reality interfaces

ABSTRACT This study aims to understand the usability of common button colours and background choices in the existing augmented reality (AR) interface. AR is increasingly used in dynamic situations, such as driving using an AR head-up display. However, studies regarding the usability of AR interfaces, especially about the colours, are scarce. Twenty-four participants performed button selection tasks in an experiment using Microsoft HoloLens. Three measures (task completion time, the number of errors, and button colour visibility were collected for the 12 experimental conditions (3 backgrounds by 4 button colours). Significant differences were found in completion time between backgrounds and blue button visibility. On a moving background, the most effective button colour is orange, and the most minor effective button colour is blue. The results of this study can be used for colour choices in designing AR applications and devices.

Flight Guidance Concepts to Mitigate Flight Control System Degradation in Urban Air Mobility Scenarios

Continued safe flight under all foreseeable scenarios will be crucial to assure airworthiness and build trust in future vertical takeoff and landing (VTOL) urban air mobility (UAM) concepts. Successful pilot intervention will be especially important for handling emergency procedures. This paper demonstrates the use of human-centric flight guidance concepts to mitigate the effects of non-catastrophic failures in future VTOL UAM aircraft. Specifically, a single failure in the lateral steering function of an automatic flight control system is considered. A new waypoint guidance law is proposed to generate the required roll command to manually steer the aircraft toward a predefined safe landing zone. Two display concepts are proposed to aid the manual execution of the roll steering command: a head-down display (HDD), and a head-mounted display (HMD). The guidance law and the display concepts were integrated into a high-fidelity, wide field-of-view rotorcraft simulation environment. Human-in-the-loop experiments were performed with non-pilot test subjects. The effectiveness of the guidance law and display concepts along with the associated piloting performance and human factor impacts were assessed. The results indicated that all test subjects were able to reach a close vicinity of the final waypoint. The HMD induced lower workload whereas the HDD resulted in better tracking performance.

Preview the Products on Display at Display Week 2023

Preview the Products on Display at Display Week 2023

Applying multi-focus filtering plane in double-step Fresnel diffraction method for enhanced holographic projection region

Computer-generated holography has been anticipated in augmented reality (AR) since it can fully provide multi-depth 3D information to users, but speckle noises are often accompanied as one of the most significant degradation factors. Since complex amplitude encoding can suppress speckle noises, applying double-phase encoding to a lensless holographic projection system using a double-step Fresnel diffraction (DSF) algorithm has been reported to reduce those noises. However, a single filtering focus at a virtual focal plane might significantly reduce the projection region compared to the maximum diffraction bandwidth of the spatial light modulator. In this paper, we applied multiple foci for the focal plane of the DSF method with off-axis phase encoding, which can expand the projection region and is applicable to display multi-depth images with DSF. Numerical analysis conducted for image quality compared with conventional single focus DSF algorithm shows that the speckle performance does not degrade so much as the number of foci increase. We expect the proposed method to be useful for holographic projection optical systems such as AR head-up display.

Design of head-up display interfaces for automated vehicles

This study aimed to identify which information should be displayed on a head-up display (HUD) in semi-automated vehicles to enable the driver to maintain better situational awareness during the manual operation of the vehicle. It further explored how does the size of the field of view of the HUD affects such information presentation, while taking into consideration user's personal preferences and opinions. Four head-up display interfaces were developed differing in the amount, frequency and field of view of visual information presentation, and were implemented in a simulated semi-automated vehicle. The HUDs were evaluated in a user-study with a within-subject design with 30 participants. The obtained results revealed that versions with smaller and larger field of view HUDsevoke similar driving performance, levels of cognitive load, user experience and perceived usability, suggesting that the HUD's field of view size does not have an overall significant effect on the driver's situational awareness. The results reveal that display of information, which can help with obtaining and maintaining higher situational awareness levels, contribute to better driving performance. Drivers on the other hand, prefer also display of information for lower situational awareness levels.

Assessing the Impact of AR HUDs and Risk Level on User Experience in Self-Driving Cars: Results from a Realistic Driving Simulation

The adoption of self-driving technologies requires addressing public concerns about their reliability and trustworthiness. To understand how user experience in self-driving vehicles is influenced by the level of risk and head-up display (HUD) information, using virtual reality (VR) and a motion simulator, we simulated risky situations including accidents with HUD information provided under different conditions. The findings revealed how HUD information related to the immediate environment and the accident’s severity influenced the user experience (UX). Further, we investigated galvanic skin response (GSR) and self-reported emotion (Valence and Arousal) annotation data and analyzed correlations between them. The results indicate significant differences and correlations between GSR data and self-reported annotation data depending on the level of risk and whether or not information was provisioned through HUD. Hence, VR simulations combined with motion platforms can be used to observe the UX (trust, perceived safety, situation awareness, immersion and presence, and reaction to events) of self-driving vehicles while controlling the road conditions such as risky situations. Our results indicate that HUD information provision significantly increases trust and situation awareness of the users, thus improving the user experience in self-driving vehicles.

53.2: Invited Paper: Evaluation of Optical Performance and Image Quality for Aviation Head‐up Displays (HUDs) Applied Under High Ambient Light

The HUD (Head‐Up Display) is an optical display system that is commonly used in aircraft as a flight aid to project aircraft flight information onto the display screen (often a type of transparent flat lens) directly in front of the pilot's field of view. This allows the pilot to maintain a level view while taking into account instrument parameters and external visual references in the same field of view. HUDs allow pilots to see crucial navigation, flying, and energy management data on a screen at eye level without having to glance down at their instruments or take their eyes off the outside world. It has been demonstrated that HUDs lessen the pilot workload, improve situational awareness, and decrease accidents. Thus, it is crucial to characterize optical performance and image quality accurately and precisely for aircraft HUDs. In this paper, we reviewed the parameters of optical performance and image quality for aviation‐type HUDs applied under high ambient light levels.