Head-up Display System Research Articles

Design of a dual-focal-plane head-up display system using double freeform mirrors

This paper presents the design of a dual-focal-plane head-up display (HUD) system utilizing double freeform mirrors. The system is specifically developed for automotive applications, aiming to enhance driving safety and experience by projecting vehicle information within the driver’s line of sight. Compared to the traditional WHUD, the ARHUD offers a broader field of view and a greater virtual image distance. If the ARHUD system has double focal planes, it can separately display basic and interactive driving information to drivers, which is currently a significant research direction in the ARHUD system design. The proposed design only uses double freeform mirrors and uses a single display source for both focal planes, thus reducing the overall system complexity, assembly difficulty, and cost. Detailed design methods and results are discussed, showcasing the system’s good imaging performance and compact structure, which provide valuable insights for the development of next-generation dual-focal-plane HUDs for vehicles. The proposed system demonstrates significant advancements in reducing fabrication and testing difficulties compared to traditional dual-focal-plane HUD designs. The study’s findings contribute to the future development of dual-focal-plane head-up display systems.

Phase space framework enables a variable-scale diffraction model for coherent imaging and display

The fast algorithms in Fourier optics have invigorated multifunctional device design and advanced imaging technologies. However, the necessity for fast computations limits the widely used conventional Fourier methods, where the image plane has a fixed size at certain diffraction distances. These limitations pose challenges in intricate scaling transformations, 3D reconstructions, and full-color displays. Currently, the lack of effective solutions makes people often resort to pre-processing that compromises fidelity. In this paper, leveraging a higher-dimensional phase space method, a universal framework is proposed for customized diffraction calculation methods. Within this framework, a variable-scale diffraction computation model is established for adjusting the size of the image plane and can be operated by fast algorithms. The model’s robust variable-scale capabilities and its aberration automatic correction capability are validated for full-color holography, and high fidelity is achieved. The tomography experiments demonstrate that this model provides a superior solution for holographic 3D reconstruction. In addition, this model is applied to achieve full-color metasurface holography with near-zero crosstalk, showcasing its versatile applicability at nanoscale. Our model presents significant prospects for applications in the optics community, such as beam shaping, computer-generated holograms (CGHs), augmented reality (AR), metasurface optical elements (MOEs), and advanced holographic head-up display (HUD) systems.

Design of a zoom head-up display optical system based on a nodal aberration theory

We propose a zoom dual-path augmented reality head-up display optical system. The system uses two image generation units to display the basic and interactive information. Based on the node aberration theory, the aberration evaluation function was constructed. By solving the optimal solution of the aberration evaluation function, the initial structure of the interactive information display optical path was obtained. In order to solve the vergence-accommodation conflict in the process of vehicle speed change, the interactive information optical path was designed as a zoom optical path. The zoom distance is 10–20 m, and the basic information optical path will not be interfered with in the process. In addition, the tolerance analysis of the system was carried out to prove the machinability and stability of the system. This study makes the imaging of the augmented reality head-up display system more in line with the characteristics of the human eye and improves driving comfort.

Design and Optimization of a Head-Up Display System with Variable Projection Distance

Design and Optimization of a Head-Up Display System with Variable Projection Distance

Low complexity local dimming algorithm for high quality head up displays in automotive vehicles

We present a high image quality head up display (HUD) based on the local dimming liquid crystal display (LCD). The proposed HUD provides high white luminance to improve day-time visibility and low black luminance to cope with the postcard effect during the night by the local dimming technique. To reduce the computational complexity, the pixel-based boosting values for the pixel compensation are computed by simple bi-linear interpolation, complex power functions are implemented by look-up tables, some multiplications are replaced with the sums of shifted values, and the numbers of look-up tables and multipliers are reduced by sequential process. The simulation results of the proposed local dimming algorithm for 640×1280 LCD panel and 10×22 LED backlight ensure higher peak signal-to-noise ratio than 40 dB and larger power reduction performance than 73% over 7 HUD images. In addition, the postcard effect improvement is verified over images on LCD panel as well as windshield. Maximum luminance and contrast ratio on the windshield of the prototype HUD system are measured as 16,088 cd/m 2 and 342 × 10 6 :1.

Initial Structure Design and Optimization of an Automotive Remote Head-Up Display

Aiming at the problems of difficult construction and occlusion of the initial structure of an automotive augmented reality head-up display (AR-HUD), a method to quickly build the initial structure of an automotive AR-HUD is proposed. Firstly, the position of the mirrors in the initial structure is calculated based on the Rodrigues rotation formula. Secondly, the position of the mirrors is restricted by constraints during optimization to prevent the problem of structural occlusion. Finally, a virtual image display system with a visual distance of 7.5 m and a field of view angle of 10° × 5° is designed. The image quality analysis of the optimized system shows that the light spots in each field of view are all within Airy spots. At the spatial cutoff frequency of the virtual image plane of the optical system, the modulation transfer function (MTF) value of the full field of view is basically greater than 0.5, and the distortion is less than 1%. Finally, using the image for simulation, the results of the simulation image are satisfying, which proves the validity and feasibility of the structural design. It provides a useful reference for the structural design of the remote head-up display system.

Automated design of freeform imaging systems for automotive heads-up display applications.

The freeform imaging system is playing a significant role in developing an optical system for the automotive heads-up display (HUD), which is a typical application of augmented reality (AR) technology. There exists a strong necessity to develop automated design algorithms for automotive HUDs due to its high complexity of multi-configuration caused by movable eyeballs as well as various drivers' heights, correcting additional aberrations introduced by the windshield, variable structure constraints originated from automobile types, which, however, is lacking in current research community. In this paper, we propose an automated design method for the automotive AR-HUD optical systems with two freeform surfaces as well as an arbitrary type of windshield. With optical specifications of sagittal and tangential focal lengths, and required structure constraints, our given design method can generate initial structures with different optical structures with high image quality automatically for adjusting the mechanical constructions of different types of cars. And then the final system can be realized by our proposed iterative optimization algorithms with superior performances due to the extraordinary starting point. We first present the design of a common two-mirror HUD system with longitudinal and lateral structures with high optical performances. Moreover, several typical double mirror off-axis layouts for HUDs were analyzed from the aspects of imaging performances and volumes. The most suitable layout scheme for a future two-mirror HUD is selected. The optical performance of all the proposed AR-HUD designs for an eye-box of 130 mm × 50 mm and a field of view of 13° × 5° is superior, demonstrating the feasibility and superiority of the proposed design framework. The flexibility of the proposed work for generating different optical configurations can largely reduce the efforts for the HUD design of different automotive types.

Wide field of view flat-panel head-up display system

We propose a novel flat panel-based head-up display (HUD) to realize a large viewing angle automotive display. The proposed flat-panel geometry is straightforward to integrate into the dashboard. We also provide a flat-panel compatible solution to eliminate the ghost image and stray-light in large field of view using space-variant polarization sheets and optimized directional filtering, respectively. The proposed system provides an image area that is about 14 times larger compared to commercial HUD, and the P-polarized virtual image is also viewable with polarized sunglasses. The virtual image distance in our experimental system is closer than in commercial HUD.

Comparison of the Safety and Efficacy of a 3-Dimensional Heads-up Display vs a Standard Operating Microscope in Retinal Detachment Repair.

Purpose: To evaluate the safety, efficacy, and efficiency of the Ngenuity 3-dimensional (3D) heads-up display (HUD) visualization system for primary rhegmatogenous retinal detachment (RRD) repair at a large academic medical center in the United States. Methods: This retrospective review comprised consecutive patients aged 18 years or older who had primary RRD repair (pars plana vitrectomy [PPV] alone or combined PPV and scleral buckle) performed by the same fellowship-trained vitreoretinal surgeon using the 3D visualization system and a traditional standard operating microscope (SOM) at Massachusetts Eye and Ear from June 2017 to December 2021. The minimum follow-up was 90 days. Results: The 3D HUD group comprised 50 eyes of 47 patients and the SOM group, 138 eyes of 136 patients. There were no between-group differences in single surgery anatomic success rates at 3 months (98% HUD vs 99% SOM; P = 1.00) or at the last follow-up (94% HUD vs 98% SOM; P = .40). The rate of postoperative proliferative vitreoretinopathy was similar between the 2 groups (3 months: 3% HUD vs 5% SOM, P = .94; last follow-up, 2% HUD vs 3% SOM, P = .93). There was no difference in the mean duration of surgery (57.4 ± 28.9 minutes HUD vs 59.4 ± 29.9 minutes SOM; P = .68). Conclusions: Anatomic and functional outcomes, in addition to surgical efficiency, of noncomplex primary RRD repair with a 3D HUD system were similar to those of surgery performed with an SOM.

A Visual Reasoning-based AR-HUD Service Design Approach for Better Driving Experience

Smart traffic or transportation, as a critical part of realizing the smart city, utilizes various smart product-service systems to support it. Among them, the augmented reality head-up display (AR-HUD) system is a typical smart driving solution, which projects vital information about driving situations with pre-defined AR graphics onto the windshield of vehicles. The AR-HUD service design attracts increasing concerns owing to its enhancement of drivers’ situational awareness without glancing down at the instrument cluster. Nevertheless, current in-car AR-HUDs only stack some simple driving information from sensors, such as the driving speed. It's difficult to meet the demand of drivers since they ignore a higher level of drivers’ situation awareness, i.e., driving situation prediction and recommendation of driving advice. To overcome the challenge, this paper proposes a visual reasoning-based service design approach for advancing AR-HUDs’ digital servitization and better user experience. Additionally, drivers are involved in the service design development in a value co-creation manner through human-computer interaction design. With the combination of experiential driving knowledge and cognitive intelligence computing on the driving scene, the proposed novel service design approach enables AR-HUD to percept driving scenarios and infer good strategies accordingly. Finally, an illustrative example is carried out to validate the feasibility of the proposed AR-HUD service design approach.

Augmented reality system based on full-color holographic optical elements lens

Holographic optical element (HOE) lens is an imaging element fabricated through recording wavefront by interference. Because of its advantages of small form factor and wavelength, angle selectivity and arbitrary wavefront formation, it has a good application prospect in augmented reality display. To make the system more compact, the HOE lens is adopted as an off-axis optical element. At the same time, according to diffraction principle, its wavelength response is more sensitive than those of traditional refractive and reflective optical elements. Thus the fabrication and design of a full-color HOE lens is a challenge to optimizing the free-space head-up display system. To systematically analyze the HOE imaging system, the conjugate relation between the object and image is derived by scalar diffraction theory. Then the Gaussian conjugate imaging equation is obtained and the off-axis aberration of distortion and astigmatism in the HOE imaging system are analyzed. In addition, A head-up display with field of view (FOV) of 18° and eyebox of 10 mm is simulated and its imaging process is visualized through the geometric optics method of <i> <b>k</b> </i>-vector diagram and ray-tracing. A full-color HOE lens with high diffraction efficiency is fabricated by interference. Its average peak diffraction efficiency is 56.7%, reaching a high level in the world. A prototype of augmented reality system is established by integrating laser pico-projectior with HOE lens. The experimental results of distortion effect and astigmatism effect of the system are obtained, which are consistent with the simulation results. The modulation transfer function (MTF) parameter of the system is measured, and its definition basically meets the requirements of the human eyes for resolution. The aberration of the system is analyzed and the optimization method is proposed. To optimize the monochromatic image quality, an extra cylindrical lens is added to ensure the same optical power of meridian and sagittal plane to eliminate the astigmatism. Besides, a freeform wavefront is designed by the geometric construction method and forms a freeform HOE to deal with the distortion problem. The local recording freeform wavefront can be calculated by the imaging equation. When full-color HOE is applied to the display system, the images of three channels may separate in the space because of their different reconstruction wavelengths and angles. We propose a pre-compensation method of recording process to solve this problem. If these above-mentioned problems can be solved, due to its good image uniformity, sufficient field angle and eyebox area, the head-up display based on HOE lens with extra optical power will have a better application in augmented reality technology.

Multiplane holographic augmented reality head-up display with a real-virtual dual mode and large eyebox.

We propose a multiplane augmented reality (AR) head-up display (HUD) with a real-virtual dual mode based on holographic optical elements (HOEs). The picture generation unit (PGU) is only a single free-focus projector, and the optical combiner includes a HOE lens (HOEL) for long-distance virtual image display and a HOE diffuser (HOED) for in-plane real image display. A HOED with directional scattering characteristics in the real image mode can significantly increase the size of the eyebox (EB) without increasing the size of the HOE, and a HOEL with a flexible design for the optical focal length in the virtual image mode can be used to achieve a different depth of the AR display. The proposed AR HUD system, which has a compact structure and offers high light transmittance, high energy usage, a multiplane display, and a large EB, is expected to be widely used in the future.

Design of off-axis reflective imaging systems based on freeform holographic elements.

Holographic optical element (HOE) can be used in many areas in optics due to its characteristics of thin structure, flexible wavefront reconstruction/control ability and angular/wavelength selectivity. In this paper, we propose a design method of off-axis reflective imaging systems based on freeform HOEs, which are fabricated by freeform wavefronts. The freeform HOEs offer many degrees of design freedom and can correct the aberrations in nonsymmetric imaging systems. The initial imaging system with freeform HOEs is generated using a point-by-point design approach, and is used for the preliminary design of the imaging system and the freeform recording system of each HOE. Then a joint optimization is conducted for all the systems, simultaneously considering the imaging performance, the diffraction efficiency, the system constraints and fabrication to get the final design results. To validate the feasibility and effectiveness of the proposed method, an off-axis reflective head-up display system with good performance based on freeform HOEs is designed and fabricated. Detailed procedures of the design and development process of the prototype are demonstrated.

Current status of augmented reality in cerebrovascular surgery: a systematic review.

Augmented reality (AR) is an adjuvant tool in neuronavigation to improve spatial and anatomic understanding. The present review aims to describe the current status of intraoperative AR for the treatment of cerebrovascular pathology. A systematic review was conducted following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. The following databases were searched: PubMed, Science Direct, Web of Science, and EMBASE up to December, 2020. The search strategy consisted of "augmented reality," "AR," "cerebrovascular," "navigation," "neurovascular," "neurosurgery," and "endovascular" in both AND and OR combinations. Studies included were original research articles with intraoperative application. The manuscripts were thoroughly examined for study design, outcomes, and results. Sixteen studies were identified describing the use of intraoperative AR in the treatment of cerebrovascular pathology. A total of 172 patients were treated for 190 cerebrovascular lesions using intraoperative AR. The most common treated pathology was intracranial aneurysms. Most studies were cases and there was only a case-control study. A head-up display system in the microscope was the most common AR display. AR was found to be useful for tailoring the craniotomy, dura opening, and proper identification of donor and recipient vessels in vascular bypass. Most AR systems were unable to account for tissue deformation. This systematic review suggests that intraoperative AR is becoming a promising and feasible adjunct in the treatment of cerebrovascular pathology. It has been found to be a useful tool in the preoperative planning and intraoperative guidance. However, its clinical benefits remain to be seen.

A Multi-Plane Augmented Reality Head-Up Display System Based on Volume Holographic Optical Elements With Large Area

The traditional head-up display (HUD) system has the disadvantages of a small area and a single display plane, here we propose and design an augmented reality (AR) HUD system with multi-plane, large area, high diffraction efficiency and a single picture generation unit (PGU) based on holographic optical elements (HOEs). Since volume HOEs have excellent angle selectivity and wavelength selectivity, HOEs of different wavelengths can be designed to display images in different planes. Experimental and simulated results verify the feasibility of this method. Experimental results show that the diffraction efficiencies of the red, green and blue HOEs are 75.2%, 73.1% and 67.5%. And the size of HOEs is 20 cm × 15 cm. Moreover, the three HOEs of red, green and blue display images at different depths of 150 cm, 500 cm and 1000 cm, respectively. In addition, the field of view (FOV) and eye-box (EB) of the system are 12° × 10° and 9.5 cm × 11.2 cm. Furthermore, the light transmittance of the system has reached 60%. It is believed that this technique can be applied to the augmented reality navigation display of vehicles and aviation.

Lux study: Contribution of a three-dimensional, high dynamic range, ultra-high-definition heads-up visualization system to a significant delivered light intensity decrease during different types of ocular surgeries

To explore the hypothesis that using a large, three-dimensional (3D), ultra-high-definition (4K), heads-up display (HUD) system in a real-life setting may be associated with a significant reduction in light intensity (LI) delivered during various types of eye surgery. Single center, post-learning curve observational study of 142 independent consecutive cases: 73 cataracts and 69 vitrectomies (VR). For each group, the only variable setting was the LI. The LI delivered by each source was calibrated in lumens (lm). In the VR group, the delivered LI dramatically decreased from the 80% reference to 27.8%±13.2% (P<0.0001) (4.3 to 1.5lm). Among these surgeries, 91.3% needed 40% or less LI (2.3lm). The corneal transparency and quality of pupil dilation showed some correlation with the results. In the cataract group, the difference was even more spectacular, from 80% to 15%±11.3% (P<0.0001) (2.5 to 0.3lm). Among these surgeries, 80.8% required less than 20% LI (0.5lm). The surgical time, LI variations and type of cataract seemed to influence the results, down to 9.6%±5.4% (P<0.0001) (0.22lm) for nuclear cataracts, which represented 63% of the cataract sample. The digital signal amplification with this HUD system allows comfortable posterior as well as anterior segment surgery despite the decrease in incident light, suggesting an unprecedented reduction in the risk of phototoxicity compared to conventional systems as well as a likely improvement in patient comfort.

Achromatic suppression of ghost images in head-up displays

We examine the chromatic properties of ghost images appearing in expanded viewing angle polarization-type head-up display systems. We propose an achromatic polarization correction method to suppress ghost images in the whole visible spectral range. The method is based on the use of two half-wave plates of orthogonal optical axis–one being laminated on the combiner, the other attached to the head-up display (HUD) projector. We show that in this configuration the dispersion of the two waveplates cancel each other. Based on the results, a ghost image free colour HUD is demonstrated with 13 ° × 40 ° FOV. We present our modeling and experimental results and our demonstration HUD system.

Low-Complexity Pupil Tracking for Sunglasses-Wearing Faces for Glasses-Free 3D HUDs

This study proposes a pupil-tracking method applicable to drivers both with and without sunglasses on, which has greater compatibility with augmented reality (AR) three-dimensional (3D) head-up displays (HUDs). Performing real-time pupil localization and tracking is complicated by drivers wearing facial accessories such as masks, caps, or sunglasses. The proposed method fulfills two key requirements: low complexity and algorithm performance. Our system assesses both bare and sunglasses-wearing faces by first classifying images according to these modes and then assigning the appropriate eye tracker. For bare faces with unobstructed eyes, we applied our previous regression-algorithm-based method that uses scale-invariant feature transform features. For eyes occluded by sunglasses, we propose an eye position estimation method: our eye tracker uses nonoccluded face area tracking and a supervised regression-based pupil position estimation method to locate pupil centers. Experiments showed that the proposed method achieved high accuracy and speed, with a precision error of &lt;10 mm in &lt;5 ms for bare and sunglasses-wearing faces for both a 2.5 GHz CPU and a commercial 2.0 GHz CPU vehicle-embedded system. Coupled with its performance, the low CPU consumption (10%) demonstrated by the proposed algorithm highlights its promise for implementation in AR 3D HUD systems.

Employing Emerging Technologies to Develop and Evaluate In-Vehicle Intelligent Systems for Driver Support: Infotainment AR HUD Case Study

The plurality of current infotainment devices within the in-vehicle space produces an unprecedented volume of incoming data that overwhelm the typical driver, leading to higher collision probability. This work presents an investigation to an alternative option which aims to manage the incoming information while offering an uncluttered and timely manner of presenting and interacting with the incoming data safely. The latter is achieved through the use of an augmented reality (AR) head-up display (HUD) system, which projects the information within the driver’s field of view. An uncluttered gesture recognition interface provides the interaction with the AR visuals. For the assessment of the system’s effectiveness, we developed a full-scale virtual reality driving simulator which immerses the drivers in challenging, collision-prone, scenarios. The scenarios unfold within a digital twin model of the surrounding motorways of the city of Glasgow. The proposed system was evaluated in contrast to a typical head-down display (HDD) interface system by 30 users, showing promising results that are discussed in detail.

Notice of Violation of IEEE Publication Principles: Design and Performance of an Off-Axis Free-Form Mirror for a Rear Mounted Augmented-Reality Head-Up Display System

A novel optical-transmission rear mounted augmented reality head-up display, which can be used to improve the safety of a driving vehicle, is introduced and investigated in this paper. The proposed system uses an off-axis free-form mirror that serves as asymmetric aberration corrector. The off-axis aberration of the system is analyzed and the mirror is designed to correct both linear astigmatism and spherical aberration. This makes it possible to optimize the whole optical system. We also analyze the relationship between human eyes and the virtual image to obtain a registration matrix. The experimental results show that the size of the virtual image is 11 inches, the imaging distance is 2.8 meters, and the area of the eye box is 140 * 60 mm2. The novel system makes it possible to fuse the virtual image with the real-life scene to improve driver safety.