Augmented Reality Near-eye Display Research Articles

Design of Compact Dielectric Metalens Visor for Augmented Reality Using Spin-Dependent Supercells

Augmented reality overlays computer-generated virtual information onto real-world scenes, enhancing user interaction and perception. However, traditional augmented reality optical systems are usually large, bulky, and have limited optical performance. In this paper, we propose a novel compact monochrome reflective dielectric metalens visor with see-through properties, engineered using a periodic structure of spin-dependent supercells. The supercell, which is composed of staggered twin nanofins, provides spin-dependent destructive or constructive interference with different circularly polarized incidences. The design combines the principles of interference with the Pancharatnam–Berry phase to enhance reflection at a working wavelength of 650 nm while maintaining good transmission. Right circularly polarized light incident from the substrate side causes destructive interference, enabling the supercell to work in reflection mode, while left circularly polarized light causes constructive interference, enabling the supercell to work in transmission mode. Furthermore, the supercell-constructed metalens can achieve near-diffraction-limited reflective focusing and a broad diagonal field of view of approximately 96°. In addition, compared to transmissive metalens visors, the reflective design eliminates the need for a beam splitter, significantly reducing the size and weight of the system. Our work could facilitate the development of compact and lightweight imaging systems and provide valuable insights for augmented reality near-eye display applications.

Design and fabrication method of holographic waveguide near-eye display with 2D eye box expansion.

Augmented reality near-eye display (AR-NED) technology has attracted enormous interests for its widespread potential applications. In this paper, two-dimensional (2D) holographic waveguide integrated simulation design and analysis, holographic optical elements (HOEs) exposure fabrication, prototype performance evaluation and imaging analysis are completed. In the system design, a 2D holographic waveguide AR-NED integrated with a miniature projection optical system is presented to achieve a larger 2D eye box expansion (EBE). A design method for controlling the luminance uniformity of 2D-EPE holographic waveguide by dividing the two thicknesses of HOEs is proposed, which is easy to fabricate. The optical principle and design method of the HOE-based 2D-EBE holographic waveguide are described in detail. In the system fabrication, laser exposure fabrication method of eliminating stray light for HOEs is proposed, and a prototype system is fabricated and demonstrated. The properties of the fabricated HOEs and the prototype are analyzed in detail. The experimental results verified that the 2D-EBE holographic waveguide has a diagonal field of view (FOV) of 45°, an ultra-thin thickness of 1 mm, and an eye box of 16 mm × 13 mm at an eye relief (ERF) of 18 mm, the MTF values of different FOVs at different 2D-EPE positions can be better than 0.2 at 20 lp/mm, and the whole luminance uniformity is 58%.

Sparse holographic imaging for an integrated augmented reality near-eye display.

Diffraction is the main physical effect involved in the imaging process of holographic displays. In the application of near-eye displays, it generates physical limits that constrain the field of view of the devices. In this contribution, we evaluate experimentally an alternative approach for a holographic display based mainly on refraction. This unconventional imaging process, based on sparse aperture imaging, could lead to integrated near-eye displays through retinal projection, with a larger field of view. We introduce for this evaluation an in-house holographic printer that allows the recording of holographic pixel distributions at a microscopic scale. We show how these microholograms can encode angular information that overcomes the diffraction limit and could alleviate the space bandwidth constraint usually associated with conventional display design.

Compact full-color augmented reality near-eye display using freeform optics and a holographic optical combiner.

We develop a compact full-color augmented reality near-eye display system with a multicolor holographic optical combiner and a freeform relay system. The digital image is produced by a full-color micro organic light-emitting diode (Micro-OLED) display module. The freeform relay system includes four freeform optics and a holographic optical mirror, which are employed to correct both the monochromatic and chromatic aberrations caused by the holographic optical combiner. The two multicolor holographic mirrors have a three-layer laminated structure and are delicately fabricated to yield an improved diffractive efficiency and a reduced efficiency difference for red, green, and blue colors. The high degrees of freedom of freeform optics, and the thin and light nature of the holographic optical combiner yield a compact form factor near-eye display system with a diagonal field of view (FOV) of 20° and the eye-box of 5 mm × 5 mm. Two prototypes are built to demonstrate the feasibility of the proposed display system.

Chromatic aberration correction in bi-focal augmented reality display by the multi-layer Pancharatnam-Berry phase lens.

Chromatic aberration is a main obstacle for the commercial application of augmented reality displays. The current digital and optical compensation methods of reducing the chromatic aberration suffer from processing time, power consumption or complex design. Here, a simple strategy of chromatic aberration correction in bi-focal augmented reality near-eye display based on multi-layer Pancharatnam-Berry phase lens has been demonstrated and verified by experimental results. The multi-layer Pancharatnam-Berry phase lens, as a part of optical combiner, is fabricated by three liquid crystal polymer phase lenses with central wavelength in red, green, and blue, respectively. The multi-layer Pancharatnam-Berry phase lens can effectively reduce the chromatic aberration in both convex and concave mode of bi-focal augmented reality system, where the color breakup of virtual images captured in bi-focal augmented reality display is significantly alleviated. Comparing to the value of ΔK = 1.3 m-1 in single green Pancharatnam-Berry phase lens, the multi-layer Pancharatnam-Berry phase lens system significantly reduce the ΔK to 0.45 m-1 with reduction of 65.4%, which finally decreases the longitudinal chromatic aberration and improve the quality of images. The proposed broadband multi-layer Pancharatnam-Berry phase lens can benefit augmented reality displays and find widespread application in the near-eye displays.

Challenges and Advancements for AR Optical See-Through Near-Eye Displays: A Review

Optical see-through near-eye display (NED) technologies for augmented reality (AR) have achieved significant advancements recently with investments from both academia and industry. Although various AR NED products have been successfully commercialized and even deployed into applications, there are still challenges with present AR NED technologies (e.g., limited eyebox, fixed focus, bulky form factors). In this review, we present a brief overview of leading AR NED technologies and then focus on the state-of-the-art research works to counter the respective key challenges with each of the leading AR NED technologies. We also introduce a number of emerging technologies that are worthy of close study.

Augmented Reality Near-Eye Display System Based on Cylindrical Holographic Waveguide

Augmented Reality Near-Eye Display System Based on Cylindrical Holographic Waveguide

Conjugate wavefront encoding: an efficient eyebox extension approach for holographic Maxwellian near-eye display.

Conventional holographic display suffers from the conjugate light issue. In this Letter, we propose to efficiently extend the eyebox of holographic Maxwellian near-eye display by encoding the conjugate wavefront as the multiplication of plane wave phase with the target image. It is interesting that after being focused by the lens, the generated conjugate viewpoints also present erect virtual images with the same image quality as the signal viewpoints. Multiple plane wave encoding is used for eyebox extension, and, because of the utilization of conjugate light, the effect of eyebox extension is doubled. That is, the space bandwidth of the amplitude-type hologram is fully used. A speckless holographic image is produced in mid-air with high quality within a large depth range. The proposed display is compact and promising for the augmented reality near-eye display. Furthermore, it may inspire better solutions for the conjugate light issue of amplitude-type holography.

Lensless full-color holographic Maxwellian near-eye display with a horizontal eyebox expansion.

A lensless full-color holographic Maxwellian near-eye display using a single amplitude-type spatial light modulator is proposed in this Letter. The color holographic image is directly projected onto the retina without any eyepiece. The color crosstalk is clearly separated from the signal in the space owing to the encoded spherical wave and carrier wave. An aperture numerical filter and a real polarized filter are used at the pupil plane to accurately stop the crosstalk light. A high-quality dynamic speckless color holographic image was produced in the mid-air within a specific depth range. The horizontal eyebox expansion is achieved simply through multiple spherical wave encoding and verified through an optical experiment. The proposed display is compact and promising as the augmented reality near-eye display.

Generating starting points for designing freeform imaging optical systems based on deep learning.

Deep learning is an important aspect of artificial intelligence and has been applied successfully in many optics-related fields. This paper proposes a generalized framework for generation of starting points for freeform imaging optical design based on deep learning. Compared with our previous work, this framework can be used for highly nonrotationally symmetric freeform refractive, reflective, and catadioptric systems. The system parameters can be advanced and the ranges of these system parameters can be wide. Using a special system evolution method and a K-nearest neighbor method, a full dataset consisting of the primary and secondary parts can be generated automatically. The deep neural network can then be trained in a supervised manner and can be used to generate good starting points directly. The convenience and feasibility of the proposed framework are demonstrated by designing a freeform off-axis three-mirror imaging system, a freeform off-axis four-mirror afocal telescope, and a freeform prism for an augmented reality near-eye display. The design framework reduces the designer's time and effort significantly and their dependence on advanced design skills. The framework can also be integrated into optical design software and cloud servers for the convenience of more designers.

Aberration-corrected full-color holographic augmented reality near-eye display using a Pancharatnam-Berry phase lens.

We present a full-color holographic augmented reality near-eye display using a Pancharatnam-Berry phase lens (PBP lens) and its aberration correction method. Monochromatic and chromatic aberrations of the PBP lens are corrected by utilizing complex wavefront modulation of the holographic display. A hologram calculation method incorporating the phase profile of the PBP lens is proposed to correct the monochromatic aberration. Moreover, the chromatic aberration is corrected by warping the image using the mapping function obtained from ray tracing. The proposed system is demonstrated with the benchtop prototype, and the experimental results show that the proposed system offers 50° field of view full-color holographic images without optical aberrations.

Holographically Printed Freeform Mirror Array for Augmented Reality Near-Eye Display

We propose augmented reality (AR) near-eye display using a holographically printed freeform mirror array. A wide depth of field AR system is implemented by the retinal projection via a small mirror array. Since we use a holographic mirror array, more advantages over a traditional physical mirror array are achieved. Due to the see-through characteristics of the holographic optical element (HOE), the array structure does not obstruct the sight of an observer. Also, the holographic printing technique for the freeform mirror array enables a wide depth of field system without using a floating lens and compensates for the astigmatism of the HOE. The detailed design method of the freeform mirror array HOE is presented. Through the retinal image simulations and the experimental results, the feasibility of the proposed method is verified.

Effect of surface roughness of optical waveguide on imaging quality and a formula of RSE tolerance and incident angle.

The optical waveguide is a lightweight and portable scheme for augmented reality near-eye display devices. However, the surface roughness of the waveguide affects its imaging performance, which has not been studied. In this work, we investigate the light scattering caused by the root-mean-square roughness of the waveguide surface and present two methods to numerically analyze the modulation transfer function (MTF) of the display system. Here, we consider the effects of different surface roughness, incident angle, and incident wavelength on the scattering distribution when other conditions are constant. For a simplified optical waveguide display system, the MTF degradation and the variation of the tolerance is calculated. And when the MTF (@ 40 cycles/mm) is required to be 0.3 and the incident angles of the total reflection surface are 45°, 55°, 65° and 75°, the random surface error (RSE) tolerances are 0.207λ0, 0.255λ0, 0.347λ0 and 0.566λ0 (λ0=0.5461µm), respectively. We find a formula descripting the relationship between RSE tolerance and incident angle. If the RSE tolerance exceeds the value of the formula at an angle, the imaging quality of the system will drop significantly. The formula can predict tolerances and incident angles and provide basic tool for imaging quality analysis and manufacturing for optical waveguide AR/VR display systems.

Holographically customized optical combiner for eye-box extended near-eye display.

We propose a customizing method of a holographic optical element (HOE) by using a holographic printer, which extends the eye-box with high field of view (FOV) for a holographic augmented reality near-eye display (AR NED). The holographic printer setup to manufacture HOE is presented and a prototype of the AR NED is implemented. To make a simple AR NED system, we propose a total internal reflection holographic printing method using an index-matching optical frame. As a result, the eye-box of the AR NED is extended in both vertical and horizontal directions and FOV of 50° is achieved at the center of the eye-box. Through the simulations and the experimental results, the feasibility of the proposed method is verified.

Manufacturing Application-Driven Foveated Near-Eye Displays.

Traditional optical manufacturing poses a great challenge to near-eye display designers due to large lead times in the order of multiple weeks, limiting the abilities of optical designers to iterate fast and explore beyond conventional designs. We present a complete near-eye display manufacturing pipeline with a day lead time using commodity hardware. Our novel manufacturing pipeline consists of several innovations including a rapid production technique to improve surface of a 3D printed component to optical quality suitable for near-eye display application, a computational design methodology using machine learning and ray tracing to create freeform static projection screen surfaces for near-eye displays that can represent arbitrary focal surfaces, and a custom projection lens design that distributes pixels non-uniformly for a foveated near-eye display hardware design candidate. We have demonstrated untethered augmented reality near-eye display prototypes to assess success of our technique, and show that a ski-goggles form factor, a large monocular field of view (30o×55o), and a resolution of 12 cycles per degree can be achieved.

Retinal 3D

We introduce an augmented reality near-eye display dubbed "Retinal 3D." Key features of the proposed display system are as follows: Focus cues are provided by generating the pupil-tracked light field that can be directly projected onto the retina. Generated focus cues are valid over a large depth range since laser beams are shaped for a large depth of field (DOF). Pupil-tracked light field generation significantly reduces the needed information/computation load. Also, it provides "dynamic eye-box" which can be a break-through that overcome the drawbacks of retinal projection-type displays. For implementation, we utilized a holographic optical element (HOE) as an image combiner, which allowed high transparency with a thin structure. Compared with current augmented reality displays, the proposed system shows competitive performances of a large field of view (FOV), high transparency, high contrast, high resolution, as well as focus cues in a large depth range. Two prototypes are presented along with experimental results and assessments. Analysis on the DOF of light rays and validity of focus cue generation are presented as well. Combination of pupil tracking and advanced near-eye display technique opens new possibilities of the future augmented reality.