Multi-plane Display Research Articles

Why 2D layout in 3D images matters: evidence from visual search and eyetracking.

Precise perception of three-dimensional (3D) images is crucial for a rewarding experience when using novel displays. However, the capability of the human visual system to perceive binocular disparities varies across the visual field meaning that depth perception might be affected by the two-dimensional (2D) layout of items on the screen. Nevertheless, potential difficulties in perceiving 3D images during free viewing have received only a little attention so far, limiting opportunities to enhance visual effectiveness of information presentation. The aim of this study was to elucidate how the 2D layout of items in 3D images impacts visual search and distribution of maintaining attention based on the analysis of the viewer's gaze. Participants were searching for a target which was projected one plane closer to the viewer compared to distractors on a multi-plane display. The 2D layout of items was manipulated by changing the item distance from the center of the display plane from 2° to 8°. As a result, the targets were identified correctly when the items were displayed close to the center of the display plane, however, the number of errors grew with an increase in distance. Moreover, correct responses were given more often when subjects paid more attention to targets compared to other items on the screen. However, a more balanced distribution of attention over time across all items was characteristic of the incorrectly completed trials. Thus, our results suggest that items should be displayed close to each other in a 2D layout to facilitate precise perception of 3D images and considering distribution of attention maintenance based on eye-tracking might be useful in the objective assessment of user experience for novel displays.

Multiplane holographic augmented reality head-up display with a real-virtual dual mode and large eyebox.

We propose a multiplane augmented reality (AR) head-up display (HUD) with a real-virtual dual mode based on holographic optical elements (HOEs). The picture generation unit (PGU) is only a single free-focus projector, and the optical combiner includes a HOE lens (HOEL) for long-distance virtual image display and a HOE diffuser (HOED) for in-plane real image display. A HOED with directional scattering characteristics in the real image mode can significantly increase the size of the eyebox (EB) without increasing the size of the HOE, and a HOEL with a flexible design for the optical focal length in the virtual image mode can be used to achieve a different depth of the AR display. The proposed AR HUD system, which has a compact structure and offers high light transmittance, high energy usage, a multiplane display, and a large EB, is expected to be widely used in the future.

Method of Speckle Noise Suppression for Holographic Zoom Display Based on Layered-Pixel-Scanning Algorithm

In this paper, a method of speckle noise suppression for holographic zoom display based on layered-pixel-scanning (LPS) algorithm is proposed. Different from the traditional algorithm, the proposed method suppresses the speckle noise based on a new algorithm design and light source optimization. The LPS algorithm is designed to calculate the phase-only hologram of the 3D object. By extracting the initial complex hologram of the 3D object and scanning the hologram by pixel, the errors of the complex amplitude can be reduced as much as possible. Then a hologram subsampling method based on the LPS algorithm generated is proposed, so the scaling of the reproduced image can be achieved easily. At the same time, the pattern with different random phase is generated and loaded on the spatial light modulator so that the reconstructed light can be optimized. When the optimized light irradiates on the phase hologram of the object, the reconstructed image with suppressed speckle noise can be displayed. The experimental results show that the proposed method has an excellent reproduction effect. Moreover, the feasibility and unique advantages of the proposed method in the holographic augmented-reality display, zoom display and multi-plane display have been proved. We believe that this method can promote the application of the holographic display.

Multi-plane augmented reality display based on cholesteric liquid crystal reflective films.

To address the accommodation-convergence conflict problem in conventional augmented reality (AR) head-mounted displays, we propose a compact multi-plane display design based on cholesteric liquid crystal (CLC) reflective films and a polarization switch. Because of the polarization selectivity of CLC films, circularly-polarized light with different handedness is reflected by different CLC films, resulting in different optical path lengths and different image depths by the lens. A flicker-free dual-plane prototype with correct focus cues and relatively low operating voltage has been implemented. Moreover, a multi-plane AR display scheme with more than 2 depth planes is proposed by stacking multiple CLC films and polarization switches together.

Fabrication Process of a Reverse Mode Polymer-Dispersed Liquid Crystal using UV Curing

We propose a multi-plane display based on polymer stabilized cholesteric texture (PSCT) Liquid Crystal. The idea of a multi-plane display has shown progress in solving the vergence-accommodation conflict with the eye-brain system. Nowadays new emerging technologies such as the Augmented Reality (AR) and Virtual Reality (VR) share that problem. VR and AR wearable displays, work using stereoscopic 3D glasses, which push the vergence-accommodation conflict. The advantage of the UV-curing method to create a polymer network is the ease to use it on normal lab environments. This method can achieve acceptable response time for a transparent-scattered state on a LC cell, which is needed for a multi-plane display. On this work, we are explaining the fabrication process to create such PSCT LC cell.

A Perception-driven Hybrid Decomposition for Multi-layer Accommodative Displays.

Multi-focal plane and multi-layered light-field displays are promising solutions for addressing all visual cues observed in the real world. Unfortunately, these devices usually require expensive optimizations to compute a suitable decomposition of the input light field or focal stack to drive individual display layers. Although these methods provide near-correct image reconstruction, a significant computational cost prevents real-time applications. A simple alternative is a linear blending strategy which decomposes a single 2D image using depth information. This method provides real-time performance, but it generates inaccurate results at occlusion boundaries and on glossy surfaces. This paper proposes a perception-based hybrid decomposition technique which combines the advantages of the above strategies and achieves both real-time performance and high-fidelity results. The fundamental idea is to apply expensive optimizations only in regions where it is perceptually superior, e.g., depth discontinuities at the fovea, and fall back to less costly linear blending otherwise. We present a complete, perception-informed analysis and model that locally determine which of the two strategies should be applied. The prediction is later utilized by our new synthesis method which performs the image decomposition. The results are analyzed and validated in user experiments on a custom multi-plane display.

Polarization-multiplexed multiplane display.

We demonstrate a polarization-multiplexed multiplane display system for near-eye applications. A polarization-sensitive Pancharatnam-Berry phase lens is implemented to generate two focal depths simultaneously. A spatial polarization modulator is utilized to direct the two images to designated focal planes. Based on this design, a dual-focal-plane display system is constructed without space- or time-multiplexing operations, to suppress the vergence-accommodation conflict successfully.

Reverse-mode PSLC multi-plane optical see-through display for AR applications.

In this paper we propose an optical see-through multi-plane display with reverse-mode polymer-stabilized liquid crystal (PSLC). Our design solves the problem of accommodation-vergence conflict with correct focus cues. In the reverse mode PSLC system, power consumption could be reduced to ~1/(N-1) of that in a normal mode system if N planes are displayed. The PSLC films fabricated in our experiment exhibit a low saturation voltage ~20 Vrms, a high transparent-state transmittance (92%), and a fast switching time within 2 ms and polarization insensitivity. A proof-of-concept two-plane color display prototype and a four-plane monocolor display prototype were implemented.

Layered Display with Accommodation Cue Using Scattering Polarizers

A projection-type layered display inducing the accommodation cue is proposed. To support the accommodation cue, the concept of multiplane displays, especially the principle of additive type compressive light field display, is adopted. To achieve the additive type light field display, two layered scattering polarizers and a single projector which is synchronized with polarization rotator are used. Scattering polarizer diffuses light rays with specific linear polarization state while transmits the other rays with the orthogonal polarization state. Utilizing this characteristic, two layer images can be displayed at different depths presenting multiplane display. By adopting an additional computational process, several viewpoints can be provided to a single eye and support the accommodation cue. We expect that the proposed method can be adopted to virtual reality system. Detailed principle of the proposed system and its feasibility are explained with simulations and experiments.

Blur and the perception of depth at occlusions.

The depth ordering of two surfaces, one occluding the other, can in principle be determined from the correlation between the occlusion border's blur and the blur of the two surfaces. If the border is blurred, the blurrier surface is nearer; if the border is sharp, the sharper surface is nearer. Previous research has found that observers do not use this informative cue. We reexamined this finding. Using a multiplane display, we confirmed the previous finding: Our observers did not accurately judge depth order when the blur was rendered and the stimulus presented on one plane. We then presented the same simulated scenes on multiple planes, each at a different focal distance, so the blur was created by the optics of the eye. Performance was now much better, which shows that depth order can be reliably determined from blur information but only when the optical effects are similar to those in natural viewing. We asked what the critical differences were in the single- and multiplane cases. We found that chromatic aberration provides useful information but accommodative microfluctuations do not. In addition, we examined how image formation is affected by occlusions and observed some interesting phenomena that allow the eye to see around and through occluding objects and may allow observers to estimate depth in da Vinci stereopsis, where one eye's view is blocked. Finally, we evaluated how accurately different rendering and displaying techniques reproduce the retinal images that occur in real occlusions. We discuss implications for computer graphics.

A multi-plane optical see-through head mounted display design for augmented reality applications

Augmented reality (AR) display technology greatly enhances users' perception of and interaction with the real world by superimposing a computer-generated virtual scene on the real physical world. The main problem of the state-of-the-art 3D AR head-mounted displays (HMDs) is the accommodation-vergence conflict because the 2D images displayed by flat panel devices are at a fixed distance from the eyes. In this paper, we present a design for an optical see-through HMD utilizing multi-plane display technology for AR applications. This approach manages to provide correct depth information and solves the accommodation-vergence conflict problem. In our system, a projector projects slices of a 3D scene onto a stack of polymer-stabilized liquid crystal scattering shutters in time sequence to reconstruct the 3D scene. The polymer-stabilized liquid crystal shutters have sub-millisecond switching time that enables sufficient number of shutters to achieve high depth resolution. A proof-of-concept two-plane optical see-through HMD prototype is demonstrated. Our design can be made lightweight, compact, with high resolution, and large depth range from near the eye to infinity and thus holds great potential for fatigue-free AR HMDs.

Registration based super-resolution reconstruction for lung 4D-CT.

Lung 4D-CT plays an important role in lung cancer radiotherapy for tumor localization and treatment planning. In lung 4D-CT data, the resolution in the slice direction is often much lower than the in-plane resolution. For multi-plane display, isotropic resolution is necessary, but the commonly used interpolation operation will blur the images. In this paper, we present a registration based method for super resolution enhancement of the 4D-CT multi-plane images. Our working premise is that the low-resolution images of different phases at the corresponding position can be regarded as input "frames" to reconstruct high resolution images. First, we employ the Demons registration algorithm to estimate the motion field between different "frames". Then, the projections onto convex sets (POCS) approach is employed to reconstruction high-resolution lung images. We show that our method can get clearer lung images and enhance image structure, compared with the cubic spline interpolation and back projection method.

Creating effective focus cues in multi-plane 3D displays

Focus cues are incorrect in conventional stereoscopic displays. This causes a dissociation of vergence and accommodation, which leads to visual fatigue and perceptual distortions. Multi-plane displays can minimize these problems by creating nearly correct focus cues. But to create the appearance of continuous depth in a multi-plane display, one needs to use depth-weighted blending: i.e., distribute light intensity between adjacent planes. Akeley et al. [ACM Trans. Graph. 23, 804 (2004)] and Liu and Hua [Opt. Express 18, 11562 (2009)] described rather different rules for depth-weighted blending. We examined the effectiveness of those and other rules using a model of a typical human eye and biologically plausible metrics for image quality. We find that the linear blending rule proposed by Akeley and colleagues [ACM Trans. Graph. 23, 804 (2004)] is the best solution for natural stimuli.