Super Multi-view Research Articles

Adjustable viewpoint allocation for enhancing the viewing angle and reducing the crosstalk in super multi-view display.

In this Letter, we propose an adjustable viewpoint allocation method with forward and backward ray tracing to enhance the viewing angle and reduce the crosstalk in super multi-view (SMV) display. The synthetic image (SI) is initially calculated by backward ray tracing according to the viewing distance and the viewpoint interval. Forward ray tracing is then performed on the result of the viewpoint allocation to correct the deviation between the actual and the ideal viewpoint positions. Optical experiments are performed to verify the effectiveness of the proposed method. The results show that the proposed method effectively suppresses the crosstalk between viewpoints and eliminates artifacts in adjacent viewing areas within the ideal viewing angle, thereby enhancing the actual viewing angle. Meanwhile, due to the ability of reallocating information on the SI, the proposed method can achieve varying information density from different views. It is expected that our proposed method could be widely used in light field display to realize high quality display in the future.

Ultrathin metasurface for super multi-view 3D display with linear and circular polarization control

Multiview 3D display technologies are advantageous due to their smooth motion parallax, visual discomfort alleviation, and wide depth of focus. However, the potential applications of this technology are constrained by its bulky size and limited light modulation capability. Metasurfaces offer diverse wavefront control by manipulating the phase, amplitude, and polarization of light. They provide design freedom and high image quality for multiview 3D display technologies. Here, we propose an ultrathin metasurface-based super multiview 3D display that can effectively modulate the flexibility of linearly polarized (LP) and circularly polarized (CP) electromagnetic waves. Numerical results demonstrate that it can produce 24 views together for LP and CP waves and high-quality reconstructed 3D images are obtained. The suggested metasurface for super multiview 3D display is promising to underpin the advancement of meta-optics-based operations in 3D display, imaging and integrated optics.

Frame Loss Effects on Visual Fatigue in Super Multi-View 3D Display Technology

Super multi-view (SMV) display is a promising 3D display technology; however, potential frame loss due to bandwidth-limited video transmission could cause discomfort to viewers. Thus, an evaluation of the acceptable viewing experience will be valuable. This study investigates the effects of frame loss on visual fatigue in SMV display, focusing on quantified frame loss rates and varying frame loss modes. Experiments were conducted with 20 subjects, utilizing the Stroop test through an SMV display system to evaluate the visual fatigue under different frame loss conditions. The results show a rise in visual fatigue as the frame loss rate increases, with two critical thresholds identified. A 4% frame loss rate marks the threshold for significant loss-induced differences, beyond which visual fatigue begins to become significant in loss-induced modes compared to the normal loss-free mode. Subsequently, a 10% frame loss rate marks the threshold for significant mode-dependent differences, beyond which variations appear between different loss-induced modes, with monocular mode inducing more visual fatigue than binocular and dual-view more than single-view. Consequently, the findings advocate for refining the 3D video processing to maintain a frame loss rate below 4% for negligible effect and considering the interactions between different views for less visual fatigue. This research aims to provide insights and guidance for addressing potential challenges in developing and applying SMV display technology.

VAC-free near-eye display with thin form factor

Thin form factor and vergence-accommodation conflict-free (VAC-free) display are what the head-mounted near-eye display (NED) systems pursue all along. In this paper, we propose a NED system, with a waveguide providing parallel backlights to a display panel along different directions sequentially in each eye-piece. A converging lens is attached to a display panel, converging the directional backlights from corresponding waveguide to different viewpoints, respectively. The waveguide, display panel and converging lens of an eye-piece are closely stacked for a thin form factor, and the directional backlight guarantees a narrow beam from each pixel for VAC-free Super Multi-View (SMV) and/or Maxwellian view display. VAC-free effect is experimentally demonstrated in our proof-of-concept NED system of 11 mm in thickness.

Parameter optimization method for light field 3D display

A parameter optimization method for light field three-dimensional (3D) display is proposed to improve the performance of integral imaging (InI) and super multi-view (SMV) display. The parameters of light field display are divided into system parameters and display parameters. The multi-objective genetic algorithm is used to optimize the system parameters so that the display parameters can reach the target values. The influence of all display parameters on the imaging quality of light field 3D display are comprehensively considered. When the weight of the depth of field (DOF) and field of view (FOV) are set to the maximum, the DOF and FOV are respectively improved compared with the original display parameters. By setting the weight of resolution to the maximum, the image quality of InI and SMV is improved by 32% and 21%, respectively.

A Depth-Enhanced Holographic Super Multi-View Display Based on Depth Segmentation.

A super multi-view (SMV) near-eye display (NED) effectively provides depth cues for three-dimensional (3D) display by projecting multiple viewpoint or parallax images onto the retina simultaneously. Previous SMV NED have suffered from a limited depth of field (DOF) due to a fixed image plane. In this paper, a holographic SMV Maxwellian display based on depth segmentation is proposed to enhance the DOF. The proposed approach involves capturing a set of parallax images and their corresponding depth maps. According to the depth maps, the parallax images are segmented into N sub-parallax images at different depth ranges. These sub-parallax images are then projected onto N image-recording planes (IRPs) of the corresponding depth for hologram computation. The wavefront at each IRP is calculated by multiplying the sub-parallax images with the corresponding spherical wave phases. Then, they are propagated to the hologram plane and added together to form a DOF-enhanced hologram. The simulation and experimental results are obtained to validate the effectiveness of the proposed method in extending the DOF of the holographic SMV displays, while accurately preserving occlusion.

Depth-Enhanced Holographic Super Multi-View Maxwellian Display Based on Variable Filter Aperture.

The super multi-view (SMV) near-eye display (NED) effectively provides depth cues for three-dimensional (3D) displays by projecting multiple viewpoint images or parallax images onto the retina simultaneously. Previous SMV NED suffers from a limited depth of field (DOF) due to the fixed image plane. Aperture filtering is widely used to enhance the DOF; however, an invariably sized aperture may have opposite effects on objects with different reconstruction depths. In this paper, a holographic SMV display based on the variable filter aperture is proposed to enhance the DOF. In parallax image acquisition, multiple groups of parallax images, each group recording a part of the 3D scene on a fixed depth range, are captured first. In the hologram calculation, each group of wavefronts at the image recording plane (IRP) is calculated by multiplying the parallax images with the corresponding spherical wave phase. Then, they are propagated to the pupil plane and multiplied by the corresponding aperture filter function. The size of the filter aperture is variable which is determined by the depth of the object. Finally, the complex amplitudes at the pupil plane are back-propagated to the holographic plane and added together to form the DOF-enhanced hologram. Simulation and experimental results verify the proposed method could improve the DOF of holographic SMV display, which will contribute to the application of 3D NED.

56.2: A dynamic backlight for a wide‐angle light field display based on supermulti‐view technology

In naked‐eye 3D display, the light field display has been widely concerned because it can provide vivid and natural 3D performance, and light field display technology based on super‐multi‐view (SMV) has also become an important research direction of naked‐eye 3D display. However, insufficient viewing angle is one of the main factors limiting the development and application of light field display technology. In this paper, we propose a wide‐angle‐splicing display method for dynamic backlight system based on the light field display with SMV technology. The experimental results show that the viewing angle is increased by about 50% from the original 60° to more than 90°, and a time‐multiplexed light field display with a 90° wide viewing angle is demonstrated. At the same time, it can provide the viewer with smooth and continuous motion parallax, and clear 3D images can be perceived at the entire range of viewing angle, which greatly improves the viewer's visual experience.

Super multi-view display based on near-eye pinholes.

Vergence-accommodation-conflict-free super multi-view (SMV) display based on more than one near-eye pinhole group for each pupil of the viewer is developed in this paper. Two-dimensionally arranged pinholes of a group correspond to different subscreens of the display screen, with perspective views projected by each subscreen through a corresponding pinhole splicing into an image of enlarged field of view (FOV). By sequentially switching on/off different pinhole groups, more than one mosaic image is projected to each pupil of the viewer. Adjacent pinholes of a group are endowed with different timing-polarizing characteristics to generate an effective noise-free region for each pupil. In the experiment, four groups of 3×3 pinholes are configured for a proof-of-concept SMV display on a 240Hz display screen, with a diagonal FOV of 55deg and a depth of field reaching 1.2m.

Super multi-view near-eye virtual reality with directional backlights from wave-guides.

Directional backlights have often been employed for generating multiple view-zones in three-dimensional (3D) display, with each backlight converging into a corresponding view-zone. By designing the view-zone interval for each pupil smaller than the pupil's diameter, super multi-view (SMV) can get implemented for a VAC-free 3D display. However, expanding the backlight from a light-source to cover the corresponding display panel often needs an extra thickness, which results in a thicker structure and is unwanted by a near-eye display. In this paper, two wave-guides are introduced into a near-eye virtual reality (NEVR) system, for sequentially guiding more than one directional backlight to each display panel for SMV display without bringing obvious extra thickness. A prototype SMV NEVR gets demonstrated, with two backlights from each wave-guide converging into two view-zones for a corresponding pupil. Although the additional configured light-sources are positioned far from the corresponding wave-guide in our proof-of-concept prototype, multiple light-sources can be attached to the corresponding wave-guide compactly if necessary. As proof, a 3D scene with defocus-blur effects gets displayed. The design range of the backlights' total reflection angles in the wave-guide is also discussed.

Accommodation and Vergence Responses to Electronic Holographic Displays and Super Multiview Holographic Stereograms

Electronic holography is an ideal 3-D display technique to ameliorate the vergence-accommodation conflict, which is a possible cause of visual fatigue and discomfort from viewing conventional stereoscopic 3-D displays. Previous studies have measured accommodative and vergence responses to holographic images and to real objects, and the results showed that these responses are in good agreement. To demonstrate the effectiveness of electronic holography as a possible solution to the vergence-accommodation conflict caused by viewing conventional stereoscopic 3-D displays, we measured both accommodative and vergence responses to reconstructed images of holograms and super multiview holographic stereograms. We also measured responses to real objects located at various distances and conventional two-view stereo images for comparison. The results indicate that the accommodative responses to the electro-holographic display and super multiview holographic stereogram change with the vergence response in a similar manner, as is the case with real objects as compared to conventional two-view stereo display. This suggests that the electronic holographic display and super multiview holographic stereograms are promising candidates for resolving the vergence-accommodation conflict.

Holographic super multi-view Maxwellian near-eye display with eyebox expansion.

A holographic super multi-view (SMV) Maxwellian display based on flexible wavefront modulation is proposed for the first time, to the best of our knowledge. It solves the issue that the previous holographic Maxwellian displays could not provide depth cues for monocular vision. Different from the previous methods, two or more parallax images are multiplied by quadric phase distributions and converged to the viewpoints existing in the pupil to provide 3-D vision. A time division method is proposed to eliminate the cross talk caused by the coherence of different spherical waves. Experiments demonstrate that the proposed method can accurately reconstruct images at different depth without cross talk. The proposed method inherits the previous holographic Maxwellian display's advantages of flexible viewpoint position adjustment and large depth of field (DOF). Superior to geometric optics based SMV displays, the proposed system is compact without lens aberration since only a single spatial light modulator (SLM) is needed without any additional optical elements.

Polarization enlargement of FOV in Super Multi-view display based on near-eye timing-apertures.

With strip-type timing-apertures attached to each eye of a viewer, more than one perspective views can be guided to either eye sequentially through different timing-apertures, thus implementing VAC-free (vergence-accommodation conflict-free) SMV (Super Multi-view) 3D (three-dimensional) display. To overcome the FOV (field of view) limitation problem due to small size of the timing-apertures along their arrangement direction, novel polarization architectures are designed to the timing-apertures in this paper. Correspondingly, the display screen of the proposed SMV display system is divided into M > 1 sub-screens along the arrangement direction of the timing-apertures, with adjacent sub-screens emitting light of mutually orthogonal polarization. At a time-point of each time period, a group of M timing-apertures, which correspond to the M sub-screens in a one-by-one manner along the arrangement direction, are turned on for creating an M-fold FOV, with each polarized timing-aperture of the group allowing light from the corresponding sub-screen passing through and blocking light from sub-screen(s) adjacent to the corresponding sub-screen. At 2T > 1 time-points of each time period, 2T groups of timing-apertures are turned on sequentially for presenting more than one two-dimensional images of the displayed scene to each eye, to implement SMV display based on persistence of vision. M stands for the FOV magnification number and T stands for the two-dimensional image number for each eye. As proof, a 3-fold FOV of 41° gets implemented experimentally with a currently available timing-aperture array of M = 3, accompanied by an effective noise-free region (ENFR) of 8.34 mm. Furthermore, the promising of freeing FOV from timing-aperture constraint fundamentally by larger M is described, out-of-screen blur along strip direction of the timing-apertures and the problem of limited ENFR are discussed.

Super multi-view near-eye 3D display with enlarged field of view

Based on temporal-multiplexing, super multi-view (SMV) technology gets implemented on a near-eye three-dimensional (3D) display, successfully overcoming the vergence accommodation conflict (VAC). The proposed system has two eyepieces for two eyes, respectively, each of which is constructed by an organic light-emitting diode (OLED) micro-display, a projecting lens, and a gating-aperture array. The gating-aperture arrays, consisting of gating apertures with an interval smaller than the diameter of the viewer’s pupil, are attached to the projecting lenses and gate different spatial segments of the projecting lens’s exit aperture in time sequence. Through refreshing the corresponding display contents synchronously, two or more light rays passing through a displayed point are guided by the gating-aperture array into each eye sequentially. Based on the persistence of vision, these light rays converge into a spatial light spot that the eye can focus on naturally, thus enabling a consistence between focusing distance and convergence distance. Further, gating-aperture array with polarization characteristics is designed to get a larger field of view (FOV), accompanied by a half-wave variable retarder being inserted between the OLED micro-display and the gating-aperture array in each eyepiece. Employing two OLED micro-displays of 120-Hz frame rates, a near-eye 3D display prototype has been demonstrated with an FOV 40 deg and a display frequency of 40 Hz, being free from VAC.

P‐24: Evaluation Method of SMV Crosstalk Based on Human Visual Characteristics

A parameter of definition was proposed to evaluate the crosstalk of super‐multi‐view (SMV) 3D display, which could indicate the contrast decline of the different spatial frequencies at the different display depths perceived by human eyes. Two calculation methods for the definition was presented. One was indirectly calculated by the single‐view‐luminance‐curve (SVLC) with the faster calculation speed. The other was directly measured when the screen displayed the test images, while the measurement result was easily understandable but need more complex image processing. The results obtained by the two methods were in good agreement.

68‐2: View‐Dependent Light‐Field Display that Supports Accommodation Using a Commercially‐Available High Pixel Density LCD Panel

Super multiview displays generally support only a small range of viewpoints. We describe a view‐dependent light‐field display approach that combines custom optics, eye position tracking, a high‐density display and a directional backlight to provide full parallax and a 3D volume from which a viewer can see 3D objects with accommodation.

A Real-Time Super Multiview Rendering Pipeline for Wide Viewing-Angle and High-Resolution 3D Displays Based on a Hybrid Rendering Technique

Super multiview (SMV) 3D displays with a directional diffuser can provide dense viewpoints, wide viewing angles and high image quality. However, traditional SMV 3D rendering pipelines have many disadvantages, such as high time costs, low image quality, incorrect lighting effects and limited viewing angles. To overcome the abovementioned drawbacks, we propose a new SMV 3D rendering pipeline based on a hybrid rendering technique (HRT) that effectively combines and improves the Turing multiview rendering technique, the multiple view plus depth (MVD) technique and the deferred shading technique, introducing additional normal information, diffuseness information and shininess information into the SMV 3D rendering pipeline. The SMV HRT pipeline can generate SMV 3D images with accurate lighting in real time when the viewpoint number of an SMV 3D display is greater than 50, the viewing angle is greater than 100 degrees, the resolution of a single viewpoint image is more than 512*512 and the resolution of the LCD panel is 7680*4320; in particular, complex scenes can be generated in real time. The proposed HRT pipeline is the only viable real-time rendering pipeline with accurate lighting effects for super multiview 3D displays with wide viewing angle and high resolution that runs on a common PC with standard graphics hardware. An 8k SMV 3D display with an 80° viewing angle and 100 viewpoints is applied to demonstrate the validity of the HRT method in experiments. When the number of reference views is four, the experimental results show that the rendering frame rate is more than 35 fps on average for complex virtual scenes, and the peak signal-to-noise ratio (PSNR) value of the HRT is greater than 36 dB, illustrating its high image quality and accurate lighting. Compared with state-of-the-art MVD methods, HRT processes more lighting information and is more suitable for virtual scene rendering. Therefore, the HRT method has advantages regarding the image quality of the reconstructed 3D image and can easily achieve real-time rendering for SMV 3D displays with wide viewing angles.

Resolution Evaluation of a Simplified Super Multi-View Head-Mounted Display

Resolution Evaluation of a Simplified Super Multi-View Head-Mounted Display

Resolution Evaluation of a Simplified Super Multi-View Head-Mounted Display

Resolution Evaluation of a Simplified Super Multi-View Head-Mounted Display

Super multi-view near-eye display to solve vergence–accommodation conflict

A super multi-view (SMV) technique is applied to near-eye displays to solve the vergence-accommodation conflict that causes visual fatigue. The proposed SMV near-eye display employs a high-speed spatial light modulator (SLM), a two-dimensional (2D) light source array, and an imaging optics for each eye. The imaging optics produces a virtual image of the SLM and real images of the light sources to generate a 2D array of viewpoints. The SMV images are generated using a time-multiplexing technique: the multiple light sources sequentially emit light while the SLM synchronously displays corresponding parallax images. A monocular experimental system was constructed using a ferroelectric liquid crystal display and an LED array. A full-parallax SMV image generation with 21 viewpoints was demonstrated and a comparison of full-parallax and horizontal parallax SMV images provided.