Multi-view Autostereoscopic Display Research Articles

Compensation of lens manufacturing errors and inhomogeneities by filtering view images in three-dimensional lenticular displays.

For a multiview autostereoscopic three-dimensional display based on the lenticular lens, the manufacturing errors and inhomogeneities of the lenticular sheet can deteriorate the view image quality. A calibration method is proposed by filtering the pixel mapping of all view images. Each view is first sequentially turned on to show a white test image, and the intensity distribution of every view will be analyzed. Then individual filter functions are derived and applied to the corresponding view images to correct the subpixel value. Experimental results show that the proposed method can solve view image loss and significantly improve the quality of stereoscopic images.

Dynamic Multi-View Autostereoscopy

With advantages of motion parallax and viewing convenience, multi-view autostereoscopic displays have attracted increasing attention in recent years. It is obvious that increasing the number of views improves the quality of 3D images/videos and leads to better motion parallax. However, it requires huge amount of computing resources to generate large numbers of view images in real time. In principle, objects appearing near the screen plane have very small absolute disparity. It can use fewer views to present these objects for achieving the same level of motion parallax. The concept of dynamic multi-view autostereoscopy is to dynamically control the number of views to generate for the points in 3D space based on their disparity. Points with larger absolute disparity use more views, while points with smaller absolute disparity use fewer views. As a result, fewer computing resources are required for real-time generation of view images. Subjective assessments show that only slight degradation in 3D experience is resulted on its realization over 2D plus depth based multi-view autostereoscopic display. However, the amount of computation for generating view images can be reduced by about 44.3% when 3D scenes are divided into three spaces.

Real-time ultra-high definition multiview glasses-free 3D display system

The design and implementation of a real-time ultra-high definition (UHD) multiview glasses-free 3D display system always suffer from the high transmission bandwidth, high memory cost, and high computational complexity. This paper presents a glasses-free 3D display system based on depth-image-based rendering (DIBR) technique by solving these problems. It can convert a V $$+$$ D video (RGBD) stream in real time to a multiview representation suitable for a multiview autostereoscopic display. As V $$+$$ D video format is used for the system, the transmission bandwidth is reduced. For the memory cost, we introduce an asymmetric shift-sensor camera setup to avoid external memory usage and reduce the storage requirement of multiviews. For the computational complexity, as our camera setup ensures that all the virtual views can be generated with the same DIBR algorithm, the computational complexity is reduced. In addition, we simplify the view fusion method so as to rearrange the subpixels from multiviews with lower complexity to form a single glasses-free 3D image. Moreover, we propose a hardware architecture for the system and implement it using field programmable gate array. Simulation results show that our system can support the UHD V $$+$$ D videos for an 8-view glasses-free 3D display. Performance evaluation results show that the proposed system can provide reasonably good stereoscopic image quality if appropriate parameters of the system are applied.

Autostereoscopic 3D display system with dynamic fusion of the viewing zone under eye tracking: principles, setup, and evaluation [Invited

We study optical technologies for viewer-tracked autostereoscopic 3D display (VTA3D), which provides improved 3D image quality and extended viewing range. In particular, we utilize a technique-the so-called dynamic fusion of viewing zone (DFVZ)-for each 3D optical line to realize image quality equivalent to that achievable at optimal viewing distance, even when a viewer is moving in a depth direction. In addition, we examine quantitative properties of viewing zones provided by the VTA3D system that adopted DFVZ, revealing that the optimal viewing zone can be formed at viewer position. Last, we show that the comfort zone is extended due to DFVZ. This is demonstrated by a viewer's subjective evaluation of the 3D display system that employs both multiview autostereoscopic 3D display and DFVZ.

Three-dimensional display with directional beam splitter array.

Multi-view three-dimensional (3-D) displays using directional beam splitter array were proposed to achieve a perfect 3-D perception with low cross-talk. The multi-direction collimated light may project different images to different viewing zones to form the multi-view autostereoscopic display. Furthermore, a high resolution 3-D display can be realized with a sequential beam splitter array and a sequential liquid crystal display. By optimization, the cross-talk of the directional beam splitter backlight system was lowered to 5% to improve the perception of the 3-D displays.

Electrowetting Lenticular Lens for a Multi-View Autostereoscopic 3D Display

The multi-view autostereoscopic display has become a major form of technology for implementing 3D displaying because of its simple structure and good image characteristics compared with other methods. In particular, the lenticular type multi-view display has a higher luminance than the parallax barrier type display, which is a significant advantage. However, the lenticular lens is difficult to fabricate and uncontrollable in 2D–3D conversion. Here, we describe the fabrication of an electrowetting-based lenticular lens and a multi-view display system that utilizes that lens. With a lenticular shaped lens chamber and electrowetting liquids, the focal length of the lenticular lens can be controlled by applying voltage. By putting this device on a display panel, a 3D image can be observed with a convex lens state, and a 2D image is observed with a flat state. In this letter, an electrowetting lenticular lens is fabricated and a 2D and 3D mode switch multi-view display is demonstrated.

Hybrid ASIC/FPGA System for Fully Automatic Stereo-to-Multiview Conversion Using IDW

Recently, multiview autostereoscopic dis-plays (MADs), which enable a limited glasses-free 3D experience, have become commercially available. The main problem of MADs is that they require several (typically eight or nine) views, while most of the 3D video content is in stereoscopic 3D today. In order to bridge this gap, the research community started to devise automatic multiview synthesis (MVS) methods. These algorithms require real-time processing and should be portable to end-user devices to develop their full potential. To this end, we revisit an algorithmic solution based on image domain warping (IDW) and devise a hardware architecture of a complete synthesis pipeline, provide insights into where the computationally challenging parts are, and present implementation results of a hybrid field programmable gate array/application-specific integrated circuit prototype, which is the first hardware implementation of a complete IDW-based MVS system. Based on these results, we also estimate the complexity and energy efficiency of a fully integrated solution in 65- and 28-nm CMOS technology and show that a full-high-definition real-time solution on a single chip is within reach. The proposed architecture could be used as a coprocessor in a system-on-chip targeting 3D TV sets, thereby enabling efficient content generation with limited user interaction (e.g., depth range adjustment) in real time.

Diffraction effects incorporated design of a parallax barrier for a high-density multi-view autostereoscopic 3D display.

We present optical characteristics of view image provided by a high-density multi-view autostereoscopic 3D display (HD-MVA3D) with a parallax barrier (PB). Diffraction effects that become of great importance in such a display system that uses a PB, are considered in an one-dimensional model of the 3D display, in which the numerical simulation of light from display panel pixels through PB slits to viewing zone is performed. The simulation results are then compared to the corresponding experimental measurements with discussion. We demonstrate that, as a main parameter for view image quality evaluation, the Fresnel number can be used to determine the PB slit aperture for the best performance of the display system. It is revealed that a set of the display parameters, which gives the Fresnel number of ∼ 0.7 offers maximized brightness of the view images while that corresponding to the Fresnel number of 0.4 ∼ 0.5 offers minimized image crosstalk. The compromise between the brightness and crosstalk enables optimization of the relative magnitude of the brightness to the crosstalk and lead to the choice of display parameter set for the HD-MVA3D with a PB, which satisfies the condition where the Fresnel number lies between 0.4 and 0.7.

Parallax barrier engineering for image quality improvement in an autostereoscopic 3D display.

We present a image quality improvement in a parallax barrier (PB)-based multiview autostereoscopic 3D display system under a real-time tracking of positions of a viewer's eyes. The system presented exploits a parallax barrier engineered to offer significantly improved quality of three-dimensional images for a moving viewer without an eyewear under the dynamic eye tracking. The improved image quality includes enhanced uniformity of image brightness, reduced point crosstalk, and no pseudoscopic effects. We control the relative ratio between two parameters i.e., a pixel size and the aperture of a parallax barrier slit to improve uniformity of image brightness at a viewing zone. The eye tracking that monitors positions of a viewer's eyes enables pixel data control software to turn on only pixels for view images near the viewer's eyes (the other pixels turned off), thus reducing point crosstalk. The eye tracking combined software provides right images for the respective eyes, therefore producing no pseudoscopic effects at its zone boundaries. The viewing zone can be spanned over area larger than the central viewing zone offered by a conventional PB-based multiview autostereoscopic 3D display (no eye tracking). Our 3D display system also provides multiviews for motion parallax under eye tracking. More importantly, we demonstrate substantial reduction of point crosstalk of images at the viewing zone, its level being comparable to that of a commercialized eyewear-assisted 3D display system. The multiview autostereoscopic 3D display presented can greatly resolve the point crosstalk problem, which is one of the critical factors that make it difficult for previous technologies for a multiview autostereoscopic 3D display to replace an eyewear-assisted counterpart.

Crosstalk elimination in multi-view autostereoscopic display based on polarized lenticular lens array**Project supported by the National High Technology Research and Development Program of China (Grant No. 2012AA03A301), the National Natural Science Foundation of China (Grant No. 60932007), the Postdoctoral Science Programs Foundation of the Ministry of Education of China (Grant No. 0110032110029), and the Key Projects in the Tianjin Science

An autostereoscopic display composed of a directional backlight, an image display panel, a striped half-wave plate, and a polarized lenticular lens array is proposed. The directional backlight emitting the parallel light can redirect the cones of light to lenticular lens array and reduce the chromatic spatial-interference effect. The striped half-wave plate, located in front of the image display panel, transformed the polarization direction of the lights from the directional backlight into two mutually perpendicular directions. The polarized lenticular lens array not only can divide the light from the left and right view images to send to left and right eyes but also can reduce the crosstalk of the stereoscopic images. The proposed autostereoscopic display can produce high quality stereoscopic images without crosstalk at the optimal viewing distance.

Optimizing Time-Multiplexing Auto-Stereoscopic Displays With a Genetic Algorithm

A figure-of-merit (FOM) of an auto-stereoscopic display system is introduced and adopted to characterize the system performance. This FOM takes into account of the ratio of the signal to the noise arising from the crosstalk from the adjacent channels as well as the brightness uniformity of viewing areas; hence, it is directly related to the glasses-free 3D viewing comfort. With a steadily improving FOM as a target, the genetic algorithm is applied to optimize the optical system, giving rise to substantially improved characteristics of an auto-stereoscopic display system. The numerical simulation is verified with an experiment of a multi-view auto-stereoscopic display unit. It is shown that the system can provide a high fidelity of the display effect with the crosstalk ratio being reduced from around 5% to nearly 1%, which is a very low value obtainable for an auto-stereoscopic system.

Next Generation 3D Display Applications using Laser Scanning Pico Projectors

This thesis presents four novel 3D display methods and a number of new applications using commercial laser scanning pico projectors that have been developed with major contributions from our laboratory during the last decade. The new methods developed in this thesis are (i) mixed-polarization stereoscopic display using single-projector and passive polarized glasses, (ii) a new type of glasses that can relieve the accommodation and vergence conflict associated with all the stereoscopic displays, (iii) rotating screen based auto-stereoscopic display for a single-viewer using two pico-projectors, and (iv) multi-user multi-view auto-stereoscopic display using an array of pico-projectors. Furthermore, a head-mounted pico-projector is also demonstrated as a new augmented- reality application aimed at motion-capture actors. The first method introduces a new twist on glasses based stereoscopic projection displays as it utilizes polarization and color multiplexing simultaneously and avoids weaknesses associated with previous methods. This new method is named Mixed Polarization 3D. Color imbalance artifacts associated with anaglyph glasses are avoided by alternating the colors presented to each eye. In mixed polarization 3D, flicker is not observed even at 60 Hz since both eyes receive at least one color frame in every single frame. The second method, named Super Stereoscopic 3D Display (SS3D), introduces a major improvement on stereoscopic 3D displays for avoiding accommodation-vergence conflict. Our method provides at least two views to a single eye by using special apertures equipped with selective filters in front of the users’ eye. Our designs can be embedded into conventional stereoscopic glasses or can be developed as special contact lenses. The third method introduces a single viewer multi-view autostereoscopic projection display, which is invented in our laboratory by a co-worker. The technique uses two mobile projectors, a rotating retro-reflective diffuser screen, and a head-tracking camera. As two dynamic viewing slits are created at the viewers position, the slits can track the position of the eyes by rotating the screen. The last method introduces a new modular multi-user multi-view autostereoscopic display architecture based on an array of pico-projectors and a vertical diffuser screen. A single projector provides a portion of the each perspective observed through the vertical diffuser. Unlike similar projector-array based displays, the screen assembly complexity and the size of the projection display hardware have been decreased dramatically. Our proposal has the capability to provide horizontal expansion in the screen size, and an increase in the number of different perspectives as well. Lastly, an augmented reality (AR) system is proposed and developed for supporting motion capture actors. This system allows seeing and exploring the digital environment without occluding the actors visual field. An in-house prototype is built by combining a retro-reflective screen that covers the walls and a headband consisting of a laser scanning projector with a smartphone. Built-in sensors on the smartphone provide navigation capabilities in the digital world.

Latency optimization for autostereoscopic volumetric visualization in image-guided interventions

Autostereoscopic visualization in clinical image-guided interventions and therapy poses constrains on the maximal latency in the visualization chain. In order to use visual feedback in the hand-eye coordinated loop, the latency of the visualization chain should be less than approximately 250–300 ms. This is a challenging constraint for interactive autostereoscopic multi-view volume rendering of large datasets. The various building blocks of such a visualization chain and their latency aspects are explored in this paper. Two complementary strategies to improve the latency of the entire chain are introduced and examined: lowering the resolution of the rendered views and reducing the amount of rendered views by smartly interpolating the missing views at receiver side. These strategies are balanced while optimizing the latency and image quality using a fuzzy logic approach. Furthermore, the optimal view resolution for a multi-view autostereoscopic lenticular display has been determined by investigating the lenticular lattice in the frequency domain. The quantitative aspects of the latency of the proposed building blocks and the resulting image quality have been measured.

56.6L: Late‐News Paper: Modular Multi‐Projection Multi‐View Autostereoscopic Display using MEMS Laser Projectors

AbstractA new modular multi‐projection multi‐user multiview autostereoscopic display structure based on microelectromechanical systems (MEMS) laser projectors is presented. The system contains an array of MEMS laser projectors, a vertical diffuser, and the driving electronics. Each projector provides a slit of the whole image observed through the vertical diffuser. The structure does not require super position of the content among the projectors. Unlike similar type of displays, the size of the projection display hardware decreased dramatically. The structure has the capability to provide horizontal expansion in the screen size, and in number of different perspectives shown. Mentioned capabilities can be enhanced by using more projectors, by increasing the distance between projectors and the screen, and by changing the diffusion angle of the vertical diffuser in the minor axis. The inhouse built prototype with 18 projectors displays an image with a resolution of 234times848, and 36 different perspectives. The size of the image at the screen is 16 cm x 72 cm

Numerical simulation of the optical characteristics of autostereoscopic displays that have an aspherical lens array with a slanted angle

We developed a 3D simulation model describing the optical phenomena on a slanted lenticular surface with aspherical lenses for autostereoscopic displays and analyzed the optical behavior of the multiview autostereoscopic display under actual design conditions by using a 3D simulation model. Optical characteristics, such as 3D crosstalk and 3D luminance differences, are obtained from the simulation of the light distribution for the multiview autostereoscopic displays with slated angles of 0.0°, 9.46°, 12.59°, and 14.04°. By investigating the effect of the conic constant of an aspherical lens surface on the 3D crosstalk and the 3D luminance differences for given several design conditions, we find the optimal values of the conic constant for slanted angles of 0.0° and 9.46° in order to minimize the 3D crosstalk and the 3D luminance difference. From these results, we think that our simulation model is very useful for designing the lens array to optimize the optical characteristics of autostereoscopic displays.

多视点自由立体显示串扰分析与视区仿真

The cause of crosstalk based on multi-view autostereoscopic display was analyzed.The relation between sweet spots and parallax barrier parameters was deduced.Then,analysis of the model of crosstalk was processed respectively in vertical and horizontal directions to the flat-panel displays,which aimed to obtain the distribution characteristics of crosstalk in three-dimensional space before the screen. The simulation results show that viewing zone in vertical direction along the screen is greater than it in horizontal direction.And,each view has multiple observation points without any crosstalk in a horizontal direction parallel to the screen,but in the direction perpendicular to the screen,each view has only one such point.Through the simulation analysis of viewing zone,the accuracy of the crosstalk calculation was further validated.The research results of this paper establish a simulation basic platform for the optimum design and concrete realization of autostereoscopic display system.

Multi-view autostereoscopic projection display using rotating screen

A new technique for multi-view autostereoscopic projection display is proposed, and demonstrated. The technique uses two mobile projectors, a rotating retro-reflective diffuser screen, and a head-tracking camera. As two dynamic viewing slits are created at the viewer's position, the slits can track the position of the eyes by rotating the screen. The display allows a viewer to move approximately 700 mm along the horizontal axis, and 500 mm along the vertical axis with an average crosstalk below 5 %. Two screen prototypes with different diffusers have been tried, and they provide luminance levels of 60 Cd/m2, and 160 Cd/m2 within the viewing field.

최적 입체시를 제공하기 위한 상호작용 기반 다시점 모바일 디스플레이 모의장치 개발

무안경방식 입체 디스플레이에서 유발되는 크로스토크는 최적 입체시를 저해하는 요소로 작용한다. 이런 이유로 디스플레이 개발자들은 최적 입체시를 위해서 반드시 크로스토크를 고려해야 한다. 그러나 다양한 디스플레이의 특성과 시청 환경에 따라서 실제 디스플레이 설계에서 정량적으로 측정하기 어렵다. 따라서 본 논문은 무안경방식 입체 디스플레이에서 발생하는 주요 인자들을 정량적으로 측정하고 이들 상호적인 관계를 정성적으로 분석할 수 있는 시뮬레이터를 개발함으로써 최적의 모바일용 입체시를 가이드 할 수 있는 시스템을 제안한다. 제안된 시스템은 증강 현실(augmented reality) 기술을 이용하여 모바일용 시청 영역 범위를 가이드 한다. 실험 및 시뮬레이션 결과에서, 제안한 시뮬레이터는 모바일 디스플레이를 사용하는 실제 공간에서 발생할 수 있는 시역간 크로스토크를 가시화 하여 디스플레이 개발자들이 상호작용을 통하여 최적의 입체시 제공할 수 있도록 디스플레이를 직관적으로 설계할 수 있도록 한다. Crosstalk has been regarded as one of inherent problems in a factor disturbing optimal viewing experiences. For this reason, display developers address the crosstalk issues. However, due to various display characteristics and viewing environments it is hard to estimate the crosstalk generated in real space. This paper proposes an interactive multi-view display simulator based on augmented reality for developing mobile autostereoscopic displays. This display simulator can be used for the developers to estimate the crosstalk and find optimal viewing zone intuitively. Typical parameters for mobile phone use are applied for the estimation. The proposed simulator provides visualized crosstalk in real space for them to determine better viewing zone interactively. The proposed system is expected to help establish ergonomic guidelines for mobile multi-view autostereoscopic display developers, designers, and viewers.

3D-holoscopic imaging: a new dimension to enhance imaging in minimally invasive therapy in urologic oncology.

Existing imaging modalities of urologic pathology are limited by three-dimensional (3D) representation on a two-dimensional screen. We present 3D-holoscopic imaging as a novel method of representing Digital Imaging and Communications in Medicine data images taken from CT and MRI to produce 3D-holographic representations of anatomy without special eyewear in natural light. 3D-holoscopic technology produces images that are true optical models. This technology is based on physical principles with duplication of light fields. The 3D content is captured in real time with the content viewed by multiple viewers independently of their position, without 3D eyewear. We display 3D-holoscopic anatomy relevant to minimally invasive urologic surgery without the need for 3D eyewear. The results have demonstrated that medical 3D-holoscopic content can be displayed on commercially available multiview auto-stereoscopic display. The next step is validation studies comparing 3D-Holoscopic imaging with conventional imaging.

A multi-point of view 3D camera system for minimally invasive surgery

Endoscopes and laparoscopes in interventional endoscopy and in minimally invasive surgery (MIS) are the eyes of the surgeon. For this reason, research for innovative and more efficient visualization devices is always active. Furthermore, surgeons appreciate innovation and are generally open to new solutions. An example of this attitude is the rapid spreading of stereoscopy in the surgical field. Three-dimensional (3D) endoscopes are already available on the market, but all of them still present several limitations; first of all, their single user feature, which makes the surgeon isolated from the staff. This problem could find a solution in the most recent frontier in 3D vision, i.e. the multi-views autostereoscopic displays (ADs). Indeed, these monitors can be watched by more than one person in different positions without the loss of 3D perspective and without the need of glasses or helmets. However, these devices have not yet found a direct application in MIS, mainly because of the encumbrance of the image acquisition system, which hardly fits in the patient anatomy. The most challenging issue is to embed multiple cameras in a device which is able to pass through a laparoscopic incision. The aim of this work is the development of a miniaturized acquisition system for MIS, which can be interfaced with multi-view ADs, thus paving the way to the use of such devices in surgery. A robust and simple solution identified by the authors in the implementation of a module which can be inserted in the longitudinal direction, while anchored and maneuvered in the horizontal one. The presented laparoscope embeds up to seven cameras, while matching MIS access incision constraints.