Autostereoscopic Research Articles

Outcome and Safety of Radical Resection in Non-Small Cell Lung Cancer Patients via Glasses-Free 3-Dimensional Video-Assisted Thoracoscope Versus 2-Dimensional Video-Assisted Thoracoscope.

The investigation was aimed to evaluate the safety and efficacy of glasses-free 3-dimensional (3D) video-assisted thoracoscopic surgery (VATS) versus 2D VATS for radical resection of non-small cell lung cancer (NSCLC). We reviewed the clinical data of patients with pathologically proven NSCLC who underwent glasses-free 3D (the 3D group) and 2D VATS radical lobectomy (the 2D group) with systematic lymph node dissection. The outcomes of this study included operative characteristics and safety of 2D and 3D VATS, and duration of lymphadenectomy of right stations 2 and 4. A total of 190 patients were eligible for the study. The 2D group consisted of 108 patients while the 3D group included 82 patients. The 2 groups were comparable in demographic and baseline variables ( P > .05). The median number of resected lymph nodes was 19 in both groups ( P = .583). The median length of hospital stay was comparable between the 2 groups (2D, 7 days vs 3D, 8 days; P = .167). No operative mortality was reported in either group. Complications developed in 21 (19.4%) patients in the 2D group and 14 (17.1%) in the 3D group ( P = .710). A subgroup analysis of patients who underwent right station 2 and 4 lymphadenectomy showed that the mean time for right station 2 and 4 lymph node dissection was significantly shorter in the 3D group than in the 2D group (3D, 430.9 ± 237.2 vs 2D, 648.6 ± 364.1 seconds; P < .001). Glasses-free 3D VATS and 2D VATS are comparable in operative characteristics and safety profile for radical resection of NSCLC. Glasses-free 3D visualization facilitates more rapid right-sided mediastinal lymphadenectomy.

自由立体显示器定向背光侧面发光光纤

自由立体显示器定向背光侧面发光光纤

Efficient Light Field Images Compression Method Based on Depth Estimation and Optimization

Massive recent investigations from both industry and academia have been poured into autostereoscopic display. One of the emerging techniques, light field image (LFI), can provide more immersive perception by increasing the number of views and the spatial resolution. However, these advantages restrict the storage and transmission due to such dense-view image simultaneously. To solve this problem, we propose to compress the LFI using multiview video plus depth (MVD) coding architecture. In this paper, we preliminarily estimate the depth based on the concept of epipolar plane image. To achieve a depth value, we design an optimal slope decision algorithm to determine the best slope with the minimal cost. While this rough estimation produces some error points within initial depth map, therefore, we present a depth optimization algorithm using the characteristic of the associated texture image. Ultimately, a small number of selected viewpoint images are encoded with their corresponding depth maps using the MVD framework, and then, the unselected viewpoint images are synthesized by depth image-based rendering technique. To verify the validity of the proposed LFI compression scheme, extensive experiments are conducted. The simulated results demonstrate that the proposed depth map estimation algorithm is superior to other state-of-the-art methods for the LFI. Meanwhile, our LFI compression method outperforms other LFI compression algorithms significantly.

Design of autostereoscopic 3D display with full resolution by means of polarization gratings

Autostereoscopic 3D, a kind of display technology, can give people an amazing visual experience. A design of autostereoscopic 3D display based on the polarization gratings is proposed in this paper. The designed system is placed on the outside surface of LCD. Image from two video sources which have parallax, can be alternately displayed on the LCD screen at a rate of 48 frames per second, and owing to polarization gratings, the direction of emergent light of the system can be changed which depends on the handedness of the incident light. By using the unique diffraction characteristics of the polarization gratings and fast response of liquid crystal, the image from two different video sources enters people’s left and right eye respectively. Due to the existence of parallax, people can perceive depth of the image. The system is based on the time division multiplexing of LCD pixels, so there is no pixel loss and image with full resolution can be used.

Fractal tomography and its application in 3D vision

A three-dimensional artistic fractal tomography method that implements a non-glasses 3D visualization of fractal worlds in layered media is proposed. It is designed for the glasses-free 3D vision of digital art objects and films containing fractal content. Prospects for the development of this method in art galleries and the film industry are considered.

Horizontal white light flat-topped beams produced by the diffraction mask

The flat-topped beam is a special beam with wide applications in the directional backlight autostereoscopic display, and it is used as a directional backlight in the horizontal direction. However, it is still challenge to white light flat-topped beams with the traditional flat-topped beam shaper. In this paper, it is proposed that diffraction mask with butterfly-shaped hole arrays and cylindrical lens be used to produce the horizontal flat-topped white beams. The surface of the LCD backlight mask is covered by a layer of diffraction mask, where the butterfly-shaped holes are arranged in line along the vertical direction. Simultaneously, the height and width, hole center height are kept identical, and the ratio between the center depth and the perimeter of butterfly-shape hole is defined as the concavity. A flat convex cylindrical lens is placed in front of diffraction mask gaplessly. The uniform light field from LCD backlight is transformed into the white light flat-topped beams and projected on the receiving screen by the diffraction mask and cylindrical lens. Based on the Huygens-Fresnel diffraction integral, the intensity distribution formula of diffraction of the single wavelength light source on the receiving screen is derived. Furthermore, the intensity distribution formula on the screen is derived by super positioning the multiple wavelengths. The proposed method is verified by both numerical simulation and experimental validation. Numerical simulations elucidate the effects of the different transmission distance and butterfly hole concavity on the white light flat-topped beam flat-topped factor. The stimulated results show that the propagation distance does not influence the white light beam transverse intensity distribution characteristics of flat top. With the beam propagation distance increasing, the horizontal width of flat-topped beam becomes larger. When the concavity of the butterfly hole decreases, light intensity distribution shifts from Gaussian to flat type. However, the flat-topped factor decreases when the butterfly concavity is too small. The optimal concavity varies from 0.4 to 0.6, where the flattened factor of the transverse flat-topped beams reaches 0.89. In the experiments, films are produced with the diffraction of butterfly hole array mask. The height and width of butterfly are both 48 μm, and the concavities of the butterfly are 1, 0.83, 0.66 and 0.83 respectively. The cylindrical lens adopts PMMA cylindrical lens grating plate, with a thickness of 8 mm, a grating density for 18 line/inch, and the cylindrical lens curvature radius R is 2.67 mm. The experimental results show that the beam transmission is consistent with the result of numerical simulation. When concavity of the butterfly is 0.5, the flat factors of the white light horizontal of flat-topped beams are higher than 0.89 in a range from 500 mm to 2000 mm. Moreover, we also discuss the dispersion effects of shaft flat-topped beams and off-axis flat-topped beams, showing that the refraction and dispersion of the cylindrical lens can cancel out each other, so that the horizontal flat-toped white beams is basically dispersionless.

Education and training for laparoscopic gastrointestinal surgery: our 10 years' experience in Nanfang Hospital

The laparoscopic surgery for gastrointestinal cancer developed slowly and was at a crossroad of choice at the beginning of the 21st century. However, the team of laparoscopic surgery in Nanfang Hospital was keenly conscious that minimally invasive surgery (MIS) would bring new era to the treatment of gastrointestinal cancer. Therefore, our team went into the exploration of laparoscopic surgery for gastrointestinal cancer: (1) researching a series of anatomical theories for MIS; (2) lucubrating the applicable pattern of fascia and mesentery under laparoscopic view; (3) finding out the precise anatomical landmarks and surgical layers; (4) optimizing the operative strategy. Fortunately, we proposed a safe and simplified strategy of laparoscopic gastrointestinal cancer surgery for Chinese patients with locally advanced stage. Gradually, this strategy was widely adopted by most colleagues in this field. Meanwhile, our team realized the necessity and urgency of education and training for primary care physicians, thus we designed courses based on different laparoscopic levels of the trainees. Also we actively developed the teaching model suitable for the presentation of visual surgery, by taking advantages of mobile network and glasses-free 3D, to break through the limit of time and space in teaching and learning. Besides, we used the internet to create an education system of real-time, opening, practical and efficient academic communication platform, so that more surgeons across the country would be able to synchronize and interact with the experts more instantly and efficiently. All the way, our team hammered at optimizing laparoscopic surgery procedures, along with further perfecting and standardizing training and education system. This article intends to review, summarize and share our experiences in laparoscopic training and education for gastrointestinal surgery, also to remind ourselves of staying true and carry on in this field.

Autostereoscopic 3D display system with dynamic fusion of the viewing zone under eye tracking: principles, setup, and evaluation [Invited

We study optical technologies for viewer-tracked autostereoscopic 3D display (VTA3D), which provides improved 3D image quality and extended viewing range. In particular, we utilize a technique-the so-called dynamic fusion of viewing zone (DFVZ)-for each 3D optical line to realize image quality equivalent to that achievable at optimal viewing distance, even when a viewer is moving in a depth direction. In addition, we examine quantitative properties of viewing zones provided by the VTA3D system that adopted DFVZ, revealing that the optimal viewing zone can be formed at viewer position. Last, we show that the comfort zone is extended due to DFVZ. This is demonstrated by a viewer's subjective evaluation of the 3D display system that employs both multiview autostereoscopic 3D display and DFVZ.

A Novel Hybrid Kinect-Variety-Based High-Quality Multiview Rendering Scheme for Glass-Free 3D Displays

This paper presents a new hybrid Kinect-variety-based synthesis scheme that renders artifact-free multiple views for autostereoscopic/automultiscopic displays. The proposed approach does not explicitly require dense scene depth information for synthesizing novel views from arbitrary viewpoints. Instead, the integrated framework first constructs a consistent minimal image–space parameterization of the underlying 3D scene. The compact representation of scene structure is formed using only implicit sparse depth information of a few reference scene points extracted from raw RGB depth data. The views from arbitrary positions can be inferred by moving the novel camera in parameterized space by enforcing Euclidean constraints on reference scene images under a full-perspective projection model. Unlike the state-of-the-art depth image-based rendering (DIBR) methods, in which input depth map accuracy is crucial for high-quality output, our proposed algorithm does not depend on precise per-pixel geometry information. Therefore, it simply sidesteps to recover and refine the incomplete or noisy depth estimates with advanced filling or upscaling techniques. Our approach performs fairly well in unconstrained indoor/outdoor environments, where the performance of range sensors or dense depth-based algorithms could be seriously affected due to scene complex geometric conditions. We demonstrate that the proposed hybrid scheme provides guarantees on the completeness, optimality with respect to the inter-view consistency of the algorithm. In the experimental validation, we performed a quantitative evaluation as well as subjective assessment of the scene with complex geometric or surface properties. A comparison with the latest representative DIBR methods is additionally performed to demonstrate the superior performance of the proposed scheme.

3DTV at home

Stereoscopic 3D (S3D) movies have become widely popular in the movie theaters, but the adoption of S3D at home is low even though most TV sets support S3D. It is widely believed that S3D with glasses is not the right approach for the home. A much more appealing approach is to use automulti-scopic displays that provide a glasses-free 3D experience to multiple viewers. A technical challenge is the lack of native multiview content that is required to deliver a proper view of the scene for every viewpoint. Our approach takes advantage of the abundance of stereoscopic 3D movies. We propose a real-time system that can convert stereoscopic video to a high-quality multiview video that can be directly fed to automultiscopic displays. Our algorithm uses a wavelet-based decomposition of stereoscopic images with per-wavelet disparity estimation. A key to our solution lies in combining Lagrangian and Eulerian approaches for both the disparity estimation and novel view synthesis, which leverages the complementary advantages of both techniques. The solution preserves all the features of Eulerian methods, e.g., subpixel accuracy, high performance, robustness to ambiguous depth cases, and easy integration of inter-view aliasing while maintaining the advantages of Lagrangian approaches, e.g., robustness to large disparities and possibility of performing non-trivial disparity manipulations through both view extrapolation and interpolation. The method achieves real-time performance on current GPUs. Its design also enables an easy hardware implementation that is demonstrated using a field-programmable gate array. We analyze the visual quality and robustness of our technique on a number of synthetic and real-world examples. We also perform a user experiment which demonstrates benefits of the technique when compared to existing solutions.

The First Depiction of Two Superimposed Geographical Surfaces in One Autostereoscopic Map: Antarctica’s Topography and the Southern Ocean Seafloor

ABSTRACTWithin the past 10–15 years true-3D lenticular cartography has experienced a remarkable boost. In the course of this development, besides studies into its potential for thematic-cartographic representations, the synoptic depiction of physical surfaces (‘geo[graphical] surfaces’) has been playing a significant role. In this context the innovative holistic display of complex morphological and topographical conditions is of particular interest. The simultaneous representation of various cartographic parameters at different depths will deliver an enormous surplus of information transfer in the field of thematic cartography as well as in physical/topographic cartography. This paper describes the methodological development and generation of an autostereoscopic hardcopy display of Antarctic topography. The purpose was the simultaneous depiction of the two superimposed surfaces of both the ice-sheet and the rock-bed and, in addition, of the surrounding seafloor, thus displaying a vertical drop of more than 12,000 m.

Quality Assessment of Perceptual Crosstalk on Two-View Auto-Stereoscopic Displays.

Crosstalk is one of the most severe factors affecting the perceived quality of stereoscopic 3D images. It arises from a leakage of light intensity between multiple views, as in auto-stereoscopic displays. Well-known determinants of crosstalk include the co-location contrast and disparity of the left and right images, which have been dealt with in prior studies. However, when a natural stereo image that contains complex naturalistic spatial characteristics is viewed on an auto-stereoscopic display, other factors may also play an important role in the perception of crosstalk. Here, we describe a new way of predicting the perceived severity of crosstalk, which we call the Binocular Perceptual Crosstalk Predictor (BPCP). BPCP uses measurements of three complementary 3D image properties (texture, structural duplication, and binocular summation) in combination with two well-known factors (co-location contrast and disparity) to make predictions of crosstalk on two-view auto-stereoscopic displays. The new BPCP model includes two masking algorithms and a binocular pooling method. We explore a new masking phenomenon that we call duplicated structure masking, which arises from structural correlations between the original and distorted objects. We also utilize an advanced binocular summation model to develop a binocular pooling algorithm. Our experimental results indicate that BPCP achieves high correlations against subjective test results, improving upon those delivered by previous crosstalk prediction models.

Pixel arrangement optimization of two-dimensional light-emitting diode panel for low-crosstalk autostereoscopic light-emitting diode displays

Abstract. We propose an effective and efficient method that reduces the crosstalk level in autostereoscopiclight-emitting diode (LED) displays by optimizing the pixel arrangement of the associated two-dimensional(2-D) LED panel. In the proposed method, first a series of parallax barrier patterns, based on predesignatedLED packaging units, are designed and simulated by sequentially regulating the width of black stripes onthe 2-D LED panel. This design principle removes the black stripes from conventional autostereoscopicLED display pixels for optimal parallax barrier calculation. Furthermore, the mathematical relationship betweenaveragecrosstalklevelandvisualfluxdensityisobtainedfromthesimulation.Amathematicalfittingmethodthatincludes a cubic fitting function is finally applied to achieve proper pixel arrangement in the 2-D LED panel. Thesimulation results obtained for a dual viewpoint autostereoscopic LED display system indicate that the mostsuitable valuefor the widthof the blackstripesiswithin the rangeof 3.17604to 3.34277 mm,withalight-emittingpixelwidthof2mm.Thismethodcaneffectivelyguidethe2-DLEDpanel’sdesignandresultinhighperformanceautostereoscopicthree-dimensionalLEDdisplays,whichwillhavebroadapplicationprospectsinthenearfuture.

Quality of experience measurement of compressed multi-view video

Quality of Experience (QoE) measurement in stereoscopic 3D is a challenging topic and existing standards such as ITU-R BT.500 may no longer be adequate for reliability and efficiency. Yet, objective measurement of QoE is not well researched for 3D images and videos. We present a new subjective assessment methodology providing better time-efficiency and more accurate scores at the same or even less number of assessors, using new objective metrics for QoE measurement of compressed multi-view video. Our research focus is twofold. First, the novel subjective testing methodology is developed, and QoE of different multi-view video compression technologies is evaluated. A time-efficient solution is proposed as a multiple stimuli plus simultaneous presentation extension of the conventional Double Stimulus Continuous Quality Scale (DSCQS) method. Second, objective quality metrics are proposed based on structural similarity and depth maps. Experiments are conducted on an auto-stereoscopic display while H.264/MPEG-4 Advanced Video Coding (AVC)- and High Efficiency Video Coding (HEVC)-based encoders are used for compression. The results demonstrate that the new methodology is superior to the DSCQS method in terms of both evaluation time and accuracy. Furthermore, high correlation is achieved with the new objective quality metrics.

CrossbowCam: a handheld adjustable multi-camera system

This paper presents a novel multi-functional, low-cost handheld multi-camera system (one dimensional camera array) - “CrossbowCam”. The CrossbowCam is suitable for multi-viewpoint image acquisition, smooth switching, alignment and seamless stitching applications. The proposed system differs from the traditional fixed image acquisition systems which are large-sized, high-priced, single functional, and can only captured images at specific locations. With the proposed system, the users can push one single button to change the configuration of the camera array rapidly to divergence (convex arc), parallel (linear), or convergence (concave arc). The three camera configurations can each be suitable for applications such as panorama image stitching, autostereoscopic 3D display, bullet-time (time-freeze) visual effect, 3D scene reconstruction, etc. To rapidly acquire the relationship among cameras after configuration change, we propose a two-stage calibration method to compensate the mechanical misalignment. The first stage adopts the traditional checkerboard calibration method to get the intrinsic parameters (focal length, principal point) and the lens distortion for each camera. The second stage requires no auxiliary tool but utilizes a large number of common feature points from multiple viewpoint images to acquire the extrinsic parameters (translation and rotation matrix) and to compensate the vertical misalignment and the horizontal uneven angle distribution due to the mechanical structure. The proposed system can then insert virtual viewpoint images between actual viewpoint images to allow the viewpoint switching more smoothly. The proposed system has eight cameras with maximum viewing angle of 90° in divergence mode, 38 mm spacing in parallel mode, and imaging radius of 10 m ∼ 0.5 m in convergence mode. We believe that the proposed system can potentially change the consumer habits and becomes the new type of home-use handheld camcorder system in the future.

Video synthesis from stereo videos with iterative depth refinement

We propose a novel depth maps refinement algorithm and generate multi-view video sequences from two-view video sequences for modern autostereoscopic display. In order to generate realistic contents for virtual views, high-quality depth maps are very critical to the view synthesis results. Therefore, refining the depth maps is the main challenging problem in the task. We propose an iterative depth refinement algorithm, including error detection and error correction, to correct errors in depth map. Error detection aims at two types of error: across-view color-depth-inconsistency error and local color-depth-inconsistency error. Then, error pixels are corrected based on sampling local candidates. A trilateral filter that considers intensity, spatial and temporal terms into the filter weighting is applied to enhance the spatial and temporal consistency across frames. So the virtual views can be better synthesized according to the refined depth maps. To combine both warped images, disparity-based view interpolation is introduced to alleviate the translucent artifacts. Finally, a directional filter is applied to reduce the aliasing around the object boundaries to generate multiple high-quality virtual views between the two views. We demonstrate the superior image quality of the synthesized virtual views by using the proposed algorithm over the state-of-the-art view synthesis methods through experiments on benchmarking image and video datasets.

3D interactive surgical visualization system using mobile spatial information acquisition and autostereoscopic display

Three-dimensional (3D) visualization of preoperative and intraoperative medical information becomes more and more important in minimally invasive surgery. We develop a 3D interactive surgical visualization system using mobile spatial information acquisition and autostereoscopic display for surgeons to observe surgical target intuitively. The spatial information of regions of interest (ROIs) is captured by the mobile device and transferred to a server for further image processing. Triangular patches of intraoperative data with texture are calculated with a dimension-reduced triangulation algorithm and a projection-weighted mapping algorithm. A point cloud selection-based warm-start iterative closest point (ICP) algorithm is also developed for fusion of the reconstructed 3D intraoperative image and the preoperative image. The fusion images are rendered for 3D autostereoscopic display using integral videography (IV) technology. Moreover, 3D visualization of medical image corresponding to observer’s viewing direction is updated automatically using mutual information registration method. Experimental results show that the spatial position error between the IV-based 3D autostereoscopic fusion image and the actual object was 0.38±0.92mm (n=5). The system can be utilized in telemedicine, operating education, surgical planning, navigation, etc. to acquire spatial information conveniently and display surgical information intuitively.

P‐100: Approach on Two‐Step 3D Warping with Background Filling for Multi‐View Autostereoscopic Display

We propose an approach on two‐step 3D warping with background filling for the multi‐view autostereoscopic display. The method can obtain high quality virtual view images, and the multi‐view autostereoscopic image is synthesized by using reference images and virtual view images. The experimental results show the effectiveness of the method for the autostereoscopic display.

Visual comfort in autostereoscopic display

AbstractAutostereoscopic displays are likely to become widely used products in the future. However, certain physiological factors, especially visual comfort, limit their development. In this study, four observational parameters – ambient illuminance, image content, scaling ratio, and horizontal distance between major and minor objects – were evaluated to determine the degree of visual comfort offered by 3D computer‐generated images on an autostereoscopic display. Visual comfort score with the range of 0–1 is designed to represent the degree of visual comfort for the 3D images with different manipulations of ambient illuminance, image content, scaling ratio, and horizontal distance between major and minor objects in this study. Subjects were asked to indicate images that produced discomfort. The proportion of images for each condition where participants indicated that viewing the image was comfortable was computed. Images receiving a proportion of 0.5 or greater were classified as acceptable. The disparity ranges over which acceptable images were attained for each participant and for each condition were analyzed with analysis of variance. The analytical results indicate that ambient illuminance and image content have a significant effect on the acceptable disparity range, while scaling ratio and horizontal distance between major and minor objects did not.

P‐97: Parameters Design and Viewing Zone Analysis of 360‐degree Cylindrical Autostereoscopic 3D Display with Novel View‐pixels Arrangement

In this research, the parallax barrier technique is applied on a static cylindrical display to achieve the 360‐degree autostereoscopic 3D display. New parameter design rules and a novel view‐pixels arrangement to improve observation quality are proposed. We simulated the luminance distributions to demonstrate the results with different aperture ratios.