Disparity Estimation Technique Research Articles

3D reconstruction by a combined structure tensor and Hough transform light field approach

Abstract Disparity estimation using the structure tensor is a local approach to determine orientation in Epipolar Plane Images. A global extension would lead to more precise and robust estimations. In this work, a novel algorithm for 3D reconstruction from linear light fields is proposed. This method uses a modified version of the Progressive Probabilistic Hough Transform to extract orientations from Epipolar Plane Images, allowing to achieve high quality disparity maps. To this aim, the structure tensor estimates are used to speed up computation and improve the disparity estimation near occlusion boundaries. The new algorithm is evaluated on both synthetic and real light field datasets, and compared with classical local disparity estimation techniques based on the structure tensor.

Distributed perceptual compressed sensing framework for multiview images

A perceptual-based compressed sensing (CS), which focuses the measurements and the recovery on the visually important low-frequency coefficients, is applied for multi-view image signals. High correlation among different views is exploited to generate signal prediction using disparity estimation and compensation techniques. A residual-based recovery is utilised as a joint recovery for the non-reference images to enhance the reconstruction performance. The proposed framework shows remarkable performance improvement over the conventional CS with joint recovery as well as the perceptual-based CS with independent recovery.

Hand-held 3D light field photography and applications

We propose a method to acquire 3D light fields using a hand-held camera, and describe several computational photography applications facilitated by our approach. As our input we take an image sequence from a camera translating along an approximately linear path with limited camera rotations. Users can acquire such data easily in a few seconds by moving a hand-held camera. We include a novel approach to resample the input into regularly sampled 3D light fields by aligning them in the spatio-temporal domain, and a technique for high-quality disparity estimation from light fields. We show applications including digital refocusing and synthetic aperture blur, foreground removal, selective colorization, and others.

On multi-view video segmentation for object-based coding

A novel scheme for multi-view segmentation and tracking is proposed aiming to acquire perceptually consistent results for object-based coding. Firstly, a classic image segmentation technique is employed to perform initial segmentation to divide the whole image into spatially homogeneous regions. Secondly, the motion information is extracted based on frame differences and the disparity information is derived by employing a classic disparity estimation technique. Thirdly, a novel scheme is proposed to perform merging of the initial segmentation results based on both motion and disparity information to remove over-segmented regions and extract perceptually consistent semantic objects. Finally, a contour-based tracking algorithm is proposed to implement accurate and robust object tracking along both temporal and view directions. Experiments are conducted and the results demonstrate that the proposed scheme is effective and, compared with the existing technique, it can acquire more perceptually consistent results.

Stereo image reconstruction using regularized adaptive disparity estimation

In this paper, stereo image reconstruction using regularized adaptive disparity estimation is proposed. That is, by adaptively predicting the mutual correlation between stereo images pair using the proposed algorithm, the bandwidth of stereo input images pair can be compressed to the level of a conventional 2D image and a predicted image also can be effectively reconstructed using a reference image and disparity vectors. Especially, in the proposed algorithm, the first feature values are extracted from the input stereo images pair. Then, a matching window for stereo matching is adaptively selected depending on the magnitude of these feature values. That is, for the region having larger feature values, a smaller matching window is selected, while, for the opposite case, a larger matching window is selected by comparing predetermined threshold values. This approach is not only able to reduce a mismatching of disparity vectors, which occurs in the conventional dense disparity estimation with a small matching window, but is also able to reduce blocking effects that occur in the coarse disparity estimation with a large matching window. In addition, in this paper, a new regularized adaptive disparity estimation technique is proposed. That is, by regularizing the estimated disparity vector with the neighboring disparity vectors, problems of the conventional adaptive disparity estimation scheme might be solved, and the predicted stereo image can be more effectively reconstructed. From experiments using stereo sequences of “Man”, “Fichier”, “Manege”, and “Tunnel”, it is shown that the proposed algorithm improves the PSNRs of a reconstructed image to about 6.90 dB on average at ±30 search ranges as compared to those of conventional algorithms. Also, it is found that there is almost no difference between an original image and a reconstructed image through the proposed algorithm by comparison to that of conventional algorithms.

3D reconstruction from stereo images for interactions between real and virtual objects

The 3D reconstruction algorithm in a stereo image pair for realizing mutual occlusion and interactions between the real and virtual world in an image synthesis is proposed. A two-stage algorithm, consisting of disparity estimation and regularization is used to locate a smooth and precise disparity vector. The hierarchical disparity estimation technique increases the efficiency and reliability of the estimation process, and edge-preserving disparity field regularization produces smooth disparity fields while preserving discontinuities that result from object boundaries. Depth information concerning the real scene is then recovered from the estimated disparity fields by stereo camera geometry. Simulation results show that the proposed algorithm provides accurate and spatially correlated disparity vector fields in various types of images, and the reconstructed 3D model produces a natural space in which the real world and virtual objects interact with each other as if they were in the same world.

Fast technique for phase-based disparity estimation with no explicit calculation of phase

A simpler and faster technique for depth estimation, based on phase measurements of disparity, is presented. The technique provides direct evaluations of phase differences, avoids explicit calculations of single phases and the attendant problem of phase wrapping, and is suitable for efficient software and hardware implementations.

Sequential construction of 3-D-based scene description

Binocular camera systems are commonly used to construct 3-D-based scene description. However, there is a tradeoff between the length of the camera baseline and the difficulty of the matching problem and the extent of the field of view of the 3-D scene. A large baseline system provides better depth resolution than a smaller baseline system at the expense of a narrower field of view. To increase the depth resolution without increasing the difficulty of the matching problem and decreasing the field of view of the 3-D scene, a sequential 3-D-based scene description technique is proposed. Multiple small-baseline 3-D scene descriptions from a single moving camera or an array of cameras are used to sequentially construct a large baseline 3-D scene description while maintaining the field of view of a small-baseline system. A Bayesian framework using a disparity-space image (DSI) technique for disparity estimation is presented. The cost function for large baseline image matching is designed based not only on the photometric matching error, the smoothness constraint, and the ordering constraint, but also on the previous disparity estimates from smaller baseline stereo image pairs as a prior model. Texture information is registered along the scan path of the camera(s). Experimental results demonstrate the effectiveness of this technique in visual communication applications.

High accuracy solution to correspondence problem

A probabilistic relaxation technique for high accuracy disparity estimation is proposed. The relaxation rule is defined according to the likelihood that estimates obey physically meaningful constraints. High accuracy is achieved by applying the approach to points inside salient image areas previously approximated by second-order polynomials.

ADAPTIVE GABOR FILTERS FOR PHASE-BASED DISPARITY ESTIMATION

The phase-difference-based technique has become a widespread method for depth and optical flow estimation because of superior performance and better theoretical groundings. The technique is based on the convolution of the stereo image pair with Gabor filters. Gabor filters contains two parameters, the width and the tuning frequency. In order to optimize performance, these parameters have to be chosen in accordance to the characteristics of the visual signal. In this article we propose an automatic technique to locally adapt the filter parameters to the input signal. In the first part, we analyze the performance of the phase-difference-based technique for disparity estimation with respect to the choice of the Gabor filter parameters. In particular we characterize the effects of phase nonlinearities on the quality of disparity estimates. In the second part, a novel technique is introduced that reduces phase nonlinearity by means of an adaptive mechanism for the tuning frequency. The performance improvement produced by the adaptive filter is demonstrated using different types of images. Results show the proposed technique allows a significant improvement of disparity estimation.

Dense disparity estimation with a divide-and-conquer disparity space image technique

A new divide-and-conquer technique for disparity estimation is proposed in this paper. This technique performs feature matching following the high confidence first principle, starting with the strongest feature point in the stereo pair of scanlines. Once the first matching pair is established, the ordering constraint in disparity estimation allows the original intra-scanline matching problem to be divided into two smaller subproblems. Each subproblem can then be solved recursively until there is no reliable feature point within the subintervals. This technique is very efficient for dense disparity map estimation for stereo images with rich features. For general scenes, this technique can be paired up with the disparity-space image (DSI) technique to compute dense disparity maps with integrated occlusion detection. In this approach, the divide-and-conquer part of the algorithm handles the matching of stronger features and the DSI-based technique handles the matching of pixels in between feature points and the detection of occlusions. An extension to the standard disparity-space technique is also presented to compliment the divide-and-conquer algorithm. Experiments demonstrate the effectiveness of the proposed divide-and-conquer DSI algorithm.

Edge-preserving directional regularization technique for disparity estimation of stereoscopic images

Due to the advances in visual technology three dimensional (3D) imaging systems are becoming popular. One way of stimulating 3D perception is to use stereo pairs, a pair of images of the same scene acquired from different perspectives. Since there is an inherent redundancy between the images of a stereo pair, data compression algorithms should be employed to efficiently represent stereo pairs. Disparity estimation of stereoscopic image is an important stage for the stereo image compression. We propose a new disparity estimation method using a directional regularization technique to preserve edges well for stereoscopic image coding. This method smooths disparity vectors in smooth regions and preserves edges in object boundaries without over-smoothing problem. The experimental results show that the proposed method achieves close matches between a left image and a right image as well as the improved coding efficiency. We use DPCM technique followed by Huffman entropy coding for disparity map coding. In addition, the proposed disparity estimation method can be applied to the high-quality intermediate view reconstruction.