Stereo Disparity Computation Research Articles

Leveraging cost matrix structure for hardware implementation of stereo disparity computation using dynamic programming

Dynamic programming is a powerful method for solving energy minimisation problems in computer vision, for example stereo disparity computations. While it may be desirable to implement this algorithm in hardware to achieve frame-rate processing, a naı¨ve implementation may fail to meet timing requirements. In this paper, the structure of the cost matrix is examined to provide improved methods of hardware implementation. It is noted that by computing cost matrix entries along anti-diagonals instead of rows, the cost matrix entries can be computed in a pipelined architecture. Further, if only a subset of the cost matrix needs to be considered, for example by placing limits on the disparity range (include neglecting negative disparities by assuming rectified images), the resources required to compute the cost matrix in parallel can be reduced. Boundary conditions required to allow computing a subset of the cost matrix are detailed. Finally, a hardware solution of Cox’s maximum-likelihood, dynamic programming stereo disparity algorithm is implemented to demonstrate the performance achieved. The design provides high frame rate (>123fps) estimates for a large disparity range (e.g. 128 pixels), for image sizes of 640×480 pixels, and can be simply extended to work well over 200fps.

Embedded planar surface segmentation system for stereo images

An embedded system is developed to segment stereo images using disparity. The recent developments in the embedded system architecture have allowed real time implementation of low-level vision tasks such as stereo disparity computation. At the same time, an intermediate level task such as segmentation is rarely attempted in an embedded system. To solve the planar surface segmentation problem, which is iterative in nature, our system implements a Segmentation–Estimation framework. In the segmentation phase, segmentation labels are assigned based on the underlying plane parameters. Connected component analysis is carried out on the segmentation result to select the largest spatially connected area for each plane. From the largest areas, the parameters for each plane are reestimated. This iterative process was implemented on TMS320DM642 based embedded system that operates at 3–5 frames per second on images of size 320 × 240.

A Surface Reconstruction Method Using Global Graph Cut Optimization

Surface reconstruction from multiple calibrated images has been mainly approached using local methods, either as a continuous optimization problem driven by level sets, or by discrete volumetric methods such as space carving. We propose a direct surface reconstruction approach which starts from a continuous geometric functional that is minimized up to a discretization by a global graph-cut algorithm operating on a 3D embedded graph. The method is related to the stereo disparity computation based on graph-cut formulation, but fundamentally different in two aspects. First, existing stereo disparity methods are only interested in obtaining layers of constant disparity, while we focus on high resolution surface geometry. Second, most of the existing graph-cut algorithms only reach approximate solutions, while we guarantee a global minimum. The whole procedure is consistently incorporated into a voxel representation that handles both occlusions and discontinuities. We demonstrate our algorithm on real sequences, yielding remarkably detailed surface geometry up to 1/10th of a pixel.

A hybrid approach for stereo disparity computation

An alternative, hybrid approach for disparity estimation, based on the phase difference technique, is presented. The proposed technique combines the robustness of the matching method with the sub-pixel accuracy of the phase difference approach. A matching between the phases of the left and right signals is introduced in order to allow the phase difference method to work in a reduced disparity range. In this framework, a new criterion to detect signal singularities is proposed. The presented test cases show that the performance of the proposed technique in terms of accuracy and density of the disparity estimates has greatly improved.

Cortical computation of stereo disparity.

Our ability to see the world in depth is a major accomplishment of the brain. Previous models of how positionally disparate cues to the two eyes are binocularly matched limit possible matches by invoking uniqueness and continuity constraints. These approaches cannot explain data wherein uniqueness fails and changes in contrast alter depth percepts, or where surface discontinuities cause surfaces to be seen in depth, although they are registered by only one eye (da Vinci stereopsis). A new stereopsis model explains these depth percepts by proposing how cortical complex cells binocularly filter their inputs and how monocular and binocular complex cells compete to determine the winning depth signals.

Geometric constraints and stereo disparity computation

Most stereo techniques compute disparity assuming that it varies slowly along surfaces. We quantify and justify this assumption, using weak assumptions about surface orientation distributions in the world to derive the density of disparity surface orientations. The small disparity change assumption is justified by the orientation density's heavy bias toward disparity surfaces that are nearly parallel to the image plane. In addition, the bias strengthens with smaller baselines, larger focal lengths, and as surfaces move farther from the cameras. To analyze current stereo techniques, we derive three densities from the first density, those of the disparity gradient magnitude, the directional derivative of disparity, and the difference in disparity between neighboring surface points. The latter may be used in Bayesian algorithms computing dense disparity fields. The directional derivative density and the disparity difference density both show that feature-based algorithms should strongly favor small disparity changes, contrary to several well-known algorithms. Finally, we use our original surface orientation density and the gradient magnitude density to derive two new “surfaces-from-stereo” techniques, techniques combining feature-based matching and surface reconstruction. The first uses the densities to severely restrict the search range for the optimum fit. The second incorporates the surface orientation density into the optimization criteria, producing a Bayesian formulation. Both algorithms are shown to be efficient and effective.

Computing Stereo Disparity and Motion with Known Binocular Cell Properties

Many models for stereo disparity computation have been proposed, but few can be said to be truly biological. There is also a rich literature devoted to physiological studies of stereopsis. Cells sensitive to binocular disparity have been found in the visual cortex, but it is not clear whether these cells could be used to compute disparity maps from stereograms. Here we propose a model for biological stereo vision based on known receptive field profiles of binocular cells in the visual cortex and provide the first demonstration that these cells could effectively solve random dot stereograms. Our model also allows a natural integration of stereo vision and motion detection. This may help explain the existence of units tuned to both disparity and motion in the visual cortex.

A regularization model for stereo vision with controlled continuity

The problem of the computation of stereo disparity is approached as a mathematically ill-posed problem by using regularization theory. A controlled continuity constraint which provides a local spatial control over the smoothness of the solution enables the problem to be regularized while preserving the disparity discontinuities. The discontinuities are localized during the regularization process by examining the size of the disparity gradient at the gray value edges. An iterative algorithm for the computation of stereo disparity is obtained, and a computer experiment with synthetic data is shown.

Computation of stereo disparity using regularization

The computation of stereo disparity is a mathematically ill-posed problem. However, using regularization theory it may be transformed into a well-posed problem. Standard regularization can be employed to solve ill-posed problems by using stabilizing functionals that impose global smoothness constraints on acceptable solutions. However, the presence of depth discontinuities causes serious difficulties in standard regularization, since smoothness assumptions do not hold across discontinuities. This paper presents a regularization approach to stereopsis based on controlled-continuity stabilizing functionals. These functionals provides a spatial control over smoothness, allowing the introduction of discontinuities into the solution. An iterative method for the computation of stereo disparity is derived, and the result of a computer simulation with a synthetic stereo pair of images is shown.