Stereo Correspondence Algorithms Research Articles

Hardware implementation of stereo vision algorithms for depth estimation

Depth estimation has applications like robot navigation, advance driver assistance systems, 3D movies etc. Depth is represented in terms of disparity map which can be generated using various stereo correspondence algorithms. This paper presents an implementation of semi global block matching algorithm on raspberry pi to estimate the depth from the camera. The algorithm computes the disparity using block wise matching and smoothness constraint. The proposed algorithm is compared with SAD algorithm on personal computer as well as on raspberry pi. The algorithm is also further, evaluated on the standard dataset. The project aim is to detect people in an image and estimate their depth from the camera. The real time implementation of the proposed algorithm uses block size of 21 × 21 for images which has resolution of 1280 × 720 P. The algorithm estimates depth with an accuracy of 95%. The system also provides faster processing time to the proposed algorithm.

Hardware implementation of stereo vision algorithms for depth estimation

Depth estimation has applications like robot navigation, advance driver assistance systems, 3D movies etc. Depth is represented in terms of disparity map which can be generated using various stereo correspondence algorithms. This paper presents an implementation of semi global block matching algorithm on raspberry pi to estimate the depth from the camera. The algorithm computes the disparity using block wise matching and smoothness constraint. The proposed algorithm is compared with SAD algorithm on personal computer as well as on raspberry pi. The algorithm is also further, evaluated on the standard dataset. The project aim is to detect people in an image and estimate their depth from the camera. The real time implementation of the proposed algorithm uses block size of 21 × 21 for images which has resolution of 1280 × 720 P. The algorithm estimates depth with an accuracy of 95%. The system also provides faster processing time to the proposed algorithm.

Improving stereo matching algorithm with adaptive cross-scale cost aggregation

Human beings process stereoscopic correspondence across multiple purposes like robot navigation, automatic driving, and virtual or augmented reality. However, this bioinspiration is ignored by state-of-the-art dense stereo correspondence matching methods. Cost aggregation is one of the critical steps in the stereo matching method. In this article, we propose an optimized cross-scale cost aggregation scheme with coarse-to-fine strategy for stereo matching. This scheme implements cross-scale cost aggregation with the smoothness constraint on neighborhood cost, which essentially extends the idea of the inter-scale and intra-scale consistency constraints to increase the matching accuracy. The neighborhood costs are not only used in the intra-scale consistency to ensure that the regularized costs vary smoothly in an eight-connected neighbors region but also incorporated with inter-scale consistency to optimize the disparity estimation. Additionally, the improved method introduces an adaptive scheme in each scale with different aggregation methods. Finally, experimental results evaluated both on classic Middlebury and Middlebury 2014 data sets show that the proposed method performs better than the cross-scale cost aggregation. The whole stereo correspondence algorithm achieves competitive performance in terms of both matching accuracy and computational efficiency. An extensive comparison, including the KITTI benchmark, illustrates the better performance of the proposed method also.

Massively parallel GPU computing for fast stereo correspondence algorithms

A GPU-based implementation of the fixed window partially demosaiced CFA stereo matching application is presented.Accelerations up to 20 times are obtained for large size image pairs.A GPU-based implementation of the adaptive window color stereo matching application is presented.It can handle four pairs of standard images from Middlebury database within roughly 100 ms.A comparative study among different GPU-based local dense stereo matching implementations is provided. Current accurate stereo matching algorithms employ some key techniques that are not suitable for parallel GPU architecture. It will be tricky and cumbersome to directly take these techniques into GPU applications. Trying to tackle this difficulty, we design two GPU-based stereo matching algorithms, one using a local fixed aggregation window whose size is configurable, and the other using an adaptive aggregation window which only includes necessary pixels. We use the winner-takes-all (WTA) principle for optimization and a plain voting refinement for post-processing; both do not need complex data structures. We aim to implement on GPU platforms fast stereo matching algorithms that produce results with same-level quality as other WTA local dense methods that use window-based cost aggregation. In our GPU-based implementation of the fixed window partially demosaiced CFA stereo matching application, accelerations up to 20 times are obtained for large size images. In our GPU-based implementation of the adaptive window color stereo matching application, experiment results show that it can handle four pairs of standard images from Middlebury database within roughly 100 ms.

Saliency detection based on superpixels clustering and stereo disparity

Reliable saliency detection can be used to quickly and effectively locate objects in images. In this paper, a novel algorithm for saliency detection based on superpixels clustering and stereo disparity (SDC) is proposed. Firstly, we use an improved superpixels clustering method to decompose the given image. Then, the disparity of each superpixel is computed by a modified stereo correspondence algorithm. Finally, a new measure which combines stereo disparity with color contrast and spatial coherence is defined to evaluate the saliency of each superpixel. From the experiments we can see that regions with high disparity can get higher saliency value, and the saliency maps have the same resolution with the source images, objects in the map have clear boundaries. Due to the use of superpixel and stereo disparity information, the proposed method is computationally efficient and outperforms some state-of-the-art colorbased saliency detection methods.

Literature Survey on Stereo Vision Disparity Map Algorithms

This paper presents a literature survey on existing disparity map algorithms. It focuses on four main stages of processing as proposed by Scharstein and Szeliski in a taxonomy and evaluation of dense two-frame stereo correspondence algorithms performed in 2002. To assist future researchers in developing their own stereo matching algorithms, a summary of the existing algorithms developed for every stage of processing is also provided. The survey also notes the implementation of previous software-based and hardware-based algorithms. Generally, the main processing module for a software-based implementation uses only a central processing unit. By contrast, a hardware-based implementation requires one or more additional processors for its processing module, such as graphical processing unit or a field programmable gate array. This literature survey also presents a method of qualitative measurement that is widely used by researchers in the area of stereo vision disparity mappings.

Local Smoothness Enforced Cost Volume Regularization for Fast Stereo Correspondence

A novel cost volume regularization algorithm is proposed. It successfully combines weighted cost aggregation in local stereo methods and smoothness constraints in global stereo methods. By enforcing local smoothness during the weighted cost aggregation on the horizontal tree structure, cost distribution can be well regularized. Accurate disparity results with sharp boundary can be generated while keeping extremely low computational amount. A new post-processing method is also proposed. The whole stereo correspondence algorithm achieves competitive performance on the Middlebury stereo testbed in terms of both accuracy and speed.

Stereo Photo Measured ONH Shape Predicts Development of POAG in Subjects With Ocular Hypertension.

To identify objective, quantitative optic nerve head (ONH) structural features and model the contributions of glaucoma. Baseline stereoscopic optic disc images of 1635 glaucoma-free participants at risk for developing primary open-angle glaucoma (POAG) were collected as part of the Ocular Hypertension Treatment Study. A stereo correspondence algorithm designed for fundus images was applied to extract the three-dimensional (3D) information about the ONH. Principal component analysis was used to identify ONH 3D structural features and the contributions of demographic features, clinical variables, and disease were modeled using linear regression and linear component analysis. The computationally identified features were evaluated based on associations with glaucoma and ability to predict which participants would develop POAG. The computationally identified features were significantly associated with future POAG, POAG-related demographics (age, ethnicity), and clinical measurements (horizontal and vertical cup-to-disc ratio, central corneal thickness, and refraction). Models predicting future POAG development using the OHTS baseline data and STEP features achieved an AUC of 0.722 in cross-validation testing. This was a significant improvement over using only demographics (age, sex, and ethnicity), which had an AUC of 0.599. Methods for identifying objective, quantitative measurements of 3D ONH structure were developed using a large dataset. The identified features were significantly associated with POAG and POAG-related variables. Further, these features increased predictive model accuracy in predicting future POAG. The results indicate that the computationally identified features might be useful in POAG early screening programs or as endophenotypes to investigate POAG genetics.

Improved Disparity Map Computation on Stereoscopic Streaming Video with Multi-core Parallel Implementation

Stereo vision has become an important technical issue in the field of 3D imaging, machine vision, robotics, image analysis, and so on. The depth map extraction from stereo video is a key technology of stereoscopic 3D video requiring stereo correspondence algorithms. This is the matching process of the similarity measure for each disparity value, followed by an aggregation and optimization step. Since it requires a lot of computational power, there are significant speed-performance advantages when exploiting parallel processing available on processors. In this situation, multi-core CPU may allow many parallel programming technologies to be realized in users computing devices. This paper proposes parallel implementations for calculating disparity map using a shared memory programming and exploiting the streaming SIMD extension technology. By doing so, we can take advantage both of the hardware and software features of multi-core processor. For the performance evaluation, we implemented a parallel SAD algorithm with OpenMP and SSE2. Their processing speeds are compared with non parallel version on stereoscopic streaming video. The experimental results show that both technologies have a significant effect on the performance and achieve great improvements on processing speed.

A Hardware-Efficient Architecture for Accurate Real-Time Disparity Map Estimation

Emerging embedded vision systems utilize disparity estimation as a means to perceive depth information to intelligently interact with their host environment and take appropriate actions. Such systems demand high processing performance and accurate depth perception while requiring low energy consumption, especially when dealing with mobile and embedded applications, such as robotics, navigation, and security. The majority of real-time dedicated hardware implementations of disparity estimation systems have adopted local algorithms relying on simple cost aggregation strategies with fixed and rectangular correlation windows. However, such algorithms generally suffer from significant ambiguity along depth borders and areas with low texture. To this end, this article presents the hardware architecture of a disparity estimation system that enables good performance in both accuracy and speed. The architecture implements an adaptive support weight stereo correspondence algorithm that integrates image segmentation information in an attempt to increase the robustness of the matching process. The article also presents hardware-oriented algorithmic modifications/optimization techniques that make the algorithm hardware-friendly and suitable for efficient dedicated hardware implementation. A comparison to the literature asserts that an FPGA implementation of the proposed architecture is among the fastest implementations in terms of million disparity estimations per second (MDE/s), and with an overall accuracy of 90.21%, it presents an effective processing speed/disparity map accuracy trade-off.

Analog Signal Processing Approach for Coarse and Fine Depth Estimation

Imaging and Image sensors is a field that is continuously evolving. There are new products coming into the market every day. Some of these have very severe Size, Weight and Power constraints whereas other devices have to handle very high computational loads. Some require both these conditions to be met simultaneously. Current imaging architectures and digital image processing solutions will not be able to meet these ever increasing demands. There is a need to develop novel imaging architectures and image processing solutions to address these requirements. In this work we propose analog signal processing as a solution to this problem. The analog processor is not suggested as a replacement to a digital processor but it will be used as an augmentation device which works in parallel with the digital processor, making the system faster and more efficient. In order to show the merits of analog processing two stereo correspondence algorithms are implemented. We propose novel modifications to the algorithms and new imaging architectures which, significantly reduces the computation time.

Analysis and Research of Stereo Correspondence Algorithms

Stereo matching has been the focus of computer vision research. Some current researches on stereo matching algorithms were summarized, the stereo matching key techniques were analyzed; View of the current major challenges on binocular stereo matching, elaborated matching algorithm possible solutions and modification approaches; Finally, the field of technology development are prospected.

Indoor and outdoor depth imaging of leaves with time-of-flight and stereo vision sensors: Analysis and comparison

In this article we analyze the response of Time-of-Flight (ToF) cameras (active sensors) for close range imaging under three different illumination conditions and compare the results with stereo vision (passive) sensors. ToF cameras are sensitive to ambient light and have low resolution but deliver high frame rate accurate depth data under suitable conditions. We introduce metrics for performance evaluation over a small region of interest. Based on these metrics, we analyze and compare depth imaging of leaf under indoor (room) and outdoor (shadow and sunlight) conditions by varying exposure times of the sensors. Performance of three different ToF cameras (PMD CamBoard, PMD CamCube and SwissRanger SR4000) is compared against selected stereo-correspondence algorithms (local correlation and graph cuts). PMD CamCube has better cancelation of sunlight, followed by CamBoard, while SwissRanger SR4000 performs poorly under sunlight. Stereo vision is comparatively more robust to ambient illumination and provides high resolution depth data but is constrained by texture of the object along with computational efficiency. Graph cut based stereo correspondence algorithm can better retrieve the shape of the leaves but is computationally much more expensive as compared to local correlation. Finally, we propose a method to increase the dynamic range of ToF cameras for a scene involving both shadow and sunlight exposures at the same time by taking advantage of camera flags (PMD) or confidence matrix (SwissRanger).

Evaluation of stereo correspondence algorithms and their implementation on FPGA

The accuracy of stereo vision has been considerably improved in the last decade, but real-time stereo matching is still a challenge for embedded systems where the limited resources do not permit fast operation of sophisticated approaches. This work presents an evaluation of area-based algorithms used for calculating distance in stereoscopic vision systems, their hardware architectures for implementation on FPGA and the cost of their accuracies in terms of FPGA hardware resources. The results show the trade-off between the quality of such maps and the hardware resources which each solution demands, so they serve as a guide for implementing stereo correspondence algorithms in real-time processing systems.

An improved gradient-based dense stereo correspondence algorithm using guided filter

This paper presents an improved gradient-based real-time stereo correspondence algorithm using guided filter. Color intensity value is sensitive to radiometric distortions including exposure differences and illumination differences, thus the error correspondence rates of these methods are high. deMaeztu et al. [1] proposed to measure the similarity between pixels using the gradient value instead of color intensity. The method has better robustness to radiometric distortions than intensity-based local methods, but the running time is so long that it is not suitable for real-time applications, because the adaptive support weight of neighbor pixels depends on bilateral filter. Guided filter has edge-preserving character as bilateral filter, but runs faster than it, we use guided filter as adaptive support weight instead of bilateral filter of the neighbor pixels in a finite squared support window. The experimental results demonstrate that the improved algorithm performs much better compared with gradient-based method and other local methods, whether in accuracy or efficiency, according to the widely-used Middlebury stereo benchmarks, and the robustness to radiometric distortions of ours is better also.

Quantitative Evaluation of Papilledema from Stereoscopic Color Fundus Photographs

To derive a computerized measurement of optic disc volume from digital stereoscopic fundus photographs for the purpose of diagnosing and managing papilledema. Twenty-nine pairs of stereoscopic fundus photographs and optic nerve head (ONH) centered spectral domain optical coherence tomography (SD-OCT) scans were obtained at the same visit in 15 patients with papilledema. Some patients were imaged at multiple visits in order to assess their changes. Three-dimensional shape of the ONH was estimated from stereo fundus photographs using an automated multi-scale stereo correspondence algorithm. We assessed the correlation of the stereo volume measurements with the SD-OCT volume measurements quantitatively, in terms of volume of retinal surface elevation above a reference plane and also to expert grading of papilledema from digital fundus photographs using the Frisén grading scale. The volumetric measurements of retinal surface elevation estimated from stereo fundus photographs and OCT scans were positively correlated (correlation coefficient r(2) = 0.60; P < 0.001) and were positively correlated with Frisén grade (Spearman correlation coefficient r = 0.59; P < 0.001). Retinal surface elevation among papilledema patients obtained from stereo fundus photographs compares favorably with that from OCT scans and with expert grading of papilledema severity. Stereoscopic color imaging of the ONH combined with a method of automated shape reconstruction is a low-cost alternative to SD-OCT scans that has potential for a more cost-effective diagnosis and management of papilledema in a telemedical setting. An automated three-dimensional image analysis method was validated that quantifies the retinal surface topography with an imaging modality that has lacked prior objective assessment.

Hardware Design Considerations for Edge‐Accelerated Stereo Correspondence Algorithms

Stereo correspondence is a popular algorithm for the extraction of depth information from a pair of rectified 2D images. Hence, it has been used in many computer vision applications that require knowledge about depth. However, stereo correspondence is a computationally intensive algorithm and requires high‐end hardware resources in order to achieve real‐time processing speed in embedded computer vision systems. This paper presents an overview of the use of edge information as a means to accelerate hardware implementations of stereo correspondence algorithms. The presented approach restricts the stereo correspondence algorithm only to the edges of the input images rather than to all image points, thus resulting in a considerable reduction of the search space. The paper highlights the benefits of the edge‐directed approach by applying it to two stereo correspondence algorithms: an SAD‐based fixed‐support algorithm and a more complex adaptive support weight algorithm. Furthermore, we present design considerations about the implementation of these algorithms on reconfigurable hardware and also discuss issues related to the memory structures needed, the amount of parallelism that can be exploited, the organization of the processing blocks, and so forth. The two architectures (fixed‐support based versus adaptive‐support weight based) are compared in terms of processing speed, disparity map accuracy, and hardware overheads, when both are implemented on a Virtex‐5 FPGA platform.

Non-probabilistic cellular automata-enhanced stereo vision simultaneous localization and mapping

In this paper, a visual non-probabilistic simultaneous localization and mapping (SLAM) algorithm suitable for area measurement applications is proposed. The algorithm uses stereo vision images as its only input and processes them calculating the depth of the scenery, detecting occupied areas and progressively building a map of the environment. The stereo vision-based SLAM algorithm embodies a stereo correspondence algorithm that is tolerant to illumination differentiations, the robust scale- and rotation-invariant feature detection and matching speeded-up robust features method, a computationally effective v-disparity image calculation scheme, a novel map-merging module, as well as a sophisticated cellular automata-based enhancement stage. A moving robot equipped with a stereo camera has been used to gather image sequences and the system has autonomously mapped and measured two different indoor areas.

Biologically and psychophysically inspired adaptive support weights algorithm for stereo correspondence

In this paper a novel stereo correspondence algorithm is presented. It incorporates many biologically and psychologically inspired features to an adaptive weighted sum of absolute differences (SAD) framework in order to determine the correct depth of a scene. In addition to ideas already exploited, such as the color information utilization, gestalt laws of proximity and similarity, new ones have been adopted. The presented algorithm introduces the use of circular support regions, the gestalt law of continuity as well as the psychophysically-based logarithmic response law. All the aforementioned perceptual tools act complementarily inside a straightforward computational algorithm applicable to robotic applications. The results of the algorithm have been evaluated and compared to those of similar algorithms.

Stereo vision for robotic applications in the presence of non-ideal lighting conditions

Many robotic and machine-vision applications rely on the accurate results of stereo correspondence algorithms. However, difficult environmental conditions, such as differentiations in illumination depending on the viewpoint, heavily affect the stereo algorithms’ performance. This work proposes a new illumination-invariant dissimilarity measure in order to substitute the established intensity-based ones. The proposed measure can be adopted by almost any of the existing stereo algorithms, enhancing it with its robust features. The performance of the dissimilarity measure is validated through experimentation with a new adaptive support weight (ASW) stereo correspondence algorithm. Experimental results for a variety of lighting conditions are gathered and compared to those of intensity-based algorithms. The algorithm using the proposed dissimilarity measure outperforms all the other examined algorithms, exhibiting tolerance to illumination differentiations and robust behavior.