Local Stereo Matching Algorithm Research Articles

Improved stereo matching algorithm based on multi-scale fusion

Aiming at the low matching accuracy of local stereo matching algorithm in weak texture or discontinuous disparity areas, a stereo matching algorithm combining multi-scale fusion of convolutional neural network (CNN) and feature pyramid structure (FPN) is proposed. The feature pyramid is applied on the basis of the convolutional neural network to realize the multi-scale feature extraction and fusion of the image, which improves the matching similarity of the image blocks. The guide graph filter is used to quickly and effectively complete the cost aggregation. The disparity selection stage adapts the improvement dynamic programming algorithm to obtain the initial disparity map. The initial disparity map is refined so as to obtain the final disparity map. The algorithm is trained and tested on the image provided by Middlebury data set, and the result shows that the disparity map obtained by the algorithm has good effect.

Local Stereo Matching Using Adaptive Cross-Region-Based Guided Image Filtering with Orthogonal Weights

Adaptive cross-region-based guided image filtering (ACR-GIF) is a commonly used cost aggregation method. However, the weights of points in the adaptive cross-region (ACR) are generally not considered, which affects the accuracy of disparity results. In this study, we propose an improved cost aggregation method to address this issue. First, the orthogonal weight is proposed according to the structural feature of the ACR, and then the orthogonal weight of each point in the ACR is computed. Second, the matching cost volume is filtered using ACR-GIF with orthogonal weights (ACR-GIF-OW). In order to reduce the computing time of the proposed method, an efficient weighted aggregation computing method based on orthogonal weights is proposed. Additionally, by combining ACR-GIF-OW with our recently proposed matching cost computation method and disparity refinement method, a local stereo matching algorithm is proposed as well. The results of Middlebury evaluation platform show that, compared with ACR-GIF, the proposed cost aggregation method can significantly improve the disparity accuracy with less additional time overhead, and the performance of the proposed stereo matching algorithm outperforms other state-of-the-art local and nonlocal algorithms.

Efficient local stereo matching algorithm based on fast gradient domain guided image filtering

Guided image filtering (GIF) based cost aggregation or disparity refinement stereo matching algorithms are studied extensively owing to the edge-aware preserved smoothing property. However, GIF suffers from halo artifacts in sharp edges and shows high computational costs on high-resolution images. The performance of GIF in stereo matching would be limited by the above two defects. To solve these problems, a novel fast gradient domain guided image filtering (F-GDGIF) is proposed. To be specific, halo artifacts are effectively alleviated by incorporating an efficient multi-scale edge-aware weighting into GIF. With this multi-scale weighting, edges can be preserved much better. In addition, high computational costs are cut down by sub-sampling strategy, which decreases the computational complexity from O(N) to O(N/s2) (s: sub-sampling ratio) To verify the effectiveness of the algorithm, F-GDGIF is applied to cost aggregation and disparity refinement in stereo matching algorithms respectively. Experiments on the Middlebury evaluation benchmark demonstrate that F-GDGIF based stereo matching method can generate more accuracy disparity maps with low computational cost compared to other GIF based methods.

Hardware accelerator for an accurate local stereo matching algorithm using binary neural network

Convolutional neural networks (CNNs) have shown appealing performance on stereo matching tasks in recent years. However, existing deep neural networks (DNNs) based matching algorithms use semi-global matching (SGM) to aggregate the matching costs, which limit the processing speed. In this study, we present a novel binarized CNN stereo matching hardware acceleration using local methods on an FPGA, which can provide high accuracy stereo estimation and achieve high-throughput at the same time. To reduce the consumption of hardware resource and improve the processing speed, we propose pipelined architecture for BNN, and sparse local aggregation to optimize the implementation on an FPGA. We evaluated the proposed implementation on a challenging stereo dataset with a Stratix V FPGA. From the experimental results, our binarized CNN stereo matching implementation shows significant improvement in the real-time performance and the energy efficiency over other computing platforms with a 6.95% error rate on the KITTI2015 dataset.

An edge-aware based adaptive multi-feature set extraction for stereo matching of binocular images

Object Stereo Vision has conventionally been one of the deeply examined areas in computer vision. Stereo matching is employed in numerous modern applications, including robot navigation, augmented reality, and automotive applications. Even though it has a long research history, it is still challenging for the edges of textureless, discontinues, and occluded regions under radiometric variation. This research article proposes a modified histogram equalization, a novel feature extraction, a spatial gradient model, and matching cost, which is robust and stable to images taken in different radiometric variations. The proposed method reduced the average percentage of bad pixels to 3.35 and reduced the relative mean square error (RMSE) up to 30.08 on the Middlebury dataset for different illumination and exposure values. Quantitative and qualitative evaluation of the proposed method demonstrates significant improvement in increasing PSNR and decreasing bad pixel percentage against radiometric variation and state-of-the-art local stereo matching algorithms.

Local Stereo Matching: An Adaptive Weighted Guided Image Filtering-Based Approach

Innovative applications in rapidly evolving domains such as robotic navigation and autonomous (driverless) vehicles rely on binocular computer vision systems that meet stringent response time and accuracy requirements. A key problem in these vision systems is stereo matching, which involves matching pixels from two input images in order to construct the output, a 3D map. Building upon the existing local stereo matching algorithms, this paper proposes a novel stereo matching algorithm that is based on a weighted guided filtering foundation. The proposed algorithm consists of three main steps; each step is designed with the goal of improving accuracy. First, the matching costs are computed using a unique combination of complementary methods (absolute difference, Census, and gradient algorithms) to reduce errors. Second, the costs are aggregated using an adaptive weighted guided image filtering method. Here, the regularization parameters are adjusted adaptively using the Canny method, further reducing errors. Third, a disparity map is generated using the winner-take-all strategy; this map is subsequently refined using a densification method to reduce errors. Our experimental results indicate that the proposed algorithm provides a higher level of accuracy in comparison to a collection of the existing state-of-the-art local algorithms.

Stereo Matching with Multi‐scale Based on Anisotropic Match Cost

AbstractBinocular vision is a popular method for 3d measurement, but the precision and efficiency is still a limitation. In order to solve the problem of low precision and high mismatching rate of current stereo match methods in weak texture region, a local stereo matching algorithm with multi‐scale based on anisotropic match cost was proposed. Not only including the gray value between the adjacent pixels as the absolute difference (AD) method, the matching cost function also involved gradient and phase information to make an anisotropic evaluation and eliminate the outliers of the flat area. Meanwhile, a multi‐scale strategy with the image pyramid was referenced, and the dynamic disparity search range of variable window was used on the original cost aggregation framework by improving the stereo matching algorithm of cross‐scale cost aggregation. The matching cost volume in each scale space used the dynamic support window to guide the filter to aggregate the matching cost. Finally, to overcome the problems of disparity selection ambiguity of WTA (winner‐take‐all) strategy and the horizontal fringe introduced by left‐right consistency (LRC) detection, the weighted median filter based on guided filter weight was used to carry out disparity refinement. According to the experiments, the method got more high matching accuracy, and the average error on the Middlebury test platform reached 5.25%.

Three-dimensional rapid registration and reconstruction of multi-view rigid objects based on end-to-end deep surface model

Three-dimensional object reconstruction from multi-view images is an important topic in computer vision, which has attracted enormous attention during the past decades. With the further study in deep learning, remarkable progress of three-dimensional object reconstruct has been obtained in recent years. In this paper, we proposed three-dimensional rapid registration and reconstruction of multi-view rigid objects based on end-to-end deep surface model in the field of three-dimensional object reconstruction. Firstly, we introduce a matching algorithm called local stereo matching algorithm based on improved census transform and multi-scale spatial, aiming to improve the matching results for those regions. In cost aggregation step, guided map filtering algorithm with excellent gradient preserving property is introduced into Gaussian pyramid structure and regularization is added to strengthen cost volume consistency. Secondly, the improved inception RESNET module is added to improve the feature extraction ability of the network, and multiple features are extracted by using multiple network structures, and finally multiple features are sequentially input into the VRNN module to enhance the reconstruction effect of multi-view images. The experimental results show that our proposed method can not only achieve better reconstruction results, but also reconstruct more details and spend less time in training.

3D Vision System for a Robotic Arm Based on Equal Baseline Camera Array

This paper presents a lightweight 3D vision system called Equal Baseline Camera Array (EBCA). EBCA can work in different light conditions and it can be applied for measuring large range of distances. The system is a useful alternative to other known distance measuring devices such as structured-light 3D scanners, time-of-flight cameras, Light Detection and Ranging (LIDAR) devices and structure from motion techniques. EBCA can be mounted on a robotic arm without putting significant load on its construction. EBCA consists of a central camera and a ring of side cameras. The system uses stereo matching algorithms to acquire disparity maps and depth maps similarly as in case of using stereo cameras. This paper introduces methods of adapting stereo matching algorithms designed for stereo cameras to EBCA. The paper also presents the analysis of local, semi-global and global stereo matching algorithms in the context of the EBCA usage. Experiments show that, on average, results obtained from EBCA contain 37.49% less errors than the results acquired from a single stereo camera used in the same conditions.

Fixed window aggregation AD-census algorithm for phase-based stereo matching.

Phase-based stereo matching (PSM) is a vital step in binocular structured light. PSM is hard to strike a balance between efficiency and accuracy, especially because of the absolute phase matrix's (APM) double data type. It means that PSM needs more run time and memory than conventional intensity images' stereo matching. In this paper, we propose a modified absolute difference (AD)-Census algorithm called the fixed window aggregation AD-Census (FWA-AD-Census) to balance the contradiction between efficiency and accuracy in PSM. The FWA-AD-Census aggregates matching cost in a fixed support window instead of an adaptive support window in AD-Census. We analyze the reason why PSM is more suitable to aggregate the matching cost in a fixed support window. Simulations and experiments are conducted to verify the FWA-AD-Census's performances by comparing the FWA-AD-Census with two other local stereo matching algorithms. One is the AD-Census, which is more accurate but less efficient. Its matching cost is aggregated in an adaptive support window. Another is the sum of absolute difference (SAD), which is more efficient but less accurate, and its matching cost is aggregated in a fixed support window. Theoretical analysis and experimental results both indicate that the proposed algorithm can achieve similar accuracy to the AD-Census with less run time.

Semi-dense and robust image registration by shift adapted weighted aggregation and variational completion

This work presents a novel approach for both stereo and optical flow that deals with large displacements, depth/motion discontinuities and occlusions. The proposed method comprises two main steps. First, a novel local stereo matching algorithm is presented, whose main novelty relies in the block-matching aggregation step. We adopt an adaptive support weights approach in which the weight distribution favors pixels that share the same displacement with the reference one. State-of-the-art methods make the weight function depend only on image features. On the contrary, the proposed weight function depends additionally on the tested shift, by giving more importance to those pixels in the block-matching with smaller cost, as these are supposed to have the tested displacement. Moreover, the method is embedded into a pyramidal procedure to locally limit the search range, which helps to reduce ambiguities in the matching process and saves computational time. Second, the non-dense local estimation is filtered and interpolated by means of a new variational formulation making use of intermediate scale estimates of the local procedure. This permits to keep the fine details estimated at full resolution while being robust to noise and untextured areas using estimates at coarser scales. The introduced variational formulation as well as the block-matching algorithm are robust to illumination changes. We test our algorithm for both stereo and optical flow public datasets showing competitive results.

Disparity Optimization Algorithm for Stereo Matching Using Improved Guided Filter

A disparity optimization algorithm based on an improved guided filter is proposed to smooth the disparity image. A well-known problem to local stereo matching is the low matching accuracy and staircase effect in regions with weak texture and slope. Our disparity optimization method solves this problem and achieve a smooth disparity. First, the initial disparity image is obtained by a local stereo matching algorithm using segment tree. Then, the guided filter is improved by using gradient domain information. Lastly, the improved guided filter is adopted as the disparity optimization method to smooth the disparity image. Experiments conducted on the Middlebury data sets demonstrate that by using the proposed algorithm in this paper, the smoothness of the disparity map in slope regions is improved, and a higher precision of dense disparity is obtained.

Improvement of stereo matching algorithm based on guided filtering and Kernel Regression

Stereo matching is one of the most dynamic fields in computer vision. Though its relevant research has already stepped into a mature stage, there are still certain challenges to obtain real-time and high-precision disparity maps from stereo image pairs. This paper presents a novel local stereo matching algorithm with better performance in edge preserving. In the first stage, this paper measures matching cost through combining truncated absolute differences (TAD) of the color and gradient. In the cost aggregation stage, this paper is creatively to combined the weighted guided filtering and adaptive steering kernel regression algorithm, which effectively preserves image edge and depth information. In the final stage, an adaptive steering kernel regression algorithm is employed in interpolation to refine the final disparity map. According to the Middlebury benchmark experiments, the algorithm proposed in this paper could have better performance than other local stereo matching algorithms.

基于二次引导滤波的局部立体匹配算法

基于二次引导滤波的局部立体匹配算法

Tri-Stereo Image Matching using Watershed Segmentation on Disparity Space Image

Local stereo matching algorithms use winner-take-all approach to get the disparity. Many times they end up at an erroneous result. To solve this, in the present work, a novel stereo image matching technique has been developed that identifies the most likely local minimum from the several possible local minima that corresponds to true disparity. The technique uses properties of the physical continuity of the land surface by the watershed lines applied to the disparity space volume. This helps to minimize the search for the most probable local minima as the correct solution for the matching. The matching is further improved by combining the watershed lines of disparity space volume of two stereo pairs from the tri-stereo. In the present study, experiments have been carried out using the standard Middlebury stereo datasets and remotely sensed tri-stereo images. Based on this approach, the experiments are successfully carried out using the test dataset. The experimental results are compared with the results from the currently contemporary techniques of dynamic programming and semi-global matching which resulted in 2–10% improvement in density of matched points for different datasets.

A local stereo matching algorithm based on weighted guided image filtering for improving the generation of depth range images

Stereo matching is a challenging problem and highly accurate depth image is important in different applications. The main problem is to estimate the correspondence between two pixels in a stereo pair. To solve this problem, in the last decade, several cost aggregation methods aimed at improving the quality of stereo matching algorithms have been introduced. We propose a new cost aggregation method based on weighted guided image filtering (WGIF) for local stereo matching. The proposed algorithm solves multi-label problems in three steps. First, the cost volume is constructed using pixel-wise matching cost computation functions. Then, each slice of the cost volume is independently filtered using the WGIF, which substitutes for the smoothness term in the energy function. Finally, the disparity of any pixel is simply computed. The WGIF uses local weights based on a variance window of pixels in a guidance image for cost volume filtering. Experimental results using Middlebury stereo benchmark verify that the proposed method is effective due to a high quality cost volume filter.

Modified Adaptive Support Weight for Stereo Matching

Local stereo matching algorithms are very popular in the recent years. The adaptive support weight algorithms can give high accuracy results such as global methods. This paper proposed a support aggregation approach for stereo matching that computed support weight in sparse support window mask. The improvement from the previous work is that the new support weight can reduce the computation into the fourth of the earlier work and help to reach the optimum correspondence. It means sparse support weight affects the time computation that is needed in stereo matching and optimizes the disparity. This support weight is used to accomplish the stereo matching evaluation using this method.

Modified adaptive support weight and disparity search range estimation schemes for stereo matching processors

Recently, to obtain three-dimensional depth information from a set of stereo images, stereo matching processors are widely used in intelligent robots, autonomous vehicles, and the Internet of things environment, all of which require real-time processing capability with minimal hardware resources. In this paper, we propose a modified adaptive support weight scheme with rectangular ring-type window configurations that minimize hardware resources while maintaining matching accuracy. In addition, to reduce the computational overhead of window-based local stereo matching algorithms, we present a robust disparity search range estimation scheme based on stretched depth histograms. To evaluate the performance of the proposed schemes, we implemented them using C language and performed experiments. In addition, to show the feasibility of the hardware implementation of the proposed schemes, we also describe them using Verilog hardware description language and implemented them using a field-programmable gate array-based platform. Experimental results show that compared to conventional method, the proposed schemes reduced up to 57% of hardware resources and 33% of computational overhead, respectively.

Measuring height of high-voltage transmission poles using unmanned aerial vehicle (UAV) imagery

ABSTRACTAerial imagery is important in remote sensing applications. Unmanned aerial vehicle (UAV) has a wide range of applications in remote sensing and presents a substantial cost-effective solution when monitoring objects on the earth’s surface. Moreover, object detection and classification are important aspects of global information system, especially for remote sensing applications and power line monitoring, which are essential for the proper distribution of electricity to consumers. Manual inspection consumes much time and involves risk, especially in remote areas that host dangerous wildlife; hence, UAV-based approaches are more feasible for such monitoring. The authors propose an UAV approach that utilises a digital surface model and incorporates a stereo matching algorithm based on UAV stereo images. The proposed algorithm was based on a graph-cut (GC) algorithm that measured the disparity map. Results were compared with well-known algorithms; including, for example, global and local stereo matching algorithms. The proposed solution introduces and integrates ordering constraints along with a submodular energy minimisation function to/with the GC algorithm to enhance performance. The authors measured sensitivity and recall for all parameters against ground truth data for differently cropped images of 16 power poles. Results showed that the proposed model performed more accurately compared to extant methods.

Stereo Matching by Filtering-Based Disparity Propagation.

Stereo matching is essential and fundamental in computer vision tasks. In this paper, a novel stereo matching algorithm based on disparity propagation using edge-aware filtering is proposed. By extracting disparity subsets for reliable points and customizing the cost volume, the initial disparity map is refined through filtering-based disparity propagation. Then, an edge-aware filter with low computational complexity is adopted to formulate the cost column, which makes the proposed method independent on the local window size. Experimental results demonstrate the effectiveness of the proposed scheme. Bad pixels in our output disparity map are considerably decreased. The proposed method greatly outperforms the adaptive support-weight approach and other conditional window-based local stereo matching algorithms.