Disparity Search Range Research Articles

Research on Underwater Distance Measurement of Binocular Vision by Semantic Separation Picture

Aiming at the problem of large amount of calculation and slow speed of stereo matching in binocular vision ranging, this paper proposes an underwater distance measurement of binocular vision based on underwater target detection. By optimizing the SGBM algorithm, the disparity search range is limited in the underwater target recognition area, and an adaptive aggregation window is constructed to adapt to different texture changes. This method improves the matching accuracy and effectively reduces the computational burden of unrelated regions. At the same time, a texture similarity dynamic adjustment-matching window is established to flexibly respond to texture changes in different regions, thereby greatly improving the matching speed and accuracy. Experiments show that the ranging speed is increased by 26.8 %. This study can not only improve the efficiency and accuracy of underwater operations, but also lay a solid foundation for subsequent technical research and application promotion.

A fast stereo matching network based on temporal attention and 2D convolution

We propose a fast stereo matching network based on temporal attention and 2D convolution (TANet). Due to the high similarity of the disparity between consecutive frames in an image sequence, we propose a temporal attention (TA) module that uses the disparity map of the previous frame to guide the disparity search range in the current frame, thus significantly improving the efficiency of disparity calculation in the cost volume module. Additionally, we propose a hierarchical cost construction and 2D convolution aggregation module that constructs a pyramid cost volume by fusing edge cues to establish detail constraints. This overcomes the problem of difficult convergence caused by information loss when replacing 3D convolution with 2D convolution. Experimental results show that the TA module effectively optimizes the cost volume and, together with 2D convolution, improves the computational speed. Compared with state-of-the-art algorithms, TANet achieves a speedup of nearly 4x, with a running time of 0.061s, and reduces the parameter count by nearly half while decreasing accuracy by 1.1%. Code is available at https://github.com/Y0uchenZ/TANet.

A Confidence-Aware Cascade Network for Multi-Scale Stereo Matching of Very-High-Resolution Remote Sensing Images

Stereomatching plays an essential role in 3D reconstruction using very-high-resolution (VHR) remote sensing images. However, it still faces unignorable challenges due to the multi-scale objects in large scenes and the multi-modality probability distribution in challenging regions, especially the occluded and textureless areas. Accurate disparity estimation in stereo matching for multi-scale objects has become a hard but crucial task. In this paper, to tackle these problems, we design a novel confidence-aware unimodal cascade and fusion pyramid network for stereo matching. The fused cost volume from the coarsest scale is used to generate the initial disparity map, and then the learnable confidence maps are generated to construct the unimodal cost distributions, which are used to narrow down the next-stage disparity search range. Moreover, we design a cross-scale interaction aggregation module to leverage multi-scale information. Both smooth-L1 loss and stereo focal loss are applied to regularize the disparity map and unimodal cost distribution, respectively. Compared to two state-of-the-art stereo matching networks, extensive experimental results show that our proposed network outperforms them in terms of average endpoint error (EPE) and the fraction of erroneous pixels (D1).

Efficient global stereo-matching method of general images under a long baseline based on baseline estimation.

For three-dimensional (3D) measurement of a large object, stereo matching needs to be done under a relatively long baseline in order to reduce the accumulated errors caused by the splicing process, but this brings new challenges to matching efficiency. The images under a short baseline have an advantage avoiding a very large disparity level that is usually presented in a large baseline to improve efficiency, whether the image pairs to be matched are of translation or rotation configuration. This paper mainly focuses on the general case existing for both, and the purpose is to efficiently execute a stereo matching in this scene based on short baseline estimation. A rectification method of three images used for baseline estimation is first studied to make the image planes become coplanar, which removes the vertical disparity. The three rectified images can form a stereo pair with a short baseline and the other one has a long baseline. Then the relationship that exists between disparities of a pixel on rectified stereo pairs with different baselines is deduced by considering the quantization error introduced in the projection and rectification. In this way, the disparity search range of a pixel on the rectified stereo pair with the long baseline is reduced from the usual hundreds of pixels to a few pixels predicted by its disparity with the short baseline, which greatly improves the calculation efficiency and makes it especially suitable for the 3D measurement of large objects. Experimental results are presented to demonstrate the efficiency and accuracy of the method.

Stereo Matching with Multi‐scale Based on Anisotropic Match Cost

AbstractBinocular vision is a popular method for 3d measurement, but the precision and efficiency is still a limitation. In order to solve the problem of low precision and high mismatching rate of current stereo match methods in weak texture region, a local stereo matching algorithm with multi‐scale based on anisotropic match cost was proposed. Not only including the gray value between the adjacent pixels as the absolute difference (AD) method, the matching cost function also involved gradient and phase information to make an anisotropic evaluation and eliminate the outliers of the flat area. Meanwhile, a multi‐scale strategy with the image pyramid was referenced, and the dynamic disparity search range of variable window was used on the original cost aggregation framework by improving the stereo matching algorithm of cross‐scale cost aggregation. The matching cost volume in each scale space used the dynamic support window to guide the filter to aggregate the matching cost. Finally, to overcome the problems of disparity selection ambiguity of WTA (winner‐take‐all) strategy and the horizontal fringe introduced by left‐right consistency (LRC) detection, the weighted median filter based on guided filter weight was used to carry out disparity refinement. According to the experiments, the method got more high matching accuracy, and the average error on the Middlebury test platform reached 5.25%.

Efficient binocular stereo correspondence matching with 1-D Max-Trees

Extraction of depth from images is of great importance for various computer vision applications. Methods based on convolutional neural networks are very accurate but have high computation requirements, which can be achieved with GPUs. However, GPUs are difficult to use on devices with low power requirements like robots and embedded systems. In this light, we propose a stereo matching method appropriate for applications in which limited computational and energy resources are available. The algorithm is based on a hierarchical representation of image pairs which is used to restrict disparity search range. We propose a cost function that takes into account region contextual information and a cost aggregation method that preserves disparity borders. We tested the proposed method on the Middlebury and KITTI benchmark data sets and on the TrimBot2020 synthetic data. We achieved accuracy and time efficiency results that show that the method is suitable to be deployed on embedded and robotics systems.

A 1920 $\times$ 1080 25-Frames/s 2.4-TOPS/W Low-Power 6-D Vision Processor for Unified Optical Flow and Stereo Depth With Semi-Global Matching

This paper presents a unified 6-D vision processor that enables dense real-time 3-D depth and 3-D motion perception at full-high-definition (1920 × 1080, FHD) resolution. The proposed design implements a neighbor-guided semi-global matching (NG-SGM) algorithm to unify the stereo depth and optical flow matching problem and to reduce computation by 98% compared with the original SGM. We introduce a new custom-designed, high-bandwidth coalescing crossbar circuit that automatically coalesces redundant memory accesses to mitigate the highly irregular memory accesses observed in NG-SGM. The proposed 6-D vision processor also maximizes on-chip memory reuse by using 64 on-chip rotating image buffers that cover a wide optical flow and depth disparity search range of 176 pixels per dimension. The processor implements massive parallel processing with 576 compute units that are deeply pipelined with a dependency-resolving skewed-diagonal scan to hide the dynamic and variable dependency in the pipeline. The fabricated processor performs dense NG-SGM at 25 frames/s for optical flow or 30 frames/s for stereo depth at FHD resolution while consuming only 760 mW in 28-nm CMOS.

A fast temporal constraint semi-dense stereo matching method

For some applications that do not require a dense disparity map, we propose a fast semi-dense stereo matching method. First, we use grading rank transformation to obtaining the texture points of the image. Then, we use downsampling to reduce the matching complexity of texture points between consecutive frames. The disparity search range of the current frame is constrained according to the matching of texture points between consecutive frames and the disparity map obtained from the preceding frame. Finally, we perform multi-constrained stereo matching and disparity propagation to obtain the disparity map of the current frame. The proposed method reduces the complexity of stereo matching by the temporal disparity constraint and disparity propagation. Experimental results show that the semi-dense disparity map obtained is accurate and fast. For some video sequences, the proposed method can achieve real-time running speed.

An Efficient Dense Stereo Matching Method for Planetary Rover

Stereo matching is one of the most important and challenging subjects in the field of planetary rover with a stereo vision system. The generated disparity map can make rover to avoid the obstacle and explore the planetary surface automatically. In this paper, we propose an efficient dense stereo matching method to generate disparity maps for a planetary rover, which relies on 3-D plane fitting, adaptive penalties, and coarse-to-fine disparity constraint. In order to achieve efficient stereo matching at the coarsest level of the pyramid, we present a 3-D plane fitting to reduce the disparity search range and propose adaptive penalties in the more-global matching method for obtaining an accurate disparity map. At the finer level, our method then estimates the disparity search range based on coarse-to-fine disparity constraint and utilizes adaptive penalties to obtain an accurate disparity map. The extensive experiments with stereo images of Chang'e-3 rover demonstrate that our approach can generate disparity maps efficiently and accurately compared with the most state-of-the-art semi-global matching methods, especially at low texture regions, depth discontinuities, and occlusion regions.

STEREO IMAGE DENSE MATCHING BY INTEGRATING SIFT AND SGM ALGORITHM

Abstract. Semi-global matching(SGM) performs the dynamic programming by treating the different path directions equally. It does not consider the impact of different path directions on cost aggregation, and with the expansion of the disparity search range, the accuracy and efficiency of the algorithm drastically decrease. This paper presents a dense matching algorithm by integrating SIFT and SGM. It takes the successful matching pairs matched by SIFT as control points to direct the path in dynamic programming with truncating error propagation. Besides, matching accuracy can be improved by using the gradient direction of the detected feature points to modify the weights of the paths in different directions. The experimental results based on Middlebury stereo data sets and CE-3 lunar data sets demonstrate that the proposed algorithm can effectively cut off the error propagation, reduce disparity search range and improve matching accuracy.

A long baseline global stereo matching based upon short baseline estimation

In global stereo vision, balancing the matching efficiency and computing accuracy seems to be impossible because they contradict each other. In the case of a long baseline, this contradiction becomes more prominent. In order to solve this difficult problem, this paper proposes a novel idea to improve both the efficiency and accuracy in global stereo matching for a long baseline. In this way, the reference images located between the long baseline image pairs are firstly chosen to form the new image pairs with short baselines. The relationship between the disparities of pixels in the image pairs with different baselines is revealed by considering the quantized error so that the disparity search range under the long baseline can be reduced by guidance of the short baseline to gain matching efficiency. Then, the novel idea is integrated into the graph cuts (GCs) to form a multi-step GC algorithm based on the short baseline estimation, by which the disparity map under the long baseline can be calculated iteratively on the basis of the previous matching. Furthermore, the image information from the pixels that are non-occluded under the short baseline but are occluded for the long baseline can be employed to improve the matching accuracy. Although the time complexity of the proposed method depends on the locations of the chosen reference images, it is usually much lower for a long baseline stereo matching than when using the traditional GC algorithm. Finally, the validity of the proposed method is examined by experiments based on benchmark datasets. The results show that the proposed method is superior to the traditional GC method in terms of efficiency and accuracy, and thus it is suitable for long baseline stereo matching.

Disparity Estimation Using Fast Motion-Search Algorithm and Local Image Characteristics

A stereo matching method estimates the disparity value between two correspondences in both stereo images. The disparity value represents the depth information of objects obtained from stereo images which have two different viewpoints. In many papers, the stereo matching method is tested under limited disparity conditions. These conditions follow configurations of test images. However, the disparity range in real applications is not known for those conditions and configurations. In this case, we have to check all pixels in the scan line to find the correspondence. Therefore, it is a time consuming task. Thus, we propose a fast disparity estimation method using the limited search range. The proposed method limits the disparity search range with the fast motion-search algorithm and local image characteristics.

Modified adaptive support weight and disparity search range estimation schemes for stereo matching processors

Recently, to obtain three-dimensional depth information from a set of stereo images, stereo matching processors are widely used in intelligent robots, autonomous vehicles, and the Internet of things environment, all of which require real-time processing capability with minimal hardware resources. In this paper, we propose a modified adaptive support weight scheme with rectangular ring-type window configurations that minimize hardware resources while maintaining matching accuracy. In addition, to reduce the computational overhead of window-based local stereo matching algorithms, we present a robust disparity search range estimation scheme based on stretched depth histograms. To evaluate the performance of the proposed schemes, we implemented them using C language and performed experiments. In addition, to show the feasibility of the hardware implementation of the proposed schemes, we also describe them using Verilog hardware description language and implemented them using a field-programmable gate array-based platform. Experimental results show that compared to conventional method, the proposed schemes reduced up to 57% of hardware resources and 33% of computational overhead, respectively.

Stereo-camera-based object detection using fuzzy color histograms and a fuzzy classifier with depth and shape estimations

This paper proposes a new method of detecting an object containing multiple colors with non-homogeneous distributions in complex backgrounds and subsequently estimating the depth and shape of the object using a stereo camera. To extract features for object detection, this paper proposes fuzzy color histograms (FCHs) based on the self-splitting clustering (SSC) of the hue-saturation (HS) color space. For each scanning window in a pyramid of scaled images, the FCH is obtained by accumulating the fuzzy degrees of all of the pixels belonging to each cluster. The FCH is fed to a fuzzy classifier to detect an object in the left image captured by the stereo camera. To find the matched object region in the right image, the left and right images are first segmented using the SSC-partitioned HS space. The depth of the object is then found by performing stereo matching on the segmented images. To find the shape of the object, a disparity map is built using the estimated object depth to automatically determine the stereo matching window size and disparity search range. Finally, the shape of the object is segmented from the disparity map. The experimental results of the detection of different objects with depth and shape estimations are used to verify the performance of the proposed method. Comparisons with different detection and disparity map construction methods are performed to demonstrate the advantage of the proposed method.

Multi-Resolution Disparity Processing and Fusion for Large High-Resolution Stereo Image

Large panoramic views with high resolution have the advantage of a wide field of view over regular stereo views. However, the large size and high resolution impose difficulties on the stereo matching problem such as complexity and structure ambiguity, respectively. In this paper, effective multi-resolution disparity processing to resolve the difficulties is presented. We propose to adaptively determine the disparity search range based on the combined local structure from image intensity and initial disparity. The adaptive disparity range is able to propagate the smoothness property at low resolution to high resolution while preserving fine details. It reduces structure ambiguity as well as computational complexity. To reduce the disparity quantization error at the coarse level, we propose a reliable multiple fitting algorithm that is noticeably effective on the round surface. The spatial-multi-resolution total variation method is investigated to minimize inconsistency in space-scale dimension . The experimental results on the Middlebury datasets and real-world high-resolution images demonstrate that the proposed multi-resolution scheme produces high-quality and high-resolution disparity maps by fusing individual multi-scale disparity maps, while reducing complexity.

Optimized projection patterns for stereo systems

This paper describes how to generate optimal projection patterns to supplement general stereo camera systems. In contrast to structured light, the active stereo systems utilize the projected patterns only as auxiliary information in correspondence search, whereas the structured light systems have to detect the patterns and decode them to compute depth. The concept of non-recurring De Bruijn sequences is introduced, and a few algorithms based on the non-recurring De Bruijn sequence are designed to build optimized projection patterns for several stereo parameters. When only the search window size of a stereo system is given, we show that a non-recurring De Bruijn sequence with corresponding parameters makes the longest functional pattern, and presents experimental results using real scenes to show the effectiveness of the proposed projection patterns. Additionally if the pattern length is given in the form of maximum disparity search range, the algorithm using branch-and-bound search scheme to find an optimal sub-sequence of a non-recurring De Bruijn sequence is proposed.

Robust stereo matching using adaptive random walk with restart algorithm

In this paper, we propose a robust dense stereo reconstruction algorithm using a random walk with restart. The pixel-wise matching costs are aggregated into superpixels and the modified random walk with restart algorithm updates the matching cost for all possible disparities between the superpixels. In comparison to the majority of existing stereo methods using the graph cut, belief propagation, or semi-global matching, our proposed method computes the final reconstruction through the determination of the best disparity at each pixel in the matching cost update. In addition, our method also considers occlusion and depth discontinuities through the visibility and fidelity terms. These terms assist in the cost update procedure in the calculation of the standard smoothness constraint. The method results in minimal computational costs while achieving high accuracy in the reconstruction. We test our method on standard benchmark datasets and challenging real-world sequences. We also show that the processing time increases linearly in relation to an increase in the disparity search range.

Full high-definition real-time depth estimation for three-dimensional video system

Three-dimensional (3-D) video brings people strong visual perspective experience, but also introduces large data and complexity processing problems. The depth estimation algorithm is especially complex and it is an obstacle for real-time system implementation. Meanwhile, high-resolution depth maps are necessary to provide a good image quality on autostereoscopic displays which deliver stereo content without the need for 3-D glasses. This paper presents a hardware implementation of a full high-definition (HD) depth estimation system that is capable of processing full HD resolution images with a maximum processing speed of 125 fps and a disparity search range of 240 pixels. The proposed field-programmable gate array (FPGA)-based architecture implements a fusion strategy matching algorithm for efficiency design. The system performs with high efficiency and stability by using a full pipeline design, multiresolution processing, synchronizers which avoid clock domain crossing problems, efficient memory management, etc. The implementation can be included in the video systems for live 3-D television applications and can be used as an independent hardware module in low-power integrated applications.

스테레오 영상에서의 적응적 영역 가중치 매칭의 효율적 근사화 방법

영역기반 스테레오 매칭 분야에서 최근 인간의 시각체계(Human Visual System)에 기반하여 영역내의 밝기값과 거리값에 따라 적응적으로 가중치를 부여하는 적응적 영역 가중치(Adaptive Support-Weight, ASW) 방법이 좋은 매칭 결과를 보이고 있다. 그러나 이 방법은 좋은 매칭 결과에 비해서 많은 연산비용을 필요로 하게 되고, 매칭의 실시간 시스템화에 큰 장애요소로 작용한다. 이에 Bilateral filter 수식으로 근사화 후 Integral Histogram 기법을 적용하여 영역 윈도우의 크기에 상관없이 상수 시간 O(1) 내에 매칭을 수행하는 연구가 진행되었다. 하지만 이 방법은 근사화 과정에서의 원 ASW 수식을 왜곡하기 때문에 매칭 정확도의 손실을 가져오게 된다. 본 논문에서는 적응적 영역 가중치 알고리즘의 매칭 정확도를 유지하면서 적응적 영역 가중치 알고리즘의 계산 비용을 줄이는 알고리즘을 제안한다. 이를 위해 영역내의 픽셀을 그룹화하여 근사화된 매칭을 수행하는 Sub-Block 방법과 영상의 에지 정보에 따라 적응적으로 시차 탐색 범위를 조정하는 방법을 제안한다. 결과적으로 제안된 기법은 기존 방식보다 좋은 매칭 정확도를 유지하면서도 효율적으로 계산 수행 시간을 줄이게 된다. Recently in the area-based stereo matching field, Adaptive Support-Weight (ASW) method that weights matching cost adaptively according to the luminance intensity and the geometric difference shows promising matching performance. However, ASW requires more computational cost than other matching algorithms do and its real-time implementation becomes impractical. By applying Integral Histogram technique after approximating to the Bilateral filter equation, the computational time of ASW can be restricted in constant time regardless of the support window size. However, Integral Histogram technique causes loss of the matching accuracy during approximation process of the original ASW equation. In this paper, we propose a novel algorithm that maintains the ASW algorithm's matching accuracy while reducing the computational costs. In the proposed algorithm, we propose Sub-Block method that groups the pixels within the support area. We also propose the method adjusting the disparity search range depending on edge information. The proposed technique reduces the calculation time efficiently while improving the matching accuracy.

An FPGA-based RGBD imager

This paper describes a trinocular stereo vision system using a single chip of FPGA to generate the composite color (RGB) and disparity data stream at video rate, called the RGBD imager. The system uses the triangular configuration of three cameras for synchronous image capture and the trinocular adaptive cooperative algorithm based on local aggregation for smooth and accurate dense disparity mapping. We design a fine-grain parallel and pipelining architecture in FPGA for implementation to achieve a high computational and real-time throughput. A binary floating-point format is customized for data representation to satisfy the wide data range and high computation precision demands in the disparity calculation. Memory management and data bit-width control are applied in the system to reduce the hardware resource consumption and accelerate the processing speed. The system is able to produce dense disparity maps with 320 × 240 pixels in a disparity search range of 64 pixels at the rate of 30 frames per second.