Stereo Matching Algorithm Research Articles

A Stereo Matching Algorithm Guided by Multiple Linear Regression

A Stereo Matching Algorithm Guided by Multiple Linear Regression

User Authentication Model for Securing E-Health System Using Fingerprint Biometrics

The emergence of electronic health or eHealth has revolutionized the healthcare industry to offer better healthcare services at a low and affordable cost. However, it still suffers from security and privacy issues in handling health information. The privacy and security issues in eHealth domain are mainly centered around user authentication, data integrity, data confidentiality, and patient privacy protection. Biometrics technology has considerable opportunities to cope with the above security problems by providing reliable and secure user authentication. However, due to the sensitive nature of health data, the most important challenge lies toward the development of an efficient security model that can guarantee data privacy and reliability, verifying that only authorized personnel can access their corresponding health data. In this paper, we present a comprehensive overview of biometrics applications in eHealth and propose a robust and efficient scheme for user verification using fingerprint biometrics in order to enhance privacy and security in eHealth information systems. Moreover, additional issues like system complexity and processing time related to the use of biometrics should be taken into consideration. We therefore, emphasize to reduce the computation cost in biometric matching. In this work, we use local minutie features for user authentication and employ a fast stereo matching algorithm to compare the minutiae features of the test (probe) fingerprint with the minutiae features extracted from the gallery fingerprints (fingerprint database) in order to verify a person. Experimental results show that the proposed scheme leads to a compromise between the computation efficiency and the accuracy of the verification process. We believe that, our proposed scheme could be employed in real-time applications.

基于二次引导滤波的局部立体匹配算法

基于二次引导滤波的局部立体匹配算法

基于共同视域的自监督立体匹配算法

基于共同视域的自监督立体匹配算法

A Dense Stereovision System for 3D Body Imaging

Three-dimensional (3D) body imaging has become an important mean for applications based on body measurement, such as apparel customization and obesity assessment. In this article, we present a 3D body imaging system built upon stereovision technology. The system utilizes paired, high-resolution single-lens reflex (SLR) cameras to image the front and back body surfaces of a person, and robust and efficient stereo matching algorithms to reconstruct the 3D surface of the body with high-density data clouds. The system's accuracy and repeatability have been evaluated on mannequins and human subjects in comparison with other measurement methods. It was found that the geometrical measurements from reconstructed 3D body models, including body circumferences and volumes, were highly repeatable and consistent with the manual and other 3D instrumental measurements (CV <; 0.1%, and R <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> > 0.99).

As‐global‐as‐possible stereo matching with adaptive smoothness prior

More global matching (MGM) overcomes the limitation of one-dimensional scanline optimisation in semi-global matching (SGM). Nevertheless, the possible weaknesses of the MGM algorithm are as follows: (i) only two directions are considered for each image traversal direction, which may lead to massive mismatches; (ii) disparity estimation around the object boundaries usually performs terrible since the smoothness term is designed independent of the image prior. In this research, the authors consider all of the four directions for each image traversal direction through a novel model. Besides utilising the prior of neighboured pixels' correlation, adaptive smoothness terms are modelled and augmented into the energy function. These contributions encourage ‘as-global-as-possible (AGAP)’. More importantly, different from the recent works in which the aggregated algorithms have been conducted as the data term of an energy function, conversely, the authors make the energy function as a part of cost aggregation framework. Performance evaluations on Middlebury v.2 and v.3 stereo data sets demonstrate that the proposed AGAP outperforms other four most challenging stereo matching algorithms, and also performs better on Microsoft i2i stereo videos. In addition, under various strategies of parallelisation, the presented AGAP shows a near real-time execution time.

Disparity estimation in stereo video sequence with adaptive spatiotemporally consistent constraints

Numerous stereo matching algorithms have been proposed to obtain disparity estimation for a single pair of stereo images. However, simply even applying the best of them to temporal frames independently, i.e., without considering the temporal consistency between consecutive frames, may suffer from the undesirable artifacts. Here, we proposed an adaptive, spatiotemporally consistent, constraints-based systematic method that generates spatiotemporally consistent disparity maps for stereo video image sequences. Firstly, a reliable temporal neighborhood is used to enforce the “self-similarity” assumption and prevent errors caused by false optical flow matching from propagating between consecutive frames. Furthermore, we formulate the adaptive temporal predicted disparity map as prior knowledge of the current frame. It is used as a soft constraint to enhance the temporal consistency of disparities, increase the robustness to luminance variance, and restrict the range of the potential disparities for each pixel. Additionally, to further strengthen smooth variation of disparities, the adaptive temporal segment confidence is incorporated as a soft constraint to reduce ambiguities caused by under- and over-segmentation, and retain the disparity discontinuities that align with 3D object boundaries from geometrically smooth, but strong color gradient regions. Experimental evaluations demonstrate that our method significantly improves the spatiotemporal consistency both quantitatively and qualitatively compared with other state-of-the-art methods on the synthetic DCB and realistic KITTI datasets.

Stereoview to Multiview Conversion Architecture for Auto-Stereoscopic 3D Displays

In this paper, a stereoview to multiview conversion system, which includes stereo matching and depth image-based rendering (DIBR) hardware designs, is proposed. To achieve an efficient architecture, the proposed stereo matching algorithm simply generates the raw matching costs and aggregates cost based on 1D iterative aggregation schemes. For the DIBR architecture, an inpainting-based method is used to find the most similar patch from the background, according to depth information. The simulation results show that the designed architecture achieves an averaged peak signal-to-noise ratio of 30.2 dB and structure similarity of 0.94 for the tested images. The hardware design for the proposed 2D to 3D conversion system operates at a maximum clock frequency of 160.2 MHz for outputting 1080p ( $1920 \times 1080$ ) video at 60 frames per second.

Field‐based robotic phenotyping of sorghum plant architecture using stereo vision

AbstractSorghum (Sorghum bicolor) is known as a major feedstock for biofuel production. To improve its biomass yield through genetic research, manually measuring yield component traits (e.g. plant height, stem diameter, leaf angle, leaf area, leaf number, and panicle size) in the field is the current best practice. However, such laborious and time‐consuming tasks have become a bottleneck limiting experiment scale and data acquisition frequency. This paper presents a high‐throughput field‐based robotic phenotyping system which performed side‐view stereo imaging for dense sorghum plants with a wide range of plant heights throughout the growing season. Our study demonstrated the suitability of stereo vision for field‐based three‐dimensional plant phenotyping when recent advances in stereo matching algorithms were incorporated. A robust data processing pipeline was developed to quantify the variations or morphological traits in plant architecture, which included plot‐based plant height, plot‐based plant width, convex hull volume, plant surface area, and stem diameter (semiautomated). These image‐derived measurements were highly repeatable and showed high correlations with the in‐field manual measurements. Meanwhile, manually collecting the same traits required a large amount of manpower and time compared to the robotic system. The results demonstrated that the proposed system could be a promising tool for large‐scale field‐based high‐throughput plant phenotyping of bioenergy crops.

Unified Confidence Estimation Networks for Robust Stereo Matching.

We present a deep architecture that estimates a stereo confidence, which is essential for improving the accuracy of stereo matching algorithms. In contrast to existing methods based on deep convolutional neural networks (CNNs) that rely on only one of the matching cost volume or estimated disparity map, our network estimates the stereo confidence by using the two heterogeneous inputs simultaneously. Specifically, the matching probability volume is first computed from the matching cost volume with residual networks and a pooling module in a manner that yields greater robustness. The confidence is then estimated through a unified deep network that combines confidence features extracted both from the matching probability volume and its corresponding disparity. In addition, our method extracts the confidence features of the disparity map by applying multiple convolutional filters with varying sizes to an input disparity map. To learn our networks in a semi-supervised manner, we propose a novel loss function that use confident points to compute the image reconstruction loss. To validate the effectiveness of our method in a disparity post-processing step, we employ three post-processing approaches; cost modulation, ground control points-based propagation, and aggregated ground control points-based propagation. Experimental results demonstrate that our method outperforms state-of-the-art confidence estimation methods on various benchmarks.

Optimised Implementation of Dense Stereo Correspondence for Resource Limited Hardware

Computer stereo vision is a passive sensing technique that helps to recover 3D information of an environment from 2D images. The stereo correspondence is a challenging task that finds out matching pixels between the stereo image pair based on Lambertian criteria and its result is a disparity space image. The depth of the objects from the camera can be calculated from this disparity value by using the principle of triangulation. For the vision guided robot navigation, the requirement of stereo matching algorithms on low power dedicated hardware that can achieve a high frame rate is unambiguous. A new, highly optimized implementation of correlation based, Sum of Absolute Differences correspondences algorithm on a low cost resource limited FPGA is presented here. This System-on-Programmable-Chip architecture based system achieved a higher frame rate of 50 fps with 64 disparity levels without using a microprocessor. On performance evaluation, the disparity map shows a maximum error of 0.308% only. This reconfigurable parallel processing, high speed architecture of the algorithm implementation takes only 43% of available resources of low density Altera Cyclone II. This hardware implementation of stereo vision system outperforms in terms of accuracy, speed and resource utilization of all the other existing stereo systems of its similar kind. Also, it offers a better trade-off between run-time speed and accuracy and is found suitable for most of the range finding real-time applications.

Simultaneous measurements of three-dimensional trajectories and wingbeat frequencies of birds in the field.

Tracking the movements of birds in three dimensions is integral to a wide range of problems in animal ecology, behaviour and cognition. Multi-camera stereo-imaging has been used to track the three-dimensional (3D) motion of birds in dense flocks, but precise localization of birds remains a challenge due to imaging resolution in the depth direction and optical occlusion. This paper introduces a portable stereo-imaging system with improved accuracy and a simple stereo-matching algorithm that can resolve optical occlusion. This system allows us to decouple body and wing motion, and thus measure not only velocities and accelerations but also wingbeat frequencies along the 3D trajectories of birds. We demonstrate these new methods by analysing six flocking events consisting of 50 to 360 jackdaws (Corvus monedula) and rooks (Corvus frugilegus) as well as 32 jackdaws and 6 rooks flying in isolated pairs or alone. Our method allows us to (i) measure flight speed and wingbeat frequency in different flying modes; (ii) characterize the U-shaped flight performance curve of birds in the wild, showing that wingbeat frequency reaches its minimum at moderate flight speeds; (iii) examine group effects on individual flight performance, showing that birds have a higher wingbeat frequency when flying in a group than when flying alone and when flying in dense regions than when flying in sparse regions; and (iv) provide a potential avenue for automated discrimination of bird species. We argue that the experimental method developed in this paper opens new opportunities for understanding flight kinematics and collective behaviour in natural environments.

Tri-Stereo Image Matching using Watershed Segmentation on Disparity Space Image

Local stereo matching algorithms use winner-take-all approach to get the disparity. Many times they end up at an erroneous result. To solve this, in the present work, a novel stereo image matching technique has been developed that identifies the most likely local minimum from the several possible local minima that corresponds to true disparity. The technique uses properties of the physical continuity of the land surface by the watershed lines applied to the disparity space volume. This helps to minimize the search for the most probable local minima as the correct solution for the matching. The matching is further improved by combining the watershed lines of disparity space volume of two stereo pairs from the tri-stereo. In the present study, experiments have been carried out using the standard Middlebury stereo datasets and remotely sensed tri-stereo images. Based on this approach, the experiments are successfully carried out using the test dataset. The experimental results are compared with the results from the currently contemporary techniques of dynamic programming and semi-global matching which resulted in 2–10% improvement in density of matched points for different datasets.

Optimized stereo matching algorithm for integral imaging microscopy and its potential use in precise 3-D optical manipulation

Abstract In this paper, we present an integral imaging microscopy, which can achieve continuous depth extraction. The multiple stereo matching algorithm is optimized with data point interpolation and calculation range correction, which can improve the computing efficiency and accuracy. A water suspension of silica-embedded magnetic particles in light-trap environment is used as a specimen. The elementary image array is obtained through sequentially imaging the specimen by the objective lens and microlens array, which contains continuous depth information that can give the optical tweezers a feedback for precise 3-D manipulation. Experimental and calculated results show the feasibility of our method.

A parallel stereovision method used for monitoring the collapse of a three-story frame model subjected to seismic loading

In lab or actual engineering, the traditional contact sensors are always destroyed with damage of structures or models, and this leads to the failure of entire dynamic measurement. To deal with this issue, this article proposes a three-dimensional displacement measurement approach based on the parallel stereovision model. This approach combines the efficient large-scale stereo matching algorithm and the digital image correlation approach to obtain three-dimensional displacement of structural vibration response. In this article, the digital image correlation method is employed to match sequential images of left and right cameras, and the efficient large-scale stereo algorithm is used to obtain the disparity between the left and right images at each moment. Based on the parallel stereovision model, the three-dimensional coordinates of the measured point on the structural surface are achieved accordingly. A shaking table test of a three-story frame model under El Centro earthquake is performed in lab. Experimental results indicate that the displacement curves acquired by the linear variable differential transformers and the proposed method agree well, and this testifies the reliability and effectiveness of the presented approach. Only the presented stereovision approach attains the entire displacement curves of structural collapse when the model collapses, and this demonstrates the capability of the presented stereovision approach monitoring structural collapse.

A hierarchical stereo matching algorithm based on adaptive support region aggregation method

Stereo matching is a fundamental process in many application fields. An accurate depth information is useful for stereo systems to separate occluding image components. In the conception of stereo matching algorithms, various works rely on the Census Transform (CT) as a cost computation step, due to its robustness against radiometric changes. In this paper, we propose a new variant of the CT cost function which incorporates edge side information for further compensating radiometric changes. We demonstrate that the proposed variant matching cost improves significantly the quality of the disparity results. In addition, an aggregation method, based on the adaptive support region, is implemented into a hierarchical fusion processing scheme, by incorporating low frequency information into high frequency using a robust exponential function. This allows a multi-scale interaction within the adaptive cost aggregation. Hence, significant erroneous disparities are reduced, especially in textureless regions. Experiments were conducted on the KITTI benchmark and the obtained results have shown the potential merits of the proposed framework.

A Miniature Binocular Endoscope with Local Feature Matching and Stereo Matching for 3D Measurement and 3D Reconstruction.

As the traditional single camera endoscope can only provide clear images without 3D measurement and 3D reconstruction, a miniature binocular endoscope based on the principle of binocular stereoscopic vision to implement 3D measurement and 3D reconstruction in tight and restricted spaces is presented. In order to realize the exact matching of points of interest in the left and right images, a novel construction method of the weighted orthogonal-symmetric local binary pattern (WOS-LBP) descriptor is presented. Then a stereo matching algorithm based on Gaussian-weighted AD-Census transform and improved cross-based adaptive regions is studied to realize 3D reconstruction for real scenes. In the algorithm, we adjust determination criterions of adaptive regions for edge and discontinuous areas in particular and as well extract mismatched pixels caused by occlusion through image entropy and region-growing algorithm. This paper develops a binocular endoscope with an external diameter of 3.17 mm and the above algorithms are applied in it. The endoscope contains two CMOS cameras and four fiber optics for illumination. Three conclusions are drawn from experiments: (1) the proposed descriptor has good rotation invariance, distinctiveness and robustness to light change as well as noises; (2) the proposed stereo matching algorithm has a mean relative error of 8.48% for Middlebury standard pairs of images and compared with several classical stereo matching algorithms, our algorithm performs better in edge and discontinuous areas; (3) the mean relative error of length measurement is 3.22%, and the endoscope can be utilized to measure and reconstruct real scenes effectively.

Extracting depth information from stereo images using a fast correlation matching algorithm

ABSTRACT Stereo matching algorithms are essential for recovering depth information of objects in many computer vision applications including 3D reconstruction, robot navigation, autonomous driving and so on. Most of the stereo algorithms generally rely on two types of matching technique: global and local matching. The state-of-the-art stereo algorithms that measure disparity or depth with high accuracy are generally based on global methods. However, they are not suitable for real-time applications because of high computational costs. The local algorithms, on the other hand, are very fast but they provide less computational accuracy compared to the global methods. To make a tradeoff between computation speed and accuracy, this paper proposes an efficient local correlation approach for depth estimation using a pruning proposal. This paper also evaluates the performance of different matching cost functions/algorithms for disparity or dense estimation. Experimental evaluation confirms that our proposed pruning method for point correspondence is able to achieve a significant accuracy with high computational speed that can be very useful for real-time environments.

Improvement of stereo matching algorithm based on sum of gradient magnitude differences and semi‐global method with refinement step

A new stereo matching algorithm which uses improved matching cost computation and optimisation using the semi-global method (SGM) is proposed. The absolute difference is sensitive to low textured regions and high noise on the stereo images with radiometric distortions. To get over these problems, sum of gradient magnitude differences has been introduced at the first stage. This method is strong against the radiometric differences on the stereo images. Hence, this approach will reduce the error of preliminary data for stereo corresponding process. The SGM is used at the aggregation, and optimisation stage uses 16 different directions of 2D path. Additionally, the iterative guided filter is utilised at the refinement stage which minimises the errors and increases the accuracy. The proposed work produces accurate results and performs much better compared with some established algorithms based on the standard stereo benchmarking evaluation from the Middlebury and KITTI.

Design and implementation of wireless multimedia sensor network node based on FPGA and binocular vision

According to the actual demand of the wireless sensor network for multimedia information acquisition, the new multimedia wireless sensor node is designed and implemented with FPGA as the core and Zigbee wireless communication. On the multimedia nodes, sensors such as binocular camera and accelerometer are configured to realize real-time perception and acquisition of environmental information and image data, and the short distance wireless communication and detection ability is formed through Zigbee protocol. When the test distance is not more than 60 m, the measured data fluctuate around the theoretical curve, and the communication is reliable. When the distance of the node is within the 14 m, the data transmission has no error. In view of the fact that the mismatch rate of the traditional Census transform is high under the condition of accuracy and noise, an improved Census transform stereo matching algorithm is proposed, which is applied to the nodes of multimedia sensor network. The experimental results show that this scheme can realize the data transmission and processing of wireless sensor network nodes and has strong operation capability and visual matching precision while reducing power consumption.