Pixel Disparity Research Articles

X-ray pneumonia detection using angular and radial local binary patterns fusion

Detecting pneumonia via X-ray images is a crucial phase in determining any probable bacterial or viral disease. The automation of this stage is critical since it speeds up the diagnosing process. LBPs have excelled in investigating local conditions within a neighborhood, where they aggregate and analyze local statistical results. In this paper, we present a novel family of LBP descriptors based on local accumulative pixel disparities. We construct the new descriptors using a circular neighborhood acquired from the initial square filter. These descriptors consider angular and radial transitions that occur in both the microstructures and macrostructures of image textures. Eventually, a more accurate and comprehensive visual representation is obtained. Furthermore, we offer a data fusion step based on a voting mechanism to integrate the retrieved data efficiently. We show many types of analyses that demonstrate LBP's and the suggested extensions' capability to extract discriminant features from X-ray images. The suggested method is tested on two different datasets with large diversities that include images from various demographics and regions. Several measures, including accuracy rate, precision, sensitivity, F-measure, and specificity, are used to evaluate the system's efficacy. According to the testing results, the proposed system provides a best successful recognition rate of 82.7 %.

Omni-Directional Semi-Global Stereo Matching with Reliable Information Propagation

High efficiency and accuracy of semi-global matching (SGM) make it widely used in many stereo vision applications. However, SGM not only struggles in dealing with pixels in homogeneous area, but also suffers from streak artifacts. In this paper, we propose a novel omni-directional SGM (OmniSGM) with a cost volume update scheme to aggregate costs from paths along all directions and to encourage reliable information to propagate across entire image. Specifically, we perform SGM along four tree structures, namely trees in the left, right, top and bottom of root node, and then fuse the outputs to obtain final result. The contributions of pixels on each tree can be recursively computed from leaf nodes to root node, ensuring our method has linear time computational complexity. Moreover, An iterative cost volume update scheme is proposed using aggregated cost in the last pass to enhance the robustness of initial matching cost. Thus, useful information is more likely to propagate in a long distance to handle the ambiguities in low textural area. Finally, we present an efficient strategy to propagate disparities of stable pixels along the minimum spanning tree (MST) for disparity refinement. Extensive experiments in stereo matching on Middlebury and KITTI datasets demonstrate that our method outperforms typical traditional SGM-based cost aggregation methods.

MB-Net: multiscale boundary interaction learning for image manipulation localization

Image editing techniques can modify the content of images indiscriminately, which causes a grave threat to the security of society. Hence, the localization of manipulated images is inevitable. A serious challenge for image manipulation detection is the lack of strategies for perceiving global features and refining edges. In this paper, we present a multiscale boundary interaction learning network for image manipulation localization to solve both problems. This network contains an adjacent-scale mutual module to enrich the global perception domain by interactively learning adjacent scale features. It avoids the tremendous noise interference caused by the direct fusion of all scale features. To effectively suppress semantic content segmentation, the boundary pixel disparity module computes interpixel differences at specific angles to enhance boundary artifact recognition between tampered and real regions. The fusion attention module is proposed to combine scale and edge messages, integrating spatial and channel correlations in a compatible way. Extensive experimental results indicate that our proposed method is significantly superior to current state-of-the-art methods on public standard datasets.

A binocular reconstruction based on perspective projection constraints and its application on robot eye‐hand coordination

Stereo matching algorithms have been developed for many years but basically focus only on the implementation of existing datasets and are rarely applied to real scenarios, such as industrial robot scenarios. Traditional stereo matching algorithms have a high error rate, and deep learning algorithms are difficult to obtain good results in real scenarios because of their weak generalisation ability and difficult access to training data. In order to use stereo matching algorithms for industrial robot guidance, it is better to design a new traditional algorithm with low time complexity for the characteristics of industrial robot scenarios dominated by planar facets. This paper proposes a new matching method based on subrows of pixels, instead of individual pixels, in order to improve robustness of matching and reduce running time. First, the pixel strings from the same row of the left and right images are divided into several colour-identical or colour-gradient segments. Then, the colour and length of the two left and right pixel segment are used as clues to determine a matching relation and obtain the matching type. Then, all match types can be determined according to non-crossing mapping. Each match type can reason backward to the corresponding spatial state of the stimulus source so that the disparity of pixels in pixel segments representing the spatial state can be calculated. This new matching method makes full use of the stimulus homology constraints and projective geometric constraints of row-aligned images. The method can obtain good results in industrial robot scenarios and be applied for industrial robot guidance.

Four-direction global matching with cost volume update for stereovision.

The accuracy and speed of semi-global matching (SGM) make it widely used in many computer vision problems. However, SGM often struggles in dealing with pixels in the homogeneous regions and also suffers from streak artefacts for weak smoothness constraints. Meanwhile, we observe that the global method usually fails in occluded areas. The disparities for occluded pixels are typically the average of the disparity of nearby pixels. The local method can propagate the information into occluded pixels with a similar color. In this paper, we propose a novel, to the best of our knowledge, four-direction global matching with a cost volume update scheme to cope with textureless regions and occlusion. The proposed method makes two changes in the recursive formula: a) the computation process considers four visited nodes to enforce more smooth constraints; b) the recursive formula integrates cost filtering to propagate reliable information farther in nontextured regions. Thus, our method can inherit the speed of SGM, properly avoid streaking artefacts, and deal with the occluded pixel. Extensive experiments in stereo matching on Middlebury demonstrate that our method outperforms typical SGM-based cost aggregation approaches and other state-of-the-art local methods.

Non-Learning Stereo-Aided Depth Completion Under Mis-Projection via Selective Stereo Matching

We propose a non-learning depth completion method for a sparse depth map captured using a light detection and ranging (LiDAR) sensor guided by a pair of stereo images. Generally, conventional stereo-aided depth completion methods have two limiations. (i) They assume the given sparse depth map is accurately aligned to the input image, whereas the alignment is difficult to achieve in practice. (ii) They have limited accuracy in the long range because the depth is estimated by pixel disparity. To solve the abovementioned limitations, we propose selective stereo matching (SSM) that searches the most appropriate depth value for each image pixel from its neighborly projected LiDAR points based on an energy minimization framework. This depth selection approach can handle any type of mis-projection. Moreover, SSM has an advantage in terms of long-range depth accuracy because it directly uses the LiDAR measurement rather than the depth acquired from the stereo. SSM is a discrete process; thus, we apply variational smoothing with binary anisotropic diffusion tensor (B-ADT) to generate a continuous depth map while preserving depth discontinuity across object boundaries. Experimentally, compared with the previous state-of-the-art stereo-aided depth completion, the proposed method reduced the mean absolute error (MAE) of the depth estimation to 0.65 times and demonstrated approximately twice more accurate estimation in the long range. Moreover, under various LiDAR-camera calibration errors, the proposed method reduced the depth estimation MAE to 0.34-0.93 times from previous depth completion methods.

Disparity-preserving image rectangularization for stereoscopic panorama

This study aims at generating a long strip stereoscopic panorama with a rectangular boundary from a stereoscopic video. The issues arising from this goal are how to automatically select appropriate frames to reduce geometric distortion in image stitching, how to preserve disparity under image warping, and how to generate a rectangular panoramic stereoscopic image without the loss of boundary information. To deal with these issues and to generate visually smooth stereoscopic panorama, a disparity-aware image warping is proposed. Moreover, the image warping method is performed on the irregular left and right panoramic images simultaneously with a hybrid control mesh to generate a rectangular panorama while preserving the spatial shape and disparity as much as possible. Experimental results on various stereo video contents show that the proposed method can effectively preserve both the spatial shapes and pixel disparity.

Single-Shot 3D Shape Reconstruction Using Structured Light and Deep Convolutional Neural Networks.

Single-shot 3D imaging and shape reconstruction has seen a surge of interest due to the ever-increasing evolution in sensing technologies. In this paper, a robust single-shot 3D shape reconstruction technique integrating the structured light technique with the deep convolutional neural networks (CNNs) is proposed. The input of the technique is a single fringe-pattern image, and the output is the corresponding depth map for 3D shape reconstruction. The essential training and validation datasets with high-quality 3D ground-truth labels are prepared by using a multi-frequency fringe projection profilometry technique. Unlike the conventional 3D shape reconstruction methods which involve complex algorithms and intensive computation to determine phase distributions or pixel disparities as well as depth map, the proposed approach uses an end-to-end network architecture to directly carry out the transformation of a 2D image to its corresponding 3D depth map without extra processing. In the approach, three CNN-based models are adopted for comparison. Furthermore, an accurate structured-light-based 3D imaging dataset used in this paper is made publicly available. Experiments have been conducted to demonstrate the validity and robustness of the proposed technique. It is capable of satisfying various 3D shape reconstruction demands in scientific research and engineering applications.

A Novel Stereo Matching Algorithm for Digital Surface Model (DSM) Generation in Water Areas

Image dense matching has become one of the widely used means for DSM generation due to its good performance in both accuracy and efficiency. However, for water areas, the most common ground object, accurate disparity estimation is always a challenge to excellent image dense matching methods, as represented by semi-global matching (SGM), due to the poor texture. For this reason, a great deal of manual editing is always inevitable before practical applications. The main reason for this is the lack of uniqueness of matching primitives, with fixed size and shape, used by those methods. In this paper, we propose a novel DSM generation method, namely semi-global and block matching (SGBM), to achieve accurate disparity and height estimation in water areas by adaptive block matching instead of pixel matching. First, the water blocks are extracted by seed point growth, and an adaptive block matching strategy considering geometrical deformations, called end-block matching (EBM), is adopted to achieve accurate disparity estimation. Then, the disparity of all other pixels beyond these water blocks is obtained by SGM. Last, the median value of height of all pixels within the same block is selected as the final height for this block after forward intersection. Experiments are conducted on ZiYuan-3 (ZY-3) stereo images, and the results show that DSM generated by our method in water areas has high accuracy and visual quality.

A Memory- and Accuracy-Aware Gaussian Parameter-Based Stereo Matching Using Confidence Measure.

Accurate stereo matching requires a large amount of memory at a high bandwidth, which restricts its use in resource-limited systems such as mobile devices. This problem is compounded by the recent trend of applications requiring significantly high pixel resolution and disparity levels. To alleviate this, we present a memory-efficient and robust stereo matching algorithm. For cost aggregation, we employ the semiglobal parametric approach, which significantly reduces the memory bandwidth by representing the costs of all disparities as a Gaussian mixture model. All costs on multiple paths in an image are aggregated by updating the Gaussian parameters. The aggregation is performed during the scanning in the forward and backward directions. To reduce the amount of memory for the intermediate results during the forward scan, we suggest to store only the Gaussian parameters which contribute significantly to the final disparity selection. We also propose a method to enhance the overall procedure through a learning-based confidence measure. The random forest framework is used to train various features which are extracted from the cost and intensity profile. The experimental results on KITTI dataset show that the proposed method reduces the memory requirement to less than 3 percent of that of semiglobal matching (SGM) while providing a robust depth map compared to those of state-of-the-art SGM-based algorithms.

Dimension Reduction of Hand and Face Feature Level Fusion in Multimodal Biometric Authentication

The proposed work is a multimodal biometric authentication approach with image texture feature dimension reduction of trained feature vector which leads reduction in memory size and in turn reduces the computational time. In this paper hand and face features are used for person identification. The texture features of hand image are extracted using Haar and several Daubechie’s of 2D-DWT followed by 2D- edge detector gives better identification with reduction in feature vector and face features are extracted by neighborhood common characterization with block based segmentation approach to estimate the disparity in face. The neighborhood common characterization based structure recognition with a person representative per sample is more effective. The neighborhood features are constructed by extracting the similar blocks in the image, the intra pixel disparity feature is obtained by exploiting external common images to estimate the feasible facial disparities. Neighborhood common characterization reduces the overall residual of the given features over the local feature, common disparity dictionary, and shape based residual of a block. Neighbourhood common characterization representation, of face recognize with one representative per person more effectively. The system uses either of the biometric traits for person identification with 99.98% of authentication rate.

Human Detection and Depth Calculation Using Stereographic Vision with Pixel Disparity and Hog Classifier

Human Detection and Depth Calculation Using Stereographic Vision with Pixel Disparity and Hog Classifier

Focus model for metric depth estimation in standard plenoptic cameras

In recent years, a lot of efforts have been devoted to the problem of depth estimation from lightfield images captured by standard plenoptic cameras. However, most of the metric depth estimation methods in the state-of-the-art leverage pixel disparity only. In this paper, we tackle the problem of focus-based metric depth estimation in standard plenoptic cameras. For this purpose we propose a closed-form model that relates the refocusing parameter with the focus distance of a plenoptic camera in order to allow for metric depth estimation. Based on the proposed model, we develop a calibration procedure that allows finding the parameters of the model. Using measurements of a time-of-flight sensor as ground-truth, experimental validation in a distance range of 0.2–1.6 m shows that focus-based depth estimation is feasible with a root-mean-squared error of less than 5 cm.

A long baseline global stereo matching based upon short baseline estimation

In global stereo vision, balancing the matching efficiency and computing accuracy seems to be impossible because they contradict each other. In the case of a long baseline, this contradiction becomes more prominent. In order to solve this difficult problem, this paper proposes a novel idea to improve both the efficiency and accuracy in global stereo matching for a long baseline. In this way, the reference images located between the long baseline image pairs are firstly chosen to form the new image pairs with short baselines. The relationship between the disparities of pixels in the image pairs with different baselines is revealed by considering the quantized error so that the disparity search range under the long baseline can be reduced by guidance of the short baseline to gain matching efficiency. Then, the novel idea is integrated into the graph cuts (GCs) to form a multi-step GC algorithm based on the short baseline estimation, by which the disparity map under the long baseline can be calculated iteratively on the basis of the previous matching. Furthermore, the image information from the pixels that are non-occluded under the short baseline but are occluded for the long baseline can be employed to improve the matching accuracy. Although the time complexity of the proposed method depends on the locations of the chosen reference images, it is usually much lower for a long baseline stereo matching than when using the traditional GC algorithm. Finally, the validity of the proposed method is examined by experiments based on benchmark datasets. The results show that the proposed method is superior to the traditional GC method in terms of efficiency and accuracy, and thus it is suitable for long baseline stereo matching.

HYBRID-BASED DENSE STEREO MATCHING

Abstract. Stereo matching generating accurate and dense disparity maps is an indispensable technique for 3D exploitation of imagery in the fields of Computer vision and Photogrammetry. Although numerous solutions and advances have been proposed in the literature, occlusions, disparity discontinuities, sparse texture, image distortion, and illumination changes still lead to problematic issues and await better treatment. In this paper, a hybrid-based method based on semi-global matching is presented to tackle the challenges on dense stereo matching. To ease the sensitiveness of SGM cost aggregation towards penalty parameters, a formal way to provide proper penalty estimates is proposed. To this end, the study manipulates a shape-adaptive cross-based matching with an edge constraint to generate an initial disparity map for penalty estimation. Image edges, indicating the potential locations of occlusions as well as disparity discontinuities, are approved by the edge drawing algorithm to ensure the local support regions not to cover significant disparity changes. Besides, an additional penalty parameter 𝑃𝑒 is imposed onto the energy function of SGM cost aggregation to specifically handle edge pixels. Furthermore, the final disparities of edge pixels are found by weighting both values derived from the SGM cost aggregation and the U-SURF matching, providing more reliable estimates at disparity discontinuity areas. Evaluations on Middlebury stereo benchmarks demonstrate satisfactory performance and reveal the potency of the hybrid-based dense stereo matching method.

HYBRID-BASED DENSE STEREO MATCHING

Stereo matching generating accurate and dense disparity maps is an indispensable technique for 3D exploitation of imagery in the fields of Computer vision and Photogrammetry. Although numerous solutions and advances have been proposed in the literature, occlusions, disparity discontinuities, sparse texture, image distortion, and illumination changes still lead to problematic issues and await better treatment. In this paper, a hybrid-based method based on semi-global matching is presented to tackle the challenges on dense stereo matching. To ease the sensitiveness of SGM cost aggregation towards penalty parameters, a formal way to provide proper penalty estimates is proposed. To this end, the study manipulates a shape-adaptive cross-based matching with an edge constraint to generate an initial disparity map for penalty estimation. Image edges, indicating the potential locations of occlusions as well as disparity discontinuities, are approved by the edge drawing algorithm to ensure the local support regions not to cover significant disparity changes. Besides, an additional penalty parameter 𝑃𝑒 is imposed onto the energy function of SGM cost aggregation to specifically handle edge pixels. Furthermore, the final disparities of edge pixels are found by weighting both values derived from the SGM cost aggregation and the U-SURF matching, providing more reliable estimates at disparity discontinuity areas. Evaluations on Middlebury stereo benchmarks demonstrate satisfactory performance and reveal the potency of the hybrid-based dense stereo matching method.

Stereo matching based on multi-direction polynomial model

This paper presents a segmentation based stereo matching algorithm. For the purposes of both preserving the shape of object surfaces and being robust to under segmentations, we introduce a new scene formulation where the reference image is divided into overlapping lines. The disparity value and the index of pixels on lines are modeled by polynomial functions. Polynomial functions are propagated among lines to obtain smooth surfaces via solving energy minimizing problems. Finally, the disparity of pixels is estimated from the disparity fields provided by lines. Because lines in multiple directions implicitly segment different objects in an under segmentation region, our method is robust for under segmented regions where it is usually difficult for conventional region based methods to produce satisfactory results. Experimental results demonstrate that the proposed method has an outstanding performance compared with the current state-of-the-art methods. The scene representation method in this work is also a powerful approach to surface based scene representations.

Comfort-driven disparity adjustment for stereoscopic video

Pixel disparity—the offset of corresponding pixels between left and right views—is a crucial parameter in stereoscopic three-dimensional (S3D) video, as it determines the depth perceived by the human visual system (HVS). Unsuitable pixel disparity distribution throughout an S3D video may lead to visual discomfort. We present a unified and extensible stereoscopic video disparity adjustment framework which improves the viewing experience for an S3D video by keeping the perceived 3D appearance as unchanged as possible while minimizing discomfort. We first analyse disparity and motion attributes of S3D video in general, then derive a wide-ranging visual discomfort metric from existing perceptual comfort models. An objective function based on this metric is used as the basis of a hierarchical optimisation method to find a disparity mapping function for each input video frame. Warping-based disparity manipulation is then applied to the input video to generate the output video, using the desired disparity mappings as constraints. Our comfort metric takes into account disparity range, motion, and stereoscopic window violation; the framework could easily be extended to use further visual comfort models. We demonstrate the power of our approach using both animated cartoons and real S3D videos.

An accurate and cost-effective stereo matching algorithm and processor for real-time embedded multimedia systems

Stereo matching is a vision technique for finding three-dimensional (3D) distance information in various multimedia applications by calculating pixel disparities between the matching points of a stereo image pair captured from a stereo camera. The most important considerations in stereo matching are highly accurate results and real-time performance. Thus, this paper proposes an accurate stereo matching algorithm that uses both a census transform algorithm and the sum of absolute differences algorithm in a complementary manner and its real-time hardware architecture. In addition, the proposed algorithm uses a vertical census transform with cost aggregation (VCTCA) to reduce hardware costs while maintaining high matching accuracy. We model the proposed algorithm using C language and verify it in several environments. Using a hardware description language, we implement the proposed hardware architecture and verify it on a field-programmable gate array-based platform to confirm the cost and performance of the hardware. The experimental results show that the proposed algorithm using the VCTCA produces accurate 3D distance information in real environments and reduces the hardware complexity. Thus, the algorithm and its hardware architecture are suitable for real-time embedded multimedia systems.

A stereo matching in complex scene and occlusion handling model

An algorithm is proposed which the tree structure of dynamic programming with self-adaptive parameters and a disparity iterative refinement to improve stereo matching performance. The algorithm model consists of an initialization model and a disparity iterative refinement. To obtain the initial disparity map with “clear” background and whole details, the tree structure of dynamic programming with self-adaptive parameters is presented. A disparity iterative refinement is proposed to estimate the disparity of unreliable pixels in the doubtful regions. The disparity iterative refinement consists of a doubtful regions growing method and a disparity filling based on color similarity. The results of experiment evaluated with Middlebury data sets show that our algorithm can effectively ameliorate “fatting inflating” by occlusion and obtains a high quality of disparity map.