Disparity Values Research Articles

An Underwater Stereo Matching Method: Exploiting Segment-Based Method Traits without Specific Segment Operations

Stereo matching technology, enabling the acquisition of three-dimensional data, holds profound implications for marine engineering. In underwater images, irregular object surfaces and the absence of texture information make it difficult for stereo matching algorithms that rely on discrete disparity values to accurately capture the 3D details of underwater targets. This paper proposes a stereo method based on an energy function of Markov random field (MRF) with 3D labels to fit the inclined plane of underwater objects. Through the integration of a cross-based patch alignment approach with two label optimization stages, the proposed method demonstrates features akin to segment-based stereo matching methods, enabling it to handle images with sparse textures effectively. Through experiments conducted on both simulated UW-Middlebury datasets and real deteriorated underwater images, our method demonstrates superiority compared to classical or state-of-the-art methods by analyzing the acquired disparity maps and observing the three-dimensional reconstruction of the underwater target.

Broad feature extraction and multi-directional imbalanced weighted broad learning system for the unsupervised stereo matching method

The practical application of supervised stereo matching methods is often limited by the requirement of obtaining ground-truth disparity maps in advance. To address this issue, we introduce a novel unsupervised stereo matching method based on the broad feature extraction and multi-directional imbalanced weighted broad learning system. In this study, an end-to-end unsupervised stereo matching method based on broad feature extraction was developed to generate an initial disparity map. Subsequently, a multi-directional imbalanced weighted broad learning system that considers the uneven distribution of disparity values was designed to effectively remove and redefine outlier disparities and enhance the accuracy of generating multiple candidate disparity maps. Multi-direction consistency verification was integrated into the system to further eliminate anomalous disparity values. This verification relies on uniqueness, which guarantees the consistency of the disparity truth values at the same location in various directions. Finally, the local gravity weight method based on a broad feature space was introduced to select a suitable disparity value from neighboring pixels to replace invalid pixel positions. Experiments were conducted under both indoor and outdoor scenarios to demonstrate the effectiveness and versatility of our approach, confirming that it outperforms state-of-the-art techniques, highlighting its superiority.

Refining disparity maps using deep learning and edge-aware smoothing filter

Stereo matching algorithm is crucial for applications that rely on three-dimensional (3D) surface reconstruction, producing a disparity map that contains depth information by computing the disparity values between corresponding points from a stereo image pair. In order to yield desirable results, the proposed stereo matching algorithm must possess a high degree of resilience against radiometric variation and edge inconsistencies. In this article convolutional neural network (CNN) is employed in the first stage to generate the raw matching cost, which is subsequently filtered with a bilateral filter (BF) and applied with cross-based cost aggregation (CBCA) during the cost aggregation stage to enhance precision. Winner-take-all (WTA) strategy is implemented to normalise the disparity map values. Finally, the resulting output is subjected to an edge-aware smoothing filter (EASF) to reduce the noise. Due to its resistance to high contrast and brightness, the filter is found to be effective in refining and eliminating noise from the output image. Despite discontinuities like adiron's lost cup handle or artl's shattered rods, this approach, based on experimental research utilizing a Middlebury standard validation benchmark, yields a high level of accuracy, with an average non-occluded error of 6.79%, comparable to other published methods.

Disparity Refinement for Stereo Matching of High-Resolution Remote Sensing Images Based on GIS Data

With the emergence of the Smart City concept, the rapid advancement of urban three-dimensional (3D) reconstruction becomes imperative. While current developments in the field of 3D reconstruction have enabled the generation of 3D products such as Digital Surface Models (DSM), challenges persist in accurately reconstructing shadows, handling occlusions, and addressing low-texture areas in very-high-resolution remote sensing images. These challenges often lead to difficulties in calculating satisfactory disparity maps using existing stereo matching methods, thereby reducing the accuracy of 3D reconstruction. This issue is particularly pronounced in urban scenes, which contain numerous super high-rise and densely distributed buildings, resulting in large disparity values and occluded regions in stereo image pairs, and further leading to a large number of mismatched points in the obtained disparity map. In response to these challenges, this paper proposes a method to refine the disparity in urban scenes based on open-source GIS data. First, we register the GIS data with the epipolar-rectified images since there always exists unignorable geolocation errors between them. Specifically, buildings with different heights present different offsets in GIS data registering; thus, we perform multi-modal matching for each building and merge them into the final building mask. Subsequently, a two-layer optimization process is applied to the initial disparity map based on the building mask, encompassing both global and local optimization. Finally, we perform a post-correction on the building facades to obtain the final refined disparity map that can be employed for high-precision 3D reconstruction. Experimental results on SuperView-1, GaoFen-7, and GeoEye satellite images show that the proposed method has the ability to correct the occluded and mismatched areas in the initial disparity map generated by both hand-crafted and deep-learning stereo matching methods. The DSM generated by the refined disparity reduces the average height error from 2.2 m to 1.6 m, which demonstrates superior performance compared with other disparity refinement methods. Furthermore, the proposed method is able to improve the integrity of the target structure and present steeper building facades and complete roofs, which are conducive to subsequent 3D model generation.

Optimize Image Steganography Based on Distinction Disparity Value and HMPSO to Ensure Confidentiality and Integrity

To secure the massive amounts of secret messages, the integrity and confidentiality challenges must be addressed. Pressing challenges in the context of a steganography system include security, imperceptibility, and capacity issues. There are trade‐offs between payload and security that have been neglected by researchers, as fixing one issue has been indicated to affect the other, and vice versa. To overcome these challenges, a new steganography scheme based on HMPSO and DDV was introduced. The HMPSO is short for Hénon map and particle swarm optimization, while DDV is short for distinction disparity value. The proposed scheme consisted of four phases with different contributions. The first phase was used to preprocess the secret messages and cover images. In this phase, the secret message has been compressed using the Huffman method. The second phase involved the embedding procedure. This phase consists of two different contributions HMPSO and DDV. The HMPSO is responsible for the optimal pixels’ selection. While the DDV is responsible for the embedding process. The third phase involves the extraction method. In extraction, the reverse processes of the previous steps are implemented. Different evaluation parameters have been used to validate and test the proposed scheme such as SSIM, PSNR, HVS, Histogram analysis, and chi‐square test. Based on the findings, the proposed scheme has solved the confidentiality and integrity challenges and provides a robust steganography system.

Stereo matching method based on high-precision occlusion-recovering and discontinuity-preserving

Stereo matching is the core of machine vision, such as 3D reconstruction, posture perception and driverless technology in the future etc. Currently, there are still great difficulties in improving the matching accuracy of disparity discontinuity and occlusion regions. In this paper, we propose a new high-precision occlusion-recovering and discontinuity-preserving method for stereo matching. We focus on disparity refinement, and present an adaptability-based disparity reconstruction that achieves accurate occlusion-recovering by fusing color aberration, distance, and disparity value. For discontinuity-preserving, disparity-based secondary guided filtering is proposed, then introduce adaptive image enhancement and adaptive Canny edge detection to correct errors in discontinuous regions. The experiments on Middlebury standard dataset show that our algorithm has high overall accuracy, especially in the disparity discontinuity and occlusion regions. This novel method is more direct, efficient, and less sensitive to the accuracy of disparity estimation, and the live-action shooting shows the robustness of our algorithm. The method has the prospect of civil and engineering use, and presents scientific reference for high-precision target identification, target tracking, 3D measurement, etc.

Bidirectional Semi-supervised Dual-branch CNN for Robust 3D Reconstruction of Stereo Endoscopic Images via Adaptive Cross and Parallel Supervisions.

Semi-supervised learning via teacher-student network can train a model effectively on a few labeled samples. It enables a student model to distill knowledge from the teacher's predictions of extra unlabeled data. However, such knowledge flow is typically unidirectional, having the accuracy vulnerable to the quality of teacher model. In this paper, we seek to robust 3D reconstruction of stereo endoscopic images by proposing a novel fashion of bidirectional learning between two learners, each of which can play both roles of teacher and student concurrently. Specifically, we introduce two self-supervisions, i.e., Adaptive Cross Supervision (ACS) and Adaptive Parallel Supervision (APS), to learn a dual-branch convolutional neural network. The two branches predict two different disparity probability distributions for the same position, and output their expectations as disparity values. The learned knowledge flows across branches along two directions: a cross direction (disparity guides distribution in ACS) and a parallel direction (disparity guides disparity in APS). Moreover, each branch also learns confidences to dynamically refine its provided supervisions. In ACS, the predicted disparity is softened into a unimodal distribution, and the lower the confidence, the smoother the distribution. In APS, the incorrect predictions are suppressed by lowering the weights of those with low confidence. With the adaptive bidirectional learning, the two branches enjoy well-tuned mutual supervisions, and eventually converge on a consistent and more accurate disparity estimation. The experimental results on four public datasets demonstrate our superior accuracy over other state-of-the-arts with a relative decrease of averaged disparity error by at least 9.76%.

Vehicle detection system based on shape, color, and time-motion

<span lang="EN-US">Vehicle detection application can assist in-vehicle surveillance functions and have implications for various fields. A vehicle can be identified through the license number attached to its license plate, the color and its shape. Vehicle detection can make use of multimedia sensors so that the design and detection performances can be optimal. Sensor performances are influenced by factors such as the number of multimedia sensors, sensor placement, sensor positioning, and schemes in case of system failure. This study makes use of multimedia sensors with cameras equipped by a phase detection auto focus (PDAF) technology which is like a pair of eyes to see an object. This study analyses 134 vehicles with number detection and various colors to see the effect on the detection and recognition processes. The cars were passed through the camera 10 times at a speed of 10-15 km/hour with various camera distances and positions. Various values and depths of the images were generated. The farther the distance the higher the disparity values. For maximum distance of 50 m, disparity is 6.20×106 and image depth is 16.88×109. Vehicle color influences detection with orange has the best accuracy, but the gray has the largest path error value.</span>

A method for extracting 3D‐element based on local cost metrics resistant to radiometric distortion for realistic media application

AbstractThe intensity values of pairs of stereoscopic images could be affected by radiometric elements such as camera exposure, illumination direction, etc. Therefore, it is a challenging problem to get consistent and exact disparity values under a variety of real‐world environment. Here, the proposed approach extracts disparity values based on a local cost metric and is resistant to global and local radiometric changes. It is designed in a way that disparity values are extracted using local uniformity of pixel values between pairs of stereoscopic images. The proposed metric‐based method is defined as the resistant local cost metric (RLCM) model. As a result of the experiment, the proposed model performed better than the comparative methods under experimental conditions in which the radiometric distortion between pairs of stereoscopic images was significantly different. The average peak signal‐to‐noise ratio (PSNR) value of the RLCM method is 20.0 dB under the global radiometric change condition and 19.3 dB under the local condition, showing the highest value compared to the comparison group. Consequentially, it is demonstrated that the proposed method is less sensitive to various radiometric changes.

Measurement of displacement and top beam attitude angle of advanced hydraulic support based on visual detection

We constructed an advanced hydraulic support test bench and proposed a visual detection-based approach for measuring the displacement and attitude angle of the advanced hydraulic support. Firstly, our methodology involved using a binocular camera for image acquisition and stereo rectification of the advanced hydraulic support. Then, we employed the ORB algorithm to detect feature points and the SGBM matching algorithm to determine the disparity value. Finally, the attitude angle solution algorithm and the triangulation method are written to calculate the support displacement and the top beam attitude angle. Results showed that the advanced hydraulic support had an average absolute error of 8 mm in displacement measurement and 0.4° in attitude angle measurement, meeting the accuracy requirements for fully mechanized mining face operations.

Multi Cost Function Fuzzy Stereo Matching Algorithm for Object Detection and Robot Motion Control

Stereo matching algorithms work with multiple images of a scene, taken from two viewpoints, to generate depth information. Authors usually use a single matching function to generate similarity between corresponding regions in the images. In the present research, the authors have considered a combination of multiple data costs for disparity generation. Disparity maps generated from stereo images tend to have noisy sections. The presented research work is related to a methodology to refine such disparity maps such that they can be further processed to detect obstacle regions. A novel entropy based selective refinement (ESR) technique is proposed to refine the initial disparity map. The information from both the left disparity and right disparity maps are used for this refinement technique. For every disparity map, block wise entropy is calculated. The average entropy values of the corresponding positions in the disparity maps are compared. If the variation between these entropy values exceeds a threshold, then the corresponding disparity value is replaced with the mean disparity of the block with lower entropy. The results of this refinement are compared with similar methods and was observed to be better. Furthermore, in this research work, the v-disparity values are used to highlight the road surface in the disparity map. The regions belonging to the sky are removed through HSV based segmentation. The remaining regions which are our ROIs, are refined through a u-disparity area-based technique. Based on this, the closest obstacles are detected through the use of k-means segmentation. The segmented regions are further refined through a u-disparity image information-based technique and used as masks to highlight obstacle regions in the disparity maps. This information is used in conjunction with a kalman filter based path planning algorithm to guide a mobile robot from a source location to a destination location while also avoiding any obstacle detected in its path. A stereo camera setup was built and the performance of the algorithm on local real-life images, captured through the cameras, was observed. The evaluation of the proposed methodologies was carried out using real life out door images obtained from KITTI dataset and images with radiometric variations from Middlebury stereo dataset.

Multi-directional broad learning system for the unsupervised stereo matching method

The supervised stereo matching methods usually rely on ground truth disparity maps as the training labels, this limits its practical application in many situations. In this study, we propose a novel unsupervised stereo matching method based on a multi-directional broad learning system. A multi-directional broad learning system was constructed to generate multiple candidate disparity maps. During the generation of each candidate disparity map, an update criterion is proposed for the disparity value based on the maximum similarity of the inverse mapping region to remove the abnormal disparity values of the training samples. Subsequently, multi-direction consistency verification is performed to further eliminate abnormal disparity values, which are based on the uniqueness principle of disparity truth values at the same location. Finally, an invalid depth redefinition based on a local gravity weight method is introduced to select the appropriate disparity value to fill the invalid pixel positions from their neighborhood, which is calculated based on the local region of the color, matching cost, and geometric spaces in the stereo images. We provide the results of experiments on both indoor and outdoor scenarios to demonstrate the effectiveness and flexibility of our approach, including comparisons with state-of-the-art methods.

Depth hole filling and optimizing method based on binocular parallax image

AbstractEnvironment perception is one of the most vital function for autonomous robots while performing complex tasks in dynamic environment. In binocular stereo vision algorithm, the calculation of disparity image depends on matching algorithm, and the unmatched points form depth holes in the image. When encountering obstacles with less texture, such as blank walls, large area deep holes will often appear. Although the existing filtering algorithm can solve the problem of small area deep hole, it cannot be applied to large area deep hole. To solve this problem, a depth hole optimization algorithm is proposed, which detects large area depth holes from the image globally, fills the depth holes according to the disparity value distribution, and optimizes the weighted least squares filtering effect. The experimental results show that the average time of the algorithm is only 15 ms.

Ureteral inflammatory edema grading clinical application.

To evaluate the relationship between endoscopic ureteral inflammatory edema (UIE) and ureteral lumen, formulate a preliminary grading method for the severity of UIE, and analyze the impact of different grades of UIE on endoscopic ureteral calculi surgery and prognosis. We retrospectively analyzed 185 patients who underwent ureteroscopic lithotripsy (URSL) for upper urinary tract stones between January 2021 and November 2021. The UIE grade and lumen conditions were assessed by endoscopic observation. The effect of UIE grade on URSL and on patient prognosis were analyzed by multiple linear regression and binary logistic regression. A total of 185 patients were included in the study. UIE grade showed a significant correlation with age, hydronephrosis grading (HG), ureteroscope placement time (UPT), surgery time (ST), hemoglobin disparity value (HDV), and postoperative ureteral stenosis (PUS) (P < 0.05). Logistics regression analysis showed a gradual increase in intraoperative UPT and ST with increase in UIE grade. The severity of UIE showed a negative correlation with improvement of postoperative hydronephrosis (IPH) and the appearance of PUS. HDV was significantly increased in patients with UIE grade 3. UIE grading can be used as an adjunctive clinical guide for endoscopic treatment of upper urinary tract stones. The postoperative management measures proposed in this study can help inform treatment strategy for ureteral stones.

Omnidirectional Stereo Vision Study from Vertical and Horizontal Stereo Configuration

In stereo vision, an omnidirectional camera has high distortion compared to a standard camera, so the camera calibration is very decisive in its stereo matching. In this study, we will perform stereo matching for an omnidirectional camera with vertical and horizontal configuration so that the result of the image's depth has a 360-degree field of view by transforming the image using a calibration-based method. The result is that by using a vertical camera configuration, the image can be stereo matched directly, but by configuring a horizontal image, it is necessary to carry out a different stereo-matching process in each direction. Stereo matching with the semi-global matching method has better image results than block matching with more image objects detectable by the semi-global block matching method with a maximum disparity value of 32 pixels and with a window size of 21 pixels.

Semantic Guided Long Range Stereo Depth Estimation for Safer Autonomous Vehicle Applications

Autonomous vehicles in intelligent transportation systems must be able to perform reliable and safe navigation. This necessitates accurate object detection, which is commonly achieved by high-precision depth perception. Existing stereo vision-based depth estimation systems generally involve computation of pixel correspondences and estimation of disparities between rectified image pairs. The estimated disparity values will be converted into depth values in downstream applications. As most applications often work in the depth domain, the accuracy of depth estimation is often more compelling than disparity estimation. However, at large distances (> 50m), the accuracy of disparity estimation does not directly translate to the accuracy of depth estimation. In the context of learning-based stereo systems, this is mainly due to biases imposed by the choices of the disparity-based loss function and the training data. Consequently, the learning algorithms often produce unreliable depth estimates of under-represented foreground objects, particularly at large distances. To resolve this issue, we first analyze the effect of those biases and then propose a pair of depth-based loss functions for foreground objects and background separately. These loss functions can be tuned and can balance the inherent bias of the stereo learning algorithms. The efficacy of our solution is demonstrated by an extensive set of experiments, which are benchmarked against state of the art. We show on the KITTI 2015 benchmark that our proposed solution yields substantial improvements in disparity and depth estimation, particularly for objects located at distances beyond 50 meters, outperforming the previous state of the art by 10%.

Robust skin microrelief depth estimation using a mobile stereo system.

The skin surface becomes wrinkled and rough due to various internal and external factors. A three-dimensional (3D) analysis of the skin is required to improve skin conditions. Stereophotogrammetry, a noninvasive 3D analysis method, is easy to install and use, but most stereo systems have a fixed baseline and scale. Previous stereo systems are not suitable for observing micro-range skin features. Therefore, we suggest the optimal conditions and methods for the 3D analysis of skin microrelief using a multi-conditioned stereo system. We constructed a nonconvergence model using a mobile device and acquired stereo images under multiscale and multi-baseline conditions. We extracted 3D information of the skin through our process: preprocessing, skin feature extraction, feature matching, and actual depth mapping. We improved the accuracy of the 3D analysis of the skin by using disparity values instead of disparity maps. We compared and analyzed the performances of six local feature detector and descriptor algorithms. In addition, we suggested depth-mapping formulas to estimate the actual depth of the skin microrelief. We confirmed that stereo images with a working distance of 70-75mm and a baseline of 4-8mm are effective for the 3D analysis of skin microrelief. In addition, accelerated KAZE exhibited the best performance for features extraction and stereo matching. Finally, the extracted 3D information was converted to the actual depth, and the performance of the 3D analysis was verified. The proposed system and method that provide texture information are effective for 3D skin disease analysis and evaluation.

Refining Medical Image Steganography Scheme Based on Pixels Disparity Value and Huffman Coding

The steganography is a new and challenging method for transmitting confidential information over the internet utilizing a hidden cover object. Developing an efficient picture steganography system is currently fraught with difficulties because of the limited capacity, weak resilience, and imperceptibility of current steganography systems under development. This problem can only be solved by increasing the picture steganography's capability and security levels. Methods: Based on these variables, this work aims to present the state-of-the-art approach known as the Pixels Disparity Value (PDV) that hides patient information reports in the distinct medical photos for the second party. It's recommended that the procedure be broken down into three primary phases, each with its own set of instructions. Results: The statistical and economic evaluations of the suggested approach.

LANet: Stereo matching network based on linear-attention mechanism for depth estimation optimization in 3D reconstruction of inter-forest scene.

The 3D reconstruction of forests provides a strong basis for scientific regulation of tree growth and fine survey of forest resources. Depth estimation is the key to the 3D reconstruction of inter-forest scene, which directly determines the effect of digital stereo reproduction. In order to solve the problem that the existing stereo matching methods lack the ability to use environmental information to find the consistency of ill-posed regions, resulting in poor matching effect in regions with weak texture, occlusion and other inconspicuous features, LANet, a stereo matching network based on Linear-Attention mechanism is proposed, which improves the stereo matching accuracy by effectively utilizing the global and local information of the environment, thereby optimizing the depth estimation effect. An AM attention module including a spatial attention module (SAM) and a channel attention module (CAM) is designed to model the semantic relevance of inter-forest scenes from the spatial and channel dimensions. The linear-attention mechanism proposed in SAM reduces the overall complexity of Self-Attention from O(n2) to O(n), and selectively aggregates the features of each position by weighted summation of all positions, so as to learn rich contextual relations to capture long-range dependencies. The Self-Attention mechanism used in CAM selectively emphasizes interdependent channel maps by learning the associated features between different channels. A 3D CNN module is optimized to adjust the matching cost volume by combining multiple stacked hourglass networks with intermediate supervision, which further improves the speed of the model while reducing the cost of inferential calculation. The proposed LANet is tested on the SceneFlow dataset with EPE of 0.82 and three-pixel-error of 2.31%, and tested on the Forest dataset with EPE of 0.68 and D1-all of 2.15% both of which outperform some state-of-the-art methods, and the comprehensive performance is very competitive. LANet can obtain high-precision disparity values of the inter-forest scene, which can be converted to obtain depth information, thus providing key data for high-quality 3D reconstruction of the forest.

CEO Pay Disparity and Firm Value

Prior studies have reported mixed results with regard to the effect of CEO pay disparity on firm value. While one stream of studies provides evidence that, based on tournament theory, a higher wage difference between CEOs and VPs positively affects firm value, other studies refer to the managerial power theory and show that the relationship is in fact negative. In this study, I investigate the roots of prior mixed findings to identify what the true effect is, if any. I conclude that the findings of prior studies have been biased due to 1) the existence of biased observations in the ExecuComp database, 2) incomplete research models, and 3) the choices of sample period. I replicate prior studies, properly adjust their models, and identify biased observations in their sample to analyze how CEO pay disparity can affect firm value. I find that firm value is positively affected by the difference between the salary of CEOs and that of their VPs, which could also justify one of the reasons behind the increase in CEO pay disparity in recent years.