Efficient Confidence-Based Hierarchical Stereo Disparity Upsampling for Noisy Inputs

Abstract

Disparity upsampling techniques aim to restore high-resolution disparity maps from lowresolution disparity inputs. These inputs must be of high quality and are often obtained via complicated passive or active 3D reconstruction methods. Each pixel in the input disparity maps guides the disparity assignment in the upsampling process. The quality of the upsampled results will decrease if the initial disparity inputs are noisy, as the upsampled results are closely related to the initial inputs.We herein propose a hierarchical confidence-based upsampling framework that can be used to obtain relatively high quality upsampled results even under the noisy inputs. Specifically designed confidence measuring schemes are employed in our upsampling process, allowing the disparity assignment of only high-confidence pixels. For an effective depth quality evaluation, we present a novel classification of the confidence according to depth- and texture-related information and develop a confidence examination method with improved precision by combining multiple depth confidence evaluation methods. Our hierarchical pipeline contains 3 steps: confidence-based upsampling, confidence-based fine-tuning and confidence-based optimization. The upsampling combines multichannel information. Fine-tuning is carried out using the stereo texture information. Optimization is conducted utilizing the Markov random field method. All these proposed methods work together to suppress the low-confidence pixels and propagate the high-confidence pixels in the upsampling process. The cumulative error distribution is further analyzed, revealing the effectiveness of our confidence evaluation. Extensive comparison experiments are also performed using both the ground truth and stereo matching disparity maps as inputs to demonstrate the advantage of our framework over state-of-the-art upsampling methods.