Improved real-time three-dimensional stereo matching with local consistency

Abstract

Existing stereo matching algorithms are unable to meet the real-time and high-performance dual requirements in practical applications. To address this problem, a novel stereo network is proposed, which utilizes the prior of local disparity consistency to improve the performance of real-time disparity estimation. Based on the initial disparity estimation by the light-weight pyramid matching network, novel spatial consistency refinement (SCR) module and time consistency refinement (TCR) module are designed for further disparity refinement. SCR module propagates the neighborhood high-confidence predictions of sparse sampling to unreliable regions for disparity refinement. A single-layer Dynamic Local Filter (DLF) is designed to realize the content-adaptive propagation, which effectively improves the disparity quality without significantly increasing the burden of computation and memory. For real-time disparity estimation of consecutive frames, novel TCR module is further proposed to refine the disparity estimation based on the time local consistency of disparity. The proposed method is evaluated on the Scene Flow and KITTI 2015 datasets with comprehensive experiments. Experimental results demonstrated that our method can achieve high-accuracy disparity estimation and real-time running speed of over 40 FPS, which significantly outperforms the compared networks with similar runtimes.