Learning efficient multi-task stereo matching network with richer feature information

Abstract

Accurate depth estimation is a research hotspot in the field of stereo vision. The accuracy of stereo matching algorithm directly determines the quality of depth map. Recent researches have transformed stereo matching methods into a supervised learning task. However, the previous methods may have mismatches in the regions of non-textures, boundaries and tiny details. In this paper, we propose a multi-task attention stereo network (MASNet) to integrate the feature information from a stereo image pairs for disparity estimation. Firstly, a segmentation attention head module (SAH) is proposed, which adds semantic segmentation clues for disparity estimation, uses global receptive field to guide network feature extraction learning refined features, and alleviates the negative impact of depth addition of the network. Secondly, we construct a multiple cost volume (MCV) to make full use of the aggregation ability of 3D convolution and provide a better similarity measures for disparity estimation. Thirdly, we embed Top-k pooling layer into the 3D CNN module to obtain the reduced aggregation feature. The feature is fed into the proposed shallow merging network and fused with the intermediate feature to obtain richer low-level features and make up for the comprehensiveness of network neck feature. The results of experiment on Scene Flow, KITTI 2012, and KITTI 2015 datasets show that our proposed network has a significant superiority over state-of-art stereo matching methods.