RUANet: Residual Ultra-Aggregation Network for Stereo Matching

Abstract

Learning based methods have made great progress in stereo matching, and the key of these algorithms lies in the cost volume construction and the cost aggregation. Existing state-of-the-art models build 4D cost volumes to get accurate results, with huge computational complexity and memory consumption. On the other hand, 3D cost volume based methods sacrifice accuracy in exchange for running speed. This paper aims to optimize the construction mode of cost volumes to improve network efficiency, which is not only close to high-precision method based on 4D cost volume in accuracy, but also comparable to 3D cost volume based methods in time performance. For this reason, we first propose a novel type of cost volume called residual hybrid cost volume(ResHCV) to provide lightweight and efficient matching cost representations. Further, to fuse heterogeneous feature information in ResHCV, we propose residual ultra-aggregation (RUA) algorithm. Our RUANet has achieved stunning results on KITTI datasets while running at 54ms, demonstrating the superiority of our ResHCV and the effectiveness of the RUA module.