Abstract
AbstractRecently, records on stereo matching benchmarks are constantly broken by end-to-end disparity networks. However, the domain adaptation ability of these deep models is quite limited. Addressing such problem, we present a novel domain-adaptive approach called AdaStereo that aims to align multi-level representations for deep stereo matching networks. Compared to previous methods, our AdaStereo realizes a more standard, complete and effective domain adaptation pipeline. Firstly, we propose a non-adversarial progressive color transfer algorithm for input image-level alignment. Secondly, we design an efficient parameter-free cost normalization layer for internal feature-level alignment. Lastly, a highly related auxiliary task, self-supervised occlusion-aware reconstruction is presented to narrow the gaps in output space. We perform intensive ablation studies and break-down comparisons to validate the effectiveness of each proposed module. With no extra inference overhead and only a slight increase in training complexity, our AdaStereo models achieve state-of-the-art cross-domain performance on multiple benchmarks, including KITTI, Middlebury, ETH3D and DrivingStereo, even outperforming some state-of-the-art disparity networks finetuned with target-domain ground-truths. Moreover, based on two additional evaluation metrics, the superiority of our domain-adaptive stereo matching pipeline is further uncovered from more perspectives. Finally, we demonstrate that our method is robust to various domain adaptation settings, and can be easily integrated into quick adaptation application scenarios and real-world deployments.
Highlights
Stereo matching is a fundamental problem in computer vision
In order to bridge the domain gaps at these levels, i.e. input image, internal cost volume, and output disparity, we propose a standard and complete domain adaptation pipeline for stereo matching named AdaStereo, in which three particular modules are presented:
We propose a novel domain adaptation pipeline, including three modules to narrow the gaps at input image-level, internal feature-level and output space
Summary
Stereo matching is a fundamental problem in computer vision. We aim at the important but less explored problem of domain adaptation in stereo matching. We plot a histogram for the values in the cost volume, which is a crucial internal representation for stereo matching, and significant differences can be found in the distribution of cost values. In order to bridge the domain gaps at these levels, i.e. input image, internal cost volume, and output disparity, we propose a standard and complete domain adaptation pipeline for stereo matching named AdaStereo, in which three particular modules are presented:
Sign-in/Register to view highlights & summary 
Published Version (
Free)
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
