A Disparity Feature Alignment Module for Stereo Image Super-Resolution

Abstract

Recently, the performance of super-resolution has been improved by the stereo images since the additional information could be obtained from another view. However, it is a challenge to interact the cross-view information since disparities between left and right images are variable. To address this issue, we propose a disparity feature alignment module (DFAM) to exploit the disparity information for feature alignment and fusion. Specifically, we design a modified atrous spatial pyramid pooling module to estimate disparities and warp stereo features. Then we use spatial and channel attention for feature fusion. In addition, DFAM can be plugged into an arbitrary SISR network to super-resolve a stereo image pair. Extensive experiments demonstrate that DFAM incorporates stereo information with less inference time and memory cost. Moreover, RCAN equipped with DFAMs achieves better performance against state-of-the-art methods. The code can be obtained at https://github.com/JiawangDan/DFAM.