TBNet: Stereo Image Super-Resolution with Multi-Scale Attention

Abstract

With 3D products being widely applied, more attention has focused on studying stereo image super-resolution (SR). Current stereo image SR studies mainly aim to improve the performance by the additional information from a pair of low-resolution stereo images. However, it is challenging for stereo image SR to fully exploit self-similarity information from its own image and parallax information between stereo image pairs. In line with these challenges, this paper presents a Two-Branch Network (TBNet) to integrate self-similarity information and parallax information for SR. In the TBnet, a stereo parallax transfer module with an encoder–decoder structure was first proposed to sufficiently transfer multi-scale parallax information and preserve the stereo consistency between stereo images. This paper further presented a residual pyramid self-attention module to employ self-similarity information to take advantage of self-predictive power. Finally, extensive experiments demonstrate the superiority of our model over the state-of-the-art performance in terms of objective and perceptual quality and the accuracy of disparity estimation.