Progressive Multi-scale Reconstruction for Guided Depth Map Super-Resolution via Deep Residual Gate Fusion Network

Abstract

Depth maps obtained by consumer depth sensors are often accompanied by two challenging problems: low spatial resolution and insufficient quality, which greatly limit the potential applications of depth images. To overcome these shortcomings, some depth map super-resolution (DSR) methods tend to extrapolate a high-resolution depth map from a low-resolution depth map with the additional guidance of the corresponding high-resolution intensity image. However, these methods are still prone to texture copying and boundary discontinuities due to improper guidance. In this paper, we propose a deep residual gate fusion network (DRGFN) for guided depth map super-resolution with progressive multi-scale reconstruction. To alleviate the misalignment between color images and depth maps, DRGFN applies a color-guided gate fusion module to acquire content-adaptive attention for better fusing the color and depth features. To focus on restoring details such as boundaries, DRGFN applies a residual attention module to highlight the different importance of different channels. Furthermore, DRGFN applies a multi-scale fusion reconstruction module to make use of multi-scale information for better image reconstruction. Quantitative and qualitative experiments on several benchmarks fully show that DRGFN obtains the state-of-the-art performance for depth map super-resolution.