Abstract
Low spatial resolution is a well-known problem for depth maps captured by low-cost consumer depth cameras. Depth map super-resolution (SR) can be used to enhance the resolution and improve the quality of depth maps. In this paper, we propose a recumbent Y network (RYNet) to integrate the depth information and intensity information for depth map SR. Specifically, we introduce two weight-shared encoders to respectively learn multi-scale depth and intensity features, and a single decoder to gradually fuse depth information and intensity information for reconstruction. We also design a residual channel attention based atrous spatial pyramid pooling structure to further enrich the feature's scale diversity and exploit the correlations between multi-scale feature channels. Furthermore, the violations of co-occurrence assumption between depth discontinuities and intensity edges will generate texture-transfer and depth-bleeding artifacts. Thus, we propose a spatial attention mechanism to mitigate the artifacts by adaptively learning the spatial relevance between intensity features and depth features and reweighting the intensity features before fusion. Experimental results demonstrate the superiority of the proposed RYNet over several state-of-the-art depth map SR methods.
Highlights
Depth map has many applications in practice, such as autonomous driving, virtual reality, 3D reconstruction
The training dataset consisting of 82 RGB-D images is provided by [4]
The HR depth maps and HR intensity images are cropped into 96 × 96 image patches for scaling factors 2× and 4×, and 128 × 128 image patches for scaling factors 8× and 16 ×, respectively
Summary
Depth map has many applications in practice, such as autonomous driving, virtual reality, 3D reconstruction. Recent consumer depth cameras have provided a convenient way to acquire depth maps. The depth maps captured by these cameras usually suffer from low spatial resolution. The resolution of the depth map taken by Kinect v2 is only 512 × 424. Depth map SR aims to reconstruct a high-resolution (HR) depth map from its low-resolution (LR) counterpart. It has inherent ill-posedness, since there may exist multiple HR depth maps that can produce an identical LR depth map after degradation. Numerous depth map SR methods have been proposed to alleviate the ill-posedness, including filter-based methods, optimization-based methods, and learning-based methods.
Sign-in/Register to view highlights & summary 
Published Version (
Free)
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call