Single depth map super-resolution via joint non-local self-similarity modeling and local multi-directional gradient-guided regularization

Abstract

The development of consumer depth cameras makes it possible to acquire depth information of a scene in real-time. However, low resolution and low quality of a depth map has greatly constrained its applications. In this paper, we propose a novel framework for single depth map super-resolution, which considers local and non-local information jointly in the depth map. For the non-local constraint, group-based sparse representation is used to explore non-local self-similarity in the depth map. For the local constraint, a multi-directional gradient-guided regularization is proposed to describe the gradient of the depth map with spatially varying orientations. The former constraint contains the visual artifacts effectively, while the latter restores sharp edge and fine structure. Finally, the two complementary regularizers are jointly casted into a unified optimization framework, where a split Bregman-based technique is developed to tackle the optimization problem. Both quantitative and qualitative evaluations indicate that the proposed method can obtain better reconstruction performance compared with state-of-the-art methods.