Depth map super-resolution using non-local higher-order regularization with classified weights

Abstract

High-order regularization in depth map super-resolution (SR) contributes to producing smoother depth map. However, assigning appropriate weights within regularization term is also important for preserving more detail information. In this paper, a novel and more adaptive depth SR model is proposed by using non-local total generalized variation (NLTGV) with classified weights. A random forest based classifier is trained to classify the pixels of depth map into four categories on the basis of several local structure features, such as gradient magnitude and texture energy extracted from color image, and then the weights within NLTGV are assigned with four groups of parameters corresponding to the four kinds of pixels. Evaluation results demonstrate that the local features make pixels have a good separability, and the classified weights can obviously improve the accuracy of depth map SR.