Analysis and improvement of joint bilateral upsampling for depth image super-resolution

Abstract

We analyze and propose an improved implementation of joint bilateral upsampling algorithm [5] for depth image super-resolution (SR). The input to the algorithm is a low resolution (LR) depth image and its corresponding high resolution (HR) color image. With the guidance of HR color image, the depth edges can be preserved during the SR process. However, in the original implementation, the sparse sampling operation on the HR color image leads noticeable staircase effect on the generated result. In this paper, we perform a detailed analysis of the original implementation and formulate it as the joint bilateral filtering and nearest neighbor upsampling process. An improved implementation is then proposed to perform dense sampling on the guidance image. It will reduce staircase effect and demonstrated effective both quantitatively and qualitatively in the benchmark dataset.