Depth upsampling by depth prediction

Abstract

We propose a method for depth upsampling with the aid of high-resolution color image. The key novelty of our method is to exploit a spatio-temporal coherency between the color and depth image sequences. It allows us to perform a depth prediction using the responses from motion estimation in the color image sequence. The predicted depth image is able to estimate the scene boundary regions in the color image; it enables to suppress the influences of color image textures in depth upsampling. We synthesize high-resolution depth images with the help of the estimated scene boundary. Our experiments demonstrate the effectiveness of our method.