Collaborative Three-Dimensional Completion of Color and Depth in a Specified Area With Superpixels

Abstract

A persistent problem in the three-dimensional (3-D) reconstruction technique is to eliminate blank areas in the 3-D map, which commonly emerges after removing undesired objects, such as dynamic targets or occluded areas. This task is challenging for it is difficult to acquire the coherence between color and depth information, which are both lost for each pixel in the target region. Moreover, creating novel artifacts also needs to be avoided during the completion process. To address these problems, in this paper, we propose a collaborative method to complete the lost area in the color image and its corresponding depth map. In the proposed method, an examplar-based image inpainting technique combined with planarity knowledge is adopted to iteratively repair the texture and structure information. The color completion process provides candidates for plane-fitting on segmented superpixels. The depth filling step extracts image areas requiring reinpainting, which is defined by their gradient differences. The final 3-D map can then be reconstructed from both completed color and depth images. Experiments on real-world scenes demonstrate the success of our method in the completion of 3-D maps.