Depth Enhancement with Improved Inpainting Order and Smoothing Method

Abstract

Holes and noises are two main defects existing in Kinect’s depth images, which severely restrict the development of depth-based applications. This paper proposes a novel depth enhancement algorithm with improved propagation order and smoothing methods, based on the classic inpainting ideology. The propagation order is optimized by dividing the structural holes into separate connected regions, which is processed one by one according to definition of the pixel priority with confidence term and directional term, which is calculated with the isophote and the pseudo-normal. Furthermore, the improved smoothing method is presented in two distinctive ways: the linear combination of BF and JBF utilizes the different intensity similarity evaluation, while the joint trilateral filter exploits the properties of Gaussian kernel function, in which the proper combination of the colour similarity, depth similarity and geography distance are considered. Experimental results show that our proposed method obtains outstanding performance over other state-of-the-art methods and stage-tests also demonstrate the advantages of our proposed method over their alternatives. This work can benefit a lot for various depth-based applications, especially for the low cost Kinect’s popularity.