A Novel Segmentation Based Depth Map Up-Sampling

Abstract

A novel color image segmentation-based depth map upsampling method is proposed in this paper. In this method, the color image is segmented into a certain number of connected regions first. Based on the segmentation result, the target pixels will be interpolated by the seed pixels <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">11</sup> The seed pixels are directly from the low resolution depth maps, i.e., the ones that have depth values. The targets are those without depth and to be interpolated. regionally. In the segmentation part, simple linear iterative clustering is introduced to generate superpixels in the first place. Then, the obtained superpixels will be judged whether they are correct-clustered or not, and the incorrect-clustered ones will be subdivided with an adaptive region-growing strategy. Third, the regions that have no seed will be constantly merged into their nearest neighbors, until seed pixel can be found in each independent region. Finally, adjacent regions that have quite small depth gaps will be united as one. The proposed color image segmentation strictly follows the guidance of the depth; therefore, the segmented regions adhere to the depth boundary well. In the interpolation part, the targets will be interpolated with their surrounding seeds weighted by a joint trilateral filter (JTF). The JTF is constructed by three terms: the color term, the distance term, and the region term, which are driven by the previous segmentation result. Experimental results indicate that our method greatly reduces depth bleeding and depth confusion artifacts, and leads to clear depth boundary in the up-sampled image. Comparisons with the state of art verify the advantages of the proposed method in both visual experience and quantitative evaluations.