Application of exemplar-based inpainting in depth image based rendering

Abstract

Depth Image Based Rendering (DIBR) is a technology which converts two dimensional images to three dimension using colour image and its associated depth image. The performance of any DIBR system depends on the perfection of the depth image. Holes/disocclusion will occur in the virtual views generated if the depth map is not perfect. Since holes occur in the virtual views when the intensity changes abruptly in the depth map, smoothening methods were proposed. Since smoothening of the entire depth image destroys the original intensities of the depth map, Adaptive Compensation method (ADC) was proposed. The depth map is classified into three modes depending on the amount of holes. Since texture based inpainting of regions having large number of holes does not produce plausible result, exemplar-based inpainting is proposed. Adaptive edge-based smoothening is done for regions having moderate number of holes and regions with no holes are retained as such. This method results in an increased PSNR and SSIM values compared to existing methods. Apart from reducing the number of holes in the virtual views generated, this method also results in high visual quality virtual views. The increase in PSNR and SSIM comes at the expense of increase in execution time.