Sift-based improvement of depth imagery

Abstract

Depth Image Based Rendering (DIBR) is a widely used technique to enable free viewpoint television. It utilizes one or more reference texture images and their associated depth images to synthesize virtual camera views. The depth image plays a crucial role for DIBR. However, most of the conventional depth image estimation approaches determine the depth information from a limited set of nearby reference images. This leads to inconsistencies among multiple reference depth images, thus resulting in poor rendering quality. In this paper, we propose an approach that uses the Scale Invariant Feature Transform (SIFT) to improve depth images at virtual viewpoints. We extract SIFT features in left and right reference images, and use feature correspondences to improve the consistency between reference depth images. By doing so, the quality of rendered virtual views can be enhanced.