Temporally consistent adaptive depth map preprocessing for view synthesis

Abstract

In this paper, a novel Depth Image-based Rendering (DIBR) method, which generates virtual views from a video sequence and its associated Depth Maps (DMs), is presented. The proposed approach is especially designed to close holes in extrapolation scenarios, where only one original camera is available or the virtual view is placed outside the range of a set of original cameras. In such scenarios, large image regions become uncovered in the virtual view and need to be filled in a visually pleasing way. In order to handle such disocclussions, a depth preprocessing method is proposed, which is applied prior to 3-D image warping. As a first step, adaptive cross-trilateral median filtering is used to align depth discontinuities in the DM to color discontinuities in the textured image and to further reduce estimation errors in the DM. Then, a temporally consistent and adaptive asymmetric smoothing filter is designed and subsequently applied to the DM. The filter is adaptively weighted in such a way that only the DM regions that may reveal uncovered areas are filtered. Thus, strong distortions in other parts of the virtual textured image are prevented. By smoothing the depth map image, objects are slightly distorted and disocclusions in the virtual view are completely or partially covered. The proposed method shows considerable objective and subjective gains compared to the state-of-the-art one.