Stereoscopic imaging: filling disoccluded areas in depth image-based rendering

Abstract

Depth image based rendering (DIBR) is a method for converting 2D material to stereoscopic 3D. With DIBR, information contained in a gray-level (luminance intensity) depth map is used to shift pixels in the 2D image to generate a new image as if it were captured from a new viewpoint. The larger the shift (binocular parallax), the larger is the perceived depth of the generated stereoscopic pair. However, a major problem with DIBR is that the shifted pixels now occupy new positions and leave areas that they originally occupied empty. These disoccluded regions have to be filled properly, otherwise they can degrade image quality. In this study we investigated different methods for filling these disoccluded regions: (a) Filling regions with a constant color, (b) filling regions with horizontal linear interpolation of values on the hole border, (c) solving the Laplace equation on the hole boundary and propagate the values inside the region, (d) horizontal extrapolation with depth information taken into account, (e) variational inpainting with depth information taken into account, and (f) preprocessing of the depth map to prevent disoccluded regions from appearing. The methods differed in the time required for computing and filling, and the appearance of the filled-in regions. We assessed the subjective image quality outcome for several stereoscopic test images in which the left-eye view was the source and the right-eye view was a rendered view, in line with suggestions in the literature for the asymmetrical coding of stereoscopic images.