A Novel Inpainting-Based Layered Depth Video for 3DTV

Abstract

Layered Depth Video (LDV) is recognized as a promising 3D video data representation for supporting advanced 3D video services required in Multiview Video (MVV) systems such as Three-Dimensional Television (3DTV). This representation consists of one full or central view in the form of a video-plus-depth sequence as the main layer, and additional enhancement layers including residual texture and depth data that represent side views. LDV is thus both a derivative of and an alternative to Multiview Video-plus-Depth (MVD) representation by only transmit one full view with associated residual data over the channel. There is a risk, however, of residual data information rapidly increasing as the distance between the center view and side views increases. This occurs when parts of the central view are not visible in the side views, leaving blank spots called disocclusions. These disocclusions may grow larger, which increases the amount of residual data that needs to be sent with the main layer. In this paper, we address the residual layer generation problem. We propose an inpainting-based LDV generation method to reduce the amount of residual data to send by retrieving the missing pixels from the main layer. In the proposed method, we take into account the depth information by distinguishing between foreground and background parts of the scene, at low complexity, during the texture and structure propagation stage of the inpainting process. Experimental results demonstrated the effectiveness of the proposed method.