Frame loss concealment for multi-view video plus depth

Abstract

For future applications like 3DTV and free view point TV, the video data format is likely to comprise of multiple views plus pre-estimated geometric information such as a depth map corresponding to each view. By using multi-view video plus depth (MVD), it is possible to efficiently render virtual views in decoder site. However, the error and packet loss during the transmission will result in the distortion of reconstructed video. To deal with this problem, error concealment is widely used to recover the lost frames and to prevent error propagation. This paper proposes a frame loss concealment method for multi-view video plus depth. First, at the encoder, the proposed method detects the occluded regions of the video sequences, and then the colour motion information of the occluded regions is reused for the corresponding depth blocks during depth map coding. At the decoder, we take advantage of this redundant motion information to recover accurately the lost motion vector for the occluded blocks. After that, the backward 3D warping is used to conceal the non-occluded regions of the lost frame with high image quality. Experimental results show that the proposed method provides significant improvements in terms of both objective and subjective evaluations especially at high packet loss rate.