View Synthesis Based on Background Update with Gaussian Mixture Model

Abstract

AbstractView Synthesis is a key technique for 3-D video and free view video generation. In the traditional 3-D video and free view video, there have to be many real cameras to capture the scene at the large cost. With the help of the view synthesis technique, a limited number of cameras can achieve the goal of multi-view generation. However, some holes will appear in the synthesized views due to the 3-D warping process in the view synthesis system. These holes seriously affect the quality of the synthesized images, especially for the disocclusions which is caused from the occluded regions in the original view may become visible in the virtual view. In this paper, we focus on the disocclustion filling after 3-D warping in view synthesis system. An approach is proposed to fill the discocclusion by using the real background information covered in the original view, which is based on the observation that the covered information in the current frame may be visible in the next frames of the same view. In this approach, the stable texture and depth background reference frames are generated for the left and right view, respectively, which are based on the Gaussian Mixture Model (GMM). Then, in the view synthesis system, a stable background reference frame is merged by the left and right warped images with the corresponding texture and depth background reference frames. Finally, the merged frame is used to fill the disocclusion regions of each merged frame as the background reference frame. The experimental results show that the proposed scheme can achieve better objective quality, especially for the scene with moving objects.KeywordsView SynthesisGaussian Mixture ModelBackground UpdateDepth-Image-Based-Render