A layered inpainting method for virtual view synthesis

Abstract

In this paper, we presented a new approach in view synthesis based on the concept of missing area being filled with the background parts. Viewing synthesis is a method to generate a virtual scene in a desired view point. Previous approaches generally assumed that a virtual camera located between a pair of reference cameras. In contrast, we consider the case where the virtual camera located outside the reference cameras. The contents consist of two parts: layer separation and layer inpainting. In the layer separation, it extracts the background part from the images. The part is essential since layer inpainting inferred the result of it. After that, layer inpainting is implied to fill the occlusion region with the background layer. Therefore, it significantly improve the quality of inpainting in view synthesis. It should be noted that this research has only examined with image set from the Microsoft research yet. It has clear boundary between object and background, but patch based inpainting is applied in this framework. It is suggested that future studies can use better inpainting method in order to enhance the accuracy of indicator.