A neural refinement network for single image view synthesis

Abstract

Recent years have seen an increasing interest in single image view synthesis. It remains however a challenging task due to the lack of comprehensive colour and depth information from different views. In this paper, we propose a novel view synthesis approach that incorporates a Neural Image Refinement Network (NIRN) and generates both depth and colour images for the target view in an end-to-end manner. The appearance of the colour image greatly benefits from the generated depth image as it provides an intermediate projection relationship for the object in the 3D world. Since the direct application of geometric projection mapping will result in empty regions and/or distortions, our approach proposes to embed a novel refinement network into the view synthesis pipeline for improved performance. Experimental results on three publicly available datasets demonstrate that our NIRN outperforms other state-of-the-art view synthesis methods.