Image Inpainting Using Multi-Scale Neural Network and Shift-Net

Abstract

Image Inpainting is to repair the images with given mask or missing area and produce a realistic fake image similar to the original image. Methods with Generative Adversarial Networks(GANs) and deep convolutional neural network(CNNs) have achieved great performance on generating visual plausible results. However, those results still have blurry issues on restored region and unpleasant boundaries. Based on these problems, a two-phase neural network method is proposed to produce better results on image inpainting in this paper. The proposed method adopts the Shift-Net [1] as the first phase to generate primary images, which are then fed into refinement network (the second phase) for further detailed texture restoring. This model adopts improved Wasserstein GAN(WGAN-GP) to ensure the training stability and performance on generating high-resolution images. Experiments on shows the proposed model can produce clear and realistic plausible face images. Through the comparison with state-of-the-art methods on both PSNR and SSIM metrics, the proposed method has a better performance on recovering the feature information on missing region.