Multi-granularity generative adversarial nets with reconstructive sampling for image inpainting

Abstract

Image inpainting based on deep neural network has drawn more attention with the development of deep learning, then Generative adversarial nets(GANs) which is a combination of multiple deep networks has been applied to image inpainting and achieved top-level performance. However, GANs-based inpainting method always suffers from unstable training and vanishing gradient. In this paper, we present an image inpainting method based on GANs with reconstructive sampling and multi-granularity generative adversarial strategy. The key idea is to solve unstable training and vanishing gradient by proposed reconstructive sampling, in which we sample from reconstructive distribution than low-dimension noise. It has been proved that reconstructive sampling is effective to avoid unstable training and gradient vanish. Then, multi-granularity generative adversarial strategy, which is decomposed into two steps, is adopted to make inpainted image more continuous and realistic in texture structure and vision, respectively. Extensive experiments show that ours brings substantial improvements over other state-of-the-art image inpainting algorithms in distortion and perception evaluation. Besides, comparisons on stability with baselines show that our method gains better stability during image inpainting.