Semantic Image Inpainting Using Self-Learning Encoder-Decoder and Adversarial Loss

Abstract

Images are exposed to deterioration over years due to many factors. These factors may include, but not limited to, environmental factors, chemical processing, improper storage, etc. Image inpainting has gained significant attention from researchers to recover the deteriorated parts in images. In this paper, two new techniques for image inpainting techniques using Deep Convolution Neural Networks (CNN) are proposed. In the first technique, a self-tuned Encoder-Decoder architecture based on a Fully Convolution Network (FCN) is used to generate different sized blocks from non-deteriorated image dataset with L2 being used as a loss measure. On the other hand, the second technique is a two-step technique inspired from Context Encoders. In the first step, Context Encoders are trained on non-deteriorated image dataset to select blocks from training images with minimum L2 loss. In the second step, the selected block is applied to Generative Adversarial Networks (GAN) in order to improve the quality of the recovered image. Several simulation examples were made to proof that the performance self-tuned Encoder-Decoder and GAN is the same. Simulations have also shown that the proposed methods have superior performance in recovering missing regions in deteriorated images over other state-of-art techniques. Paris Street View dataset was used for training and validation to validate our results.