Multi-scale generative adversarial inpainting network based on cross-layer attention transfer mechanism

Abstract

Deep learning-based methods have recently shown promising results in image inpainting. These methods generate patches with visually plausible image structures and textures, which are semantically coherent with the context of surrounding regions. However, existing methods tend to generate artifacts which are inconsistent with surrounding regions, especially when dealing with complex images. Aiming at the limitations current in deep learning-based methods, this paper proposes a multi-scale generative adversarial network model based on cross-layer attention transfer mechanism. Cross-Layer Attention Transfer Module (CL-ATM) is presented to guide the filling of the corresponding low-level semantic feature map by using the high-level semantic feature map, so as to ensure visual and semantic consistency of inpainting. Meanwhile, a multi-scale generator and the multi-scale discriminators are added into the network structure. Different scales of discriminators have different receptive fields, which enable the generator to produce images with better global consistency and more details. Qualitative and quantitative experiments show that our method has superior performance against state-of-art inpainting models.