Irregular mask image inpainting based on progressive generative adversarial networks

Abstract

ABSTRACT Existing image inpainting algorithms have achieved good results in regular mask inpainting tasks, but there are still many limitations for irregular mask, and the inpainting results lack edge consistency and semantic correctness. To address these problems, we designed a progressive generative adversarial network in this paper. The generative network mainly contains a progressive feature generation module and an adaptive consistent attention module, and the discriminator network uses SN-PatchGAN. The network performs recursive inpainting from the edge to the centre, and the repaired features are used as the conditions for the next feature generation, which makes the constraints of the central content gradually strengthened. In order to obtain distant information from the feature maps as well as to consider the possible problem of inconsistency between feature mappings under different recursions when using attention directly in the progressive inpainting network, we designed an adaptive consistent attention module in this paper, which can adaptively combine scores obtained from different recursions to capture more feature information. The discriminator used in this paper is SN-PatchGAN, which directly calculates the hinge loss of each point on the output graph and can focus on different locations as well as different semantics. Comparing this paper's method on CelebA dataset and CMP Facade dataset with the latest work, the experimental results show that this paper's method performs well in the irregular mask image inpainting task, and the inpainting results have better performance in terms of edge consistency, semantic correctness, and overall image structure.