Deep generative network for image inpainting with gradient semantics and spatial-smooth attention

Abstract

As a powerful forgery operation in the image content security area, image inpainting based on deep generative networks can yield visually appealing outputs but often produces ambiguous artifacts, especially in boundary and high semantic areas. To address this issue, a novel end-to-end network with gradient semantics and spatial-smooth attention (GS-SSA) is proposed, which combines a gradient learning network and an image inpainting network. The gradient learning network is meant to properly anticipate the gradient semantics in the hole region and get a complete gradient semantic map. The image inpainting network utilizes the complete gradient semantic map to better repair the missing pixels and obtain the final inpainting results. Moreover, spatial-smooth attention is introduced into the image inpainting network, considering both the spatial structural relations and the surrounding information in the hole region. Experimental results on public datasets show the superiority of the proposed method, especially in large-hole region tasks.