A Coarse-to-Fine Deep Generative Model With Spatial Semantic Attention for High-Resolution Remote Sensing Image Inpainting

Abstract

Large area array CCD splicing technology and hardware damage lead to the lack of information in high-resolution remote sensing images (HRSIs), which seriously restricts the application of remote sensing images. However, the complex spatial geographical relationship and the serious spatial heterogeneity will cause texture blur and incomplete structure, especially in the boundary and highly textured regions of HRSI. To deal with the challenge, this article proposes a coarse-to-fine deep generative model with a novel spatial semantic attention (SSA) mechanism. SSA is deployed in the encoder and decoder of the network to ensure the continuity of local features and the relevance of global semantic information and embedded in fine network. In the coarse network, a multilevel loss is proposed to provide more accurate information. Fine network performs more detailed image inpainting operations based on coarse network. Extensive experiments on three datasets, including GF2 satellite images, aerial images, and natural ones, demonstrate that our proposed model has better inpainting effect than the state-of-the-art algorithms.