Domain-specific knowledge-driven pan-sharpening algorithm

Abstract

Pan-sharpening can provide multispectral images with high spatial resolutions, which are useful for many remote sensing image applications. Currently, deep learning technology has been widely used in pan-sharpening. Most of these deep learning-based methods ignore domain-specific knowledge that can improve spatial and spectral performance. Some improved methods adopt an injection structure which injects initial details obtained from panchromatic images into multispectral images through detail mapping. However, the initial details lack frequency-domain information. Moreover, the detail mapping is completed by convolutional neural networks, which lack sufficient nonlinearity to generate rich and diverse details. To solve the above problems, a domain-specific knowledge-driven pan-sharpening framework based on a detail injection structure is proposed, which includes two stages of knowledge-driven initial detail acquisition and data-driven detail mapping. In the first stage, in order to perform better feature reconstruction in the frequency domain, the PAN-MS method is introduced to provide initial details containing frequency-domain information. In the second stage, a newly designed detail-mapping generative adversarial network (GAN) maps initial details to more various output details. Experiments conducted on three public datasets has proven that the proposed algorithm outperforms some state-of-the-art methods in terms of spatial and spectral performance.