SwinSTFM: Remote Sensing Spatiotemporal Fusion Using Swin Transformer

Abstract

Remote sensing images with high temporal and spatial resolutions have broad market demands and various application scenarios. This paper aims to generate high-quality remote sensing image time series for feature mining of the growth quality of traditional Chinese medicine. Spatiotemporal fusion is a flexible method that combines two types of satellite images with high temporal resolution or high spatial resolution to generate high-quality remote sensing images. In recent years, many spatiotemporal fusion algorithms have been proposed, and deep learning-based methods show extraordinary talents in this field. However, the current deep learning-based methods have three problems: 1) most algorithms do not support models with large-scale learnable parameters; 2) the model structure based on convolutional neural networks will bring noise to the image fusion process; 3) current deep learning-based methods ignore some excellent modules in traditional spatiotemporal fusion algorithms. For the above problems and challenges, this paper creatively proposes a new algorithm based on Swin Transformer and linear spectral mixing theory. The algorithm makes full use of the advantages of Swin Transformer in feature extraction, and integrates the unmixing theories into the model based on the self-attention mechanism, which greatly improves the quality of generated images. In the experimental part, the proposed algorithm achieves state-of-the-art results on three well-known public datasets, and has been proved effective and reasonable in ablation study.