A Flexible Reference-Insensitive Spatiotemporal Fusion Model for Remote Sensing Images Using Conditional Generative Adversarial Network

Abstract

Due to the tradeoff between spatial and temporal resolutions of remote sensing images, spatiotemporal fusion models were proposed to synthesize the high spatiotemporal image series. Currently, spatiotemporal fusion models usually employ one coarse-resolution image acquired on a prediction date and at least another pair of coarse–fine resolution images close to the prediction time as references to derive the fine-resolution image on the prediction date. After years of development, the model accuracy has gained a certain improvement, but nearly, all the models require at least three image inputs and rigid time constraints must be applied to the references to guarantee the fusion accuracy. However, it is not always that easy to collect adequate data pairs for fine-resolution image series simulation in practice because of the bad weather condition or the time inconsistency between the coarse–fine resolution data sources, which causes some difficulties in the actual application. This article introduces the conditional generative adversarial network (CGAN) and switchable normalization technique into the spatiotemporal fusion problem and proposes a flexible deep network named the GAN-based SpatioTemporal Fusion Model (GAN-STFM) to reduce the number of model inputs and broke the time restriction on reference image selection. The GAN-STFM just needs a coarse-resolution image on the prediction date and another fine-resolution reference image at an arbitrary time in the same area for model inputs. As far as we know, this is the first spatiotemporal fusion model that requires only two images as model inputs and puts no restriction on the acquisition time of references. Even so, the GAN-STFM performs on par or better than other classical fusion models in the experiments. With this improvement, the data preparation for spatiotemporal fusion tends to be much easier than before, showing a promising perspective for practical applications.