Fusing Landsat-7, Landsat-8 and Sentinel-2 surface reflectance to generate dense time series images with 10m spatial resolution

Abstract

ABSTRACT Due to the influence of the external atmospheric environment and the physical constraints of the sensors, it is difficult for existing medium spatial resolution sensors to produce intensive time-series images. The easy accessibility and similar sensor configuration of Landsat and Sentinel-2 data make them suitable for synthesizing time-series images. Currently, some deep learning-based studies have achieved significant success in multi-sensor fusion, but these deep-learning fusion networks only use single features that are not enough to produce intensive time-series images. Shallow features are easy to extract but have weak representation, while the deep network can extract abstract features with strong representation ability but faces the problem of gradient disappearance and model degradation. To address these problems, we propose a novel spatial-temporal-spectral integration model that can simultaneously fuse Landsat-7,8 and Sentinel-2 sensors. First, an enhanced residual dense network (ERDN) is proposed to improve Landsat’s four bands (Blue, Green, Red, and Near-infrared) to 10 m spatial resolution. ERDN’s residual dense structure utilizes abundant shallow features (i.e. texture and spectral information) and deep features (i.e. angles of view and aspect ratios of objects) to reconstruct spatial structures. At the same time, the introduced high-resolution images can provide reliable spatial auxiliary information for the missing image parts. Second, we employ a time-series-based reflectance adjustment (TRA) method to reduce the reflectance differences between Landsat and Sentinel-2 images. The fusion results indicate that our fusion framework can not only fill the gap caused by the Landsat-7 scan-line corrector (SLC) failure but also restore more spatial details than previous methods. More importantly, it can predict tiny reflectance changes in land cover. Overall, our study can produce continuous reflection observations with a higher time density and provides important data guarantee for extracting dynamic changes in land coverages.