Generating Sentinel-2 all-band 10-m data by sharpening 20/60-m bands: A hierarchical fusion network

Abstract

Earth observations from the Sentinel-2 mission have been extensively accepted in a variety of land services. The thirteen spectral bands of Sentinel-2, however, are collected at three spatial resolutions of 10/20/60 m, and such a difference brings difficulties to analyze multispectral imagery at a uniform resolution. To address this problem, we developed a hierarchical fusion network (HFN) to sharpen 20/60-m bands and generate Sentinel-2 all-band 10-m data. The deep learning architecture is used to learn the complex mapping between multi-resolution input and output data. Given the deficiency of previous studies in which the spatial information is inferred only from the fine-resolution bands, the proposed hierarchical fusion framework simultaneously leverages the self-similarity information from coarse-resolution bands and the spatial structure information from fine-resolution bands, to enhance the sharpening performance. Technically, the coarse-resolution bands are super-resolved by exploiting the information from themselves and then sharpened by fusing with the fine-resolution bands. Both 20-m and 60-m bands can be sharpened via the developed approach. Experimental results regarding visual comparison and quantitative assessment demonstrate that HFN outperforms the other benchmarking models, including pan-sharpening-based, model-based, geostatistical-based, and other deep-learning-based approaches, showing remarkable performance in reproducing explicit spatial details and maintaining original spectral features. Moreover, the developed model works more effectively than the other models over the heterogeneous landscape, which is usually considered a challenging application scenario. To sum up, the fusion model can sharpen Sentinel-2 20/60-m bands, and the created all-band 10-m data allows image analysis and geoscience applications to be authentically carried out at the 10-m resolution.