$M^3\text{Fusion}$: A Deep Learning Architecture for Multiscale Multimodal Multitemporal Satellite Data Fusion

Abstract

Modern Earth Observation systems provide remote sensing data at different temporal and spatial resolutions. Among all the available spatial mission, today the Sentinel-2 program supplies high temporal (every five days) and high spatial resolution (HSR) (10 m) images that can be useful to monitor land cover dynamics. On the other hand, very HSR (VHSR) imagery is still essential to figure out land cover mapping characterized by fine spatial patterns. Understanding how to jointly leverage these complementary sources in an efficient way when dealing with land cover mapping is a current challenge in remote sensing. With the aim of providing land cover mapping through the fusion of multitemporal HSR and VHSR satellite images, we propose a suitable end-to-end deep learning framework, namely $M^3\text{Fusion}$ , which is able to simultaneously leverage the temporal knowledge contained in time series data as well as the fine spatial information available in VHSR images. Experiments carried out on the Reunion Island study area confirm the quality of our proposal considering both quantitative and qualitative aspects.