Dealing with missing modalities at test time for land cover mapping: A case study on multi-source optical data

Abstract

In recent years, multiple sources of remote sensing data have become increasingly available to monitor Earth’s surface phenomena. However, unlike High Spatial Resolution (HSR) data, Very High Spatial Resolution (VHSR) satellite images remain difficult to collect over large areas due to acquisition costs and a smaller swath. This often compromises the simultaneous use of both sources of data over same study areas for many applications. In this work, we investigate a land cover mapping setting in which both HSR and VHSR are available at the learning stage of a deep neural network while only the HSR data is available at inference time for model inference. We thus propose simple but effective strategies for enhancing the land cover classification in this scenario of incomplete multi-source remote sensing data when the model is deployed.