Abstract
Abstract. Due to the proliferation of Earth Observation programmes, information at different spatial, spectral and temporal resolution is collected by means of various sensors (optical, radar, hyperspectral, LiDAR, etc.). Despite such abundance of information, it is not always possible to obtain a complete coverage of the same area (especially for large ones) from all the different sensors due to: (i) atmospheric conditions and/or (ii) acquisition cost. In this context of data (or modalities) misalignment, only part of the area under consideration could be covered by the different sensors (modalities). Unfortunately, standard machine learning approaches commonly employed in operational Earth monitoring systems require consistency between training and test data (i.e., they need to match the same information schema). Such a constraint limits the use of additional fruitful information, i.e., information coming from a particular sensor that may be available at training but not at test time. Recently, a framework able to manage such information misalignment between training and test information is proposed under the name of Generalized Knowledge Distillation (GKD). With the aim to provide a proof of concept of GKD in the context of multi-source Earth Observation analysis, here we provide a Generalized Knowledge Distillation framework for land use land cover mapping involving radar (Sentinel-1) and optical (Sentinel-2) satellite image time series data (SITS). Considering that part of the optical information may not be available due to bad atmospheric conditions, we make the assumption that radar SITS are always available (at both training and test time) while optical SITS are only accessible when the model is learnt (i.e., it is considered as privileged information). Evaluations are carried out on a real-world study area in the southwest of France, namely Dordogne, considering a mapping task involving seven different land use land cover classes. Experimental results underline how the additional (privileged) information ameliorates the results of the radar based classification with a main gain on the agricultural classes.
Highlights
Due to the proliferation of Earth Observation programmes, information at different spatial, spectral and temporal resolution is collected by means of various sensors (Schmitt, Zhu, 2016)
With the aim to provide a proof of concept regarding the benefits deriving from Learning Under Privileged Information (LUPI) or GKD settings in the field of Earth Observation (EO) analysis, we here propose a Generalized Knowledge Distillation framework to deal with Satellite Image Time Series (SITS) of different nature
We focus on a Land Use Land Cover (LULC) mapping task on a particular study site, for which we dispose of both radar and optical satellite image time series data (SITS) (i.e. Sentinel-1 and Sentinel-2, respectively) at training time, while only radar SITS are accessible at inference/test
Summary
Due to the proliferation of Earth Observation programmes, information at different spatial, spectral and temporal resolution is collected by means of various sensors (optical, radar, hyperspectral, LiDAR, etc.) (Schmitt, Zhu, 2016). With the aim to provide a proof of concept regarding the benefits deriving from LUPI or GKD settings in the field of EO analysis, we here propose a Generalized Knowledge Distillation framework to deal with Satellite Image Time Series (SITS) of different nature. The optical/Sentinel-2 (S2) SITS data constitutes the privileged information while the radar/Sentinel-1 (S1) SITS data is available at both training and test stages Even though such scenario focuses on a very specific case, it meets real world circumstances, where it may happen that a portion of the study site is constantly (or almost all the time) covered by clouds or shadows. The paper is organized as follows: Section 2 introduces the Dordogne study site involved in the experimental study; the Generalized Knowledge Distillation framework to cope with multi-source SITS data, named S1S2GKD, is described in Section 3; the experimental evaluation and the obtained findings are described in Section 4 while Section 5 concludes the work
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callMore From: ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
