Ticino: A multi-modal remote sensing dataset for semantic segmentation

Abstract

Multi-modal remote sensing (RS) involves the fusion of data from multiple sensors, such as RGB, Multispectral, Hyperspectral, Light Detection and Ranging, Synthetic Aperture Radar, etc., each capturing unique information across different regions of the electromagnetic spectrum. The fusion of different modalities can provide complementary information, allowing for a comprehensive understanding of the Earth’s surface.Multi-modal RS image segmentation leverages various RS modalities to achieve pixel-level semantics classification. While deep learning has demonstrated promise in this domain, the limited availability of labeled multi-modal data poses a constraint on leveraging data-intensive techniques like deep learning to their full potential. To address this gap, we present Ticino, a novel multi-modal remote sensing dataset tailored for semantic segmentation.Ticino includes five modalities, including RGB, Digital Terrain Model, Panchromatic, and Hyperspectral images within the visual-near and short-wave infrared spectrum. Specifically annotated for Land Cover and Soil Agricultural Use, the dataset serves as a valuable resource for researchers in the field. Additionally, we conduct a comparative analysis, comparing single-modality with multi-modality deep learning techniques and evaluating the effectiveness of early fusion versus middle fusion approaches.This work aims to facilitate future research efforts in the domain by providing a robust benchmark dataset and insights into the effectiveness of various segmentation approaches.