CSRL-Net: contextual self-rasterization learning network with joint weight loss for remote sensing image semantic segmentation

Abstract

ABSTRACTSemantic segmentation plays a vital role in the intelligent comprehension of remote sensing images (RSIs). However, research on semantic segmentation of RSIs still faces the following challenges: 1) The complexity of ground object structures, including variations in scale and shading environments, poses difficulties for current methods in capturing global context. 2) In long-tail distributed remote sensing data, the scarcity of tail classes makes it difficult for their features to be effectively learned, as features of head classes often overshadow them. To address these issues, we propose a contextual self-rasterization learning network (CSRL-Net) for the semantic segmentation of RSIs. Our approach comprises the following two key components. Firstly, a grid context perception mechanism is developed to collaboratively establish context dependencies within and among multi-scale grids, capturing long-range spatial correlations. Secondly, a joint weight loss function is designed to convert the prior knowledge into weight factors. This loss function combines re-weighting and logit adjustment, giving more attention to tail classes and integrally balancing learning bias. To evaluate the effectiveness of our proposed method, we apply it to the Potsdam, Vaihingen, and GID datasets and compare its performance with current advanced models. Experimental results demonstrate that our method achieves excellent performance in terms of MIoU, mean F1 and OA, with improvements ranging from 0.364% to 1.764% compared to the 17 comparison models. Notably, the proposed joint weight loss significantly improves IoU and F1 for tail classes, resulting in increases of 2.909% (IoU) and 2.085% (F1) on the Vaihingen dataset and 4.697% (IoU) and 4.043% (F1) on the GID dataset.