$\text{A}^{3}$Seg: An Annealing Augmented and Aligned Semantic Segmentation Network for High Spatial Resolution Remote Sensing Images

Abstract

Semantic segmentation of high spatial resolution (HSR) remote sensing images (RSIs) plays an important role in many applications. However, HSR RSIs have significantly larger spatial sizes than typical natural images, which results in fewer valuable samples when training models. In addition, fusing multiscale features is the key step in obtaining features with strong semantic and high spatial information. However, current feature fusion methods are too straightforward to address misalignment issues. To handle these two problems, we propose an annealing augmented and aligned segmentation network, named <inline-formula><tex-math notation="LaTeX">$\text{A}^{3}$</tex-math></inline-formula>Seg. Specifically, we propose an annealing online hard example mining (AOHEM) strategy to automatically select more valuable samples during the training stage. Based on AOHEM, a contextual augmentation block is proposed to extract sufficient contextual information using three different attention mechanisms in consideration of three different feature properties. Finally, we propose a novel feature alignment block to fuse features at different levels by alignment with the guidance of a salient feature. Experimental results on three different HSR RSIs datasets demonstrate that the proposed method outperforms the state-of-the-art general semantic segmentation methods with a better tradeoff between accuracy and complexity.