High-Order Semantic Decoupling Network for Remote Sensing Image Semantic Segmentation

Abstract

Low-order features based on convolution kernel are easy to be distorted when encountering dramatic view angle transformation and atmospheric scattering in remote sensing images. To address this concern, this paper firstly proposes to operate semantic segmentation of remote sensing images based on the high-order information, which can represent the relative relationship of low-order features and is robust and stable when suffering feature distortion. Besides, semantic decouples have recently been well researched and have achieved significant improvement in image understanding. Thus, in this paper, a High-order Semantic Decoupling Network (HSDN) is proposed to disentangle features by semantics based on high-order features. Specifically, HSDN firstly represents each pixel by calculating the pixel-level affinity as a high-order feature and then clusters these pixels into different semantics. Afterward, an attention-like mask generation module is designed for both intra and inter semantic groups, leading to three kinds of masks, including the Semantic Decoupling Mask (SDM), which utilizes each high-order cluster centroid as a mask to compact features intra cluster and expand different inter clusters, so as to improve semantic disentangle performance to a better extent; Semantic Enhancement Mask (SEM), which records pixel-level relative correlation within a class to sufficiently exploit high-order features and could enhance feature robustness; and Boundary Supplementary Mask (BSM) which aims to process borderline pixels to reduce cluster errors. Finally, by applying masks on pixels both within classes and on borderlines, semantic decoupled features are generated and concatenated to realize segmentation. The quantitative and qualitative experiments are conducted on two large-scale fine-resolution remote sensing image datasets to demonstrate the significant performance of adopting high-order representation. Besides, we also implement numerous experiments to validate the effectiveness of the proposed semantic decouple framework in dealing with complicated and distortion-prone remote sensing image segmentation tasks.