Superpixel Consistency Saliency Map Generation for Weakly Supervised Semantic Segmentation of Remote Sensing Images

Abstract

The weakly supervised semantic segmentation (WSSS) method aims to assign semantic labels to each image pixel from weak (image-level) instead of strong (pixel-level) labels, which can greatly reduce human labor costs. However, there are some problems in WSSS of remote sensing images such as how to locate labels accurately, and how to get precise segmentation edges. To address these issues, we propose a novel framework directly transferring the scene classification model to perform semantic segmentation. We first train a multi-label scene classification network as the encoder to obtain the pre-trained model, then the feature learned by the model is transferred to the decoder. Different from other methods, we propose a saliency map generator instead of the Class Activation Map for more accurate location information by making pixels belonging to the same class lie close together while different classes are separated in feature space. Meanwhile, we take the superpixel patch as processing unit to provide precise boundary inhibition for the saliency map. To assign semantic labels for each patch, combined with extracted salient region, we propose a module responsible for exploiting the consistency of spatial and semantic similarity between different patches. Finally, we incorporate the above two modules to supervise the training process of the decoder without generating pseudo labels as most methods do, thus simplifying the training process. Experimental results show that our method outperforms other weakly supervised approaches on DLRSD and WHDLD datasets with at least a 3% improvement on mean intersection over union.