Weakly supervised semantic segmentation based on superpixel affinity

Abstract

Weakly supervised semantic segmentation (WSSS) usually employs the method of modifying and extending class activation map (CAM) seeds to achieve semantic segmentation. However, the down-sampling of the network weakens the edge awareness of CAMs, leading to under- or over-activation problems, which affects the segmentation quality. Considering the excellent contour attachment property of superpixels and the high semantic similarity between pixels within the same superpixel, we propose a superpixel affinity-based method that uses multi-scale features to aggregate superpixels with the same semantics, providing complete localization supervision for the generation of CAMs. In order to improve the accuracy of semantic labels for superpixels, we utilize a method of deep feature reorganization to improve the quality of network-generated CAM seeds. The experimental results indicate that the proposed method has achieved satisfactory performance on PASCAL VOC 2012 and MS COCO 2014 datasets.