Enhancing the utilization of uncertain pixels in semi-supervised semantic segmentation

Abstract

In semi-supervised semantic segmentation, determining the correct label for uncertain pixels is crucial yet challenging. The recently proposed virtual category (VC) learning achieves excellent performance by creating a potential category (PC) set to exploit the valuable part of the uncertain pixels effectively. However, the potential category set in the existing method is not accurate enough at the beginning of iterations, which could lead to less accurate segmentation results. To better utilize uncertain pixels, we propose to improve the strategy of constructing the potential category set. Specifically, we adjust the threshold based on the model’s optimization status and split the pseudo-labels into low/high-confidence areas to create a more appropriate PC set. At the same time, due to uncertainty in boundary pixels, it is challenging to achieve precise object segmentation, and there is currently no method available to optimize these areas in semi-supervised segmentation tasks. Therefore, we propose a entropy-based method to identify boundary areas and design a novel boundary refinement network to process labeled and unlabeled data separately to optimize segmentation. The experimental results demonstrate that our proposed method excels in accurately segmenting the boundary areas of the targets. Furthermore, it achieves state-of-the-art semi-supervised segmentation on the Pascal VOC and Cityscapes datasets.