Unsupervised image segmentation with robust virtual class contrast

Abstract

Unsupervised Image Segmentation (UIS) is a challenging problem in computer vision that aims to classify pixels in an image into different semantic classes without using any labels. Among the various UIS methods, MaskContrast performs well when dealing with complex objects containing lots of detail. It utilizes prior knowledge from a pretrained saliency detection model to identify foreground pixels and leverages contrastive learning to learn the representations of those foreground pixels. However, this method is not designed to handle situations where foreground pixels belong to multiple semantic classes. To mitigate this problem, we propose a novel method for UIS called Robust Virtual Class Contrast (RVCC), which characterizes foreground pixels based on their “virtual classes” rather than just saliency masks. A virtual class can be viewed as the representation of some unknown object in the training data. Our method employs K-means clustering to find the pseudo virtual class label for every foreground pixel at the start of each training epoch. These pseudo-labels are then used to guide the learning of pixel representations. Additionally, to avoid the overconfidence of pseudo-label prediction, RVCC incorporates an additional regularization term that encourages consistency between the predictions under weak and strong augmentations. Our experiments demonstrate that RVCC outperforms existing baselines on the Pascal VOC2012 and MSRCv2 datasets, showcasing its capability for the UIS problem.