RPMG-FSS: Robust Prior Mask Guided Few-Shot Semantic Segmentation

Abstract

Few-shot semantic segmentation (FSS) has been developed to perform pixel-level segmentation with only a few dense labeled examples for training, which relieves the expensive annotation problem in traditional segmentation models. Current researches on FSS generally act the labeled masks on the corresponding support images to obtain the class-specific embeddings, and predict the pixel-level masks for query images by matching their pixels to these class-specific embeddings. Their performance is difficult to further break through because of the limited supervision from single-view support images and the neglect of position information from similar pixels between query and support images. To solve these issues, we propose a novel robust prior mask guided model named RPMG-FSS for the challenging FSS task. The core of RPMG-FSS is to produce a robust prior mask with good generalization ability on novel classes to better assist the following query mask prediction. Note that each element in the prior mask corresponds to one pixel in query image. It not only considers the interaction within one view and between multiple views of the support image, but also fuses the top- <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k</i> similarity values to all support pixels and these pixels’ position information. The parameters in RPMG-FSS are optimized with the combination of segmentation loss and multi-view contrastive loss. Comprehensive experiments on two datasets show that our RPMG-FSS achieves outstanding performance comparing with the current popular baselines. The code is released on https://github.com/dxzxy12138/RPMG-FSS/tree/master.