A learnable support selection scheme for boosting few-shot segmentation

Abstract

Upon reevaluating recent studies of Few-Shot Segmentation (FSS), a key observation is that the random selection of support images is not always the optimal. In this situation, the support images cannot provide the useful guidance for the segmentation task. Therefore, we argue that a similarity-based support selection scheme, which selects support images according to the similarity between the query and candidate support images, is able to boost the performance of an FSS network. To this end, we propose a Siamese Support Selection Network (SSSN) which can be end-to-end trained along with an FSS network. We also leverage the joint utilization of a Convolutional Neural Network (CNN) and a Transformer network on top of a new feature fusion method to further improve the performance. To our knowledge, none of the similarity-based support selection scheme and the dual-stream network have been utilized for the FSS task before. Experimental results show that our FSS approach outperforms its counterparts on three data sets. In particular, the SSSN is able to boost the performance of an FSS network. We believe that these promising results should be due to the ability of the SSSN to select the top similar support images, which are useful for the FSS task.11Code is available at https://indtlab.github.io/projects/SSSN.