Spatial Similarity Guidance for Few-Shot Segmentation

Abstract

In this work, we address the challenging issue of few-shot segmentation. Existing methods mainly explore the target object through the semantic similarity between the query and support pixels. However, the semantic similarity often fails to deal well with the target objects with large variations in appearance and the error predictions along the boundary. To this end, we propose a novel spatial similarity guidance network (S2GNet), which adaptively integrates spatial information with semantic information for building a target-aware correlation region to enhance the target object localization. To promote the overall spatial position understanding of the target object, we exploit boundaries as crucial guidance for spatial information. Thus we jointly train a boundary detection task and a segmentation task in an end-to-end way. With that, a target-aware attention module is further proposed to capture the target correlation region by combining the spatial similarity with the semantic similarity for each pair of pixels in the query image, which refines the location of the target object effectively and improves the segmentation performance. Extensive experiments on both PASCAL-5 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i</sup> and COCO-20 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i</sup> datasets show that our approach can achieve state-of-the-art performances.