A dual-stream framework guided by adaptive Gaussian maps for interactive image segmentation

Abstract

Efficiently embedding user-annotation is a key issue for deep interactive image segmentation. In this paper, we propose a dual-stream framework guided by adaptive Gaussian maps for interactive image segmentation. The network architecture consists of two branches: a traditional fully convolutional neural network that produces coarse segmentation results, and an interactive shape stream that produces target boundary information by integrating user-annotations. The boundary information combined with user-intention can suppress the feature response outside the target, which boosts the performance of coarse segmentation. Additionally, we develop an adaptive Gaussian map with distinct variances to encode user-annotations, which promotes sensitivity to details by adaptively adjusting the affected region of the annotations. Specifically, when the distance between two interactions is smaller than a threshold, we shrink the Gaussian variance of these interactions to enhance the perception of details, thereby improving the segmentation performance of the details. Extensive experiments show that our algorithm effectively reduces the burden of user interaction under the restriction of clear target boundaries and excels at fine-tuning the details with the adaptive Gaussian maps.