PFMNet: Few-Shot Segmentation with Query Feature Enhancement and Multi-Scale Feature Matching

Jingyao Li,Genping Zhao,Zeng Lu,Lianglun Cheng,Zewen Zheng,Jiahong Chen

doi:10.3390/info12100406

Abstract

The datasets in the latest semantic segmentation model often need to be manually labeled for each pixel, which is time-consuming and requires much effort. General models are unable to make better predictions, for new categories of information that have never been seen before, than the few-shot segmentation that has emerged. However, the few-shot segmentation is still faced up with two challenges. One is the inadequate exploration of semantic information conveyed in the high-level features, and the other is the inconsistency of segmenting objects at different scales. To solve these two problems, we have proposed a prior feature matching network (PFMNet). It includes two novel modules: (1) the Query Feature Enhancement Module (QFEM), which makes full use of the high-level semantic information in the support set to enhance the query feature, and (2) the multi-scale feature matching module (MSFMM), which increases the matching probability of multi-scales of objects. Our method achieves an intersection over union average score of 61.3% for one-shot segmentation and 63.4% for five-shot segmentation, which surpasses the state-of-the-art results by 0.5% and 1.5%, respectively.

Highlights

In order to solve the problem of inconsistency of segmenting object scales between support and query set, we propose the multi-scale feature matching module (MFMM) to dynamically scale the support feature target area to a medium-scale size according to the scale factor F (as shown in Equation (5)), which allows the model to focus on the semantic information of the segmenting objects and improve the probability of matching in multiscale feature maps
In order to assess the utility of prior feature matching network (PFMNet) [28], we compare it with the state-of-the-art few-shot semantic segmentation methods
We propose a prior feature matching network for few-shot segmentation, which consists of the query feature enhancement module (QFEM) and the multi-scale feature matching module (MFMM)

Summary

Introduction

The few-shot learning methods are applied to the above aspects and to the semantic segmentation [9,10,11,12,13,14,15,16,17,18,19,20] task. Semantic segmentation methods such as fully convolutional networks (FCN) [15], U-shape convolutional network (U-net) [16], SegNet [9], pyramid scene parsing network (PSPnet) [19], and AMP [13] are not designed to deal with rare and unseen classes. The more annotation with category, the more precise expressions that could have been extracted by the model from the massive data distribution. With a decrease in the number of labeled data, the expression would have shown a progressive reduction

Objectives

Methods

Results

Conclusion