Few-shot Benchmarks Research Articles

Bilaterally Normalized Scale-Consistent Sinkhorn Distance for Few-Shot Image Classification.

Few-shot image classification aims at exploring transferable features from base classes to recognize images of the unseen novel classes with only a few labeled images. Existing methods usually compare the support features and query features, which are implemented by either matching the global feature vectors or matching the local feature maps at the same position. However, few labeled images fail to capture all the diverse context and intraclass variations, leading to mismatch issues for existing methods. On one hand, due to the misaligned position and cluttered background, existing methods suffer from the object mismatch issue. On the other hand, due to the scale inconsistency between images, existing methods suffer from the scale mismatch issue. In this article, we propose the bilaterally normalized scale-consistent Sinkhorn distance (BSSD) to solve these issues. First, instead of same-position matching, we use the Sinkhorn distance to find an optimal matching between images, mitigating the object mismatch caused by misaligned position. Meanwhile, we propose the intraimage and interimage attentions as the bilateral normalization on the Sinkhorn distance to suppress the object mismatch caused by background clutter. Second, local feature maps are enhanced with the multiscale pooling strategy, making the Sinkhorn distance possible to find a consistent matching scale between images. Experimental results show the effectiveness of the proposed approach, and we achieve the state-of-the-art on three few-shot benchmarks.

CRTED: Few-Shot Object Detection via Correlation-RPN and Transformer Encoder–Decoder

Few-shot object detection (FSOD) aims to address the challenge of requiring a substantial number of annotations for training in conventional object detection, which is very labor-intensive. However, the existing few-shot methods achieve high precision with the sacrifice of time for exhaustive fine-tuning or have poor performance in novel-class adaptation. We presume the major reason is that the valuable correlation feature among different categories is insufficiently exploited, hindering the generalization of knowledge from base to novel categories for object detection. In this paper, we propose few-shot object detection via Correlation-RPN and transformer encoder–decoder (CRTED), a novel training network to learn object-relevant features of inter-class correlation and intra-class compactness while suppressing object-agnostic features in the background with limited annotated samples. And we also introduce a four-way tuple-contrast training strategy to positively activate the training progress of our object detector. Experiments over two few-shot benchmarks (Pascal VOC, MS-COCO) demonstrate that our proposed CRTED without further fine-tuning can achieve comparable performance with current state-of-the-art fine-tuned works. The codes and pre-trained models will be released.

Powerful embedding networks for few-shot image classification

Few-shot image classification commits to recognizing new concepts from limited annotated samples. Our insight is to obtain a sufficiently powerful embedding network (PEN) to solve few-shot classification tasks. We propose a method to tackle the few-shot classification tasks, namely PENs for few-shot image classification. The key core of PEN is gaining a well-trained embedding network that is capable of extracting strong discriminating representations to represent an image by utilizing two strategies. One strategy is that the multi-scale feature maps are fused instead of only utilizing the final top-level feature maps. We consider that low-level features also play an important role instead of only utilizing top-level representations. Another significant strategy is knowledge distillation (KD). The characteristics of KD can help us get better performance of an embedding network to extract features. Finally, a distance function is employed to classify unlabeled samples. Comprehensive experiments are conducted on few-shot benchmarks. Our method achieves promising performances. The results demonstrate that KD and future fusion are beneficial to gain an expected embedding network for few-shot classification tasks.

Improved prior selection using semantics in maximum a posteriori for few-shot learning

Few-shot learning is to recognize novel concepts with few labeled samples. Recently, significant progress has been made to address the overfitting caused by data scarcity, especially those on modeling the distribution of novel categories given a single point. However, they often deeply rely on the prior knowledge from base set, which is generally hard to define, and its selection can easily bias the learning. A popular pipeline is to pretrain a feature extractor with base set and generate statistics from them as prior information. Since pretrained feature extractor cannot extract accurate representations for categories have never seen, and there is only 1 or 5 support images from novel categories, making it hard to acquire accurate priors, especially when they are far away from the class center. To address these issues, in this paper, we base our network on Maximum a posteriori (MAP), proposing a strategy for better prior selection from base set. We specially introduce semantic information, which are learned from unsupervised text corpora and easily available, to alleviate biases caused by unrepresentative support samples. Our intuition is that when the support from visual information is biased, semantics can provide strong prior knowledge to assist learning. Experimental results on four few-shot benchmarks also show that it outperforms the state-of-the-art methods by a large margin, improves around 2.08%∼12.68% than the best results in each dataset on both 1- and 5-shot tasks.