Few-shot learning with long-tailed labels

Abstract

Few-Shot Learning (FSL) is a challenging classification task in machine learning, and it aims to recognize unseen examples of new classes with only a few labeled reference examples (i.e., the support set). The training phase of FSL typically requires a large amount of labeled examples (i.e., the base set) to effectively learn transferable knowledge, but it is usually difficult to obtain sufficient data annotation in practical applications. Existing semi-supervised FSL approaches can learn generalizable representations from partly labeled data, yet they do not sufficiently consider the real distribution of those labeled data. In this paper, we propose a new problem setting termed Few-Shot Learning with Long-Tailed Labels (FSL-LTL) to further consider a more practical semi-supervised scenario where the labeled examples are long-tailed. To effectively address this new problem, we build a novel two-stage training framework dubbed Reweighted Contrastive Embedding (RCE). In the first stage of RCE, we adopt the popular contrastive learning framework to pre-train a reliable network in a self-supervised manner. In the second stage, we integrate the semi-supervised empirical risk into a Weighted Random Sampling (WRS) strategy to fine-tune the pre-trained backbone with the aid of a consistency regularization. Experimental results demonstrate the feasibility of the proposed FSL-LTL problem setting and the superiority of our new RCE method over existing FSL approaches and semi-supervised learning methods. These results also suggest that the RCE approach is a promising solution for addressing the new FSL-LTL problem.