Dual class representation learning for few-shot image classification

Abstract

Few-shot learning (FSL) models are trained on base classes that have many training examples and evaluated on novel classes that have very few training examples. Since these models cannot be properly fine-tuned on the novel classes using the few training examples, FSL methods use the features extracted by the model trained on the base classes for evaluating it on the novel classes. In this work, we propose a novel few-shot learning approach that focuses on making the network representations generic, to improve its novel class performance. Specifically, our proposed approach introduces dual class information for the base class training images using a base class label and a self-supervised class label. The self-supervised classes are synthesized by applying transformations that alter the image structure and are not dependent on the base classes. Our approach trains the network to recognize both types of classes simultaneously for each training image. This process incorporates the class-invariant information of the self-supervised classes into the model representations and helps the model learn generic features which in turn improves its novel class performance. We also propose a novel approach for creating self-supervised classes that helps the network incorporate a deeper understanding of the structure of the objects and their parts. This approach significantly outperforms existing self-supervision methods in this setting. We empirically demonstrate on multiple datasets that our approach is very effective in addressing the few-shot learning problem and significantly outperforms the single class representation learning approach that uses a single image class label per training image.