Multi-domain few-shot image recognition with knowledge transfer

Abstract

Few-shot image recognition aims to recognize novel categories with only few labeled images in each class. Existing metric-based and meta-based few-shot learning algorithms have achieved significant progress, but most methods use only visual features. And our humanity can recognize novel categories by learning from prior knowledge, such as semantic information. Based on this intuition, we propose a model that can adaptively integrate visual and semantic information to recognize novel categories. Moreover, the current few-shot learning algorithms fail to generalize to unseen domains due to the domain shift across domains. To reduce the domain shift, we use the weight imprinting strategy as it provides immediate good classification performance and initialization for any further fine-tuning in the future. And we adopt a fine-tuning strategy to simulate various feature distributions under different domains. We conduct extensive experiments to evaluate the effectiveness of the proposed model on three datasets: miniImageNet, CUB, Stanford Dogs. Experimental results demonstrate that our cross-modal scheme gets encouraging improvements in the single-domain and cross-domain few-shot classification tasks.