Boosting Few-shot visual recognition via saliency-guided complementary attention

Abstract

Despite significant progress in recent deep neural networks, most deep learning algorithms rely heavily on abundant training samples. To address the issue, few-shot learning (FSL) methods are designed to learn models that can generalize to novel classes with limited training data. In this work, we propose an effective and interpretable FSL approach termed Saliency-Guided Complementary Attention (SGCA). Concretely, SGCA aims to boost few-shot visual recognition from two perspectives: learning generalizable feature representations and building a robust classification module in a unified framework. For generalizable representation learning, we propose to explore the intrinsic structure of natural images by training the feature extractor with an auxiliary task to segment foreground regions from background clutter. The guidance signals are provided during training by a saliency detector which highlights object regions in images corresponding to the human visual system. Moreover, for robust classification module building, we introduce a complementary attention mechanism based on the learned segmentation to make the classification module focus on various informative parts of the image. Extensive experiments on 5 popular FSL datasets demonstrate that SGCA can outperform state-of-the-art approaches by a significant margin. In addition, extensions of SGCA to other challenging scenarios, including generalized, transductive and semi-supervised FSL, also verify the effectiveness and flexibility of our proposed approach.