Learning feature alignment and dual correlation for few‐shot image classification

Abstract

AbstractFew‐shot image classification is the task of classifying novel classes using extremely limited labelled samples. To perform classification using the limited samples, one solution is to learn the feature alignment (FA) information between the labelled and unlabelled sample features. Most FA methods use the feature mean as the class prototype and calculate the correlation between prototype and unlabelled features to learn an alignment strategy. However, mean prototypes tend to degenerate informative features because spatial features at the same position may not be equally important for the final classification, leading to inaccurate correlation calculations. Therefore, the authors propose an effective intraclass FA strategy that aggregates semantically similar spatial features from an adaptive reference prototype in low‐dimensional feature space to obtain an informative prototype feature map for precise correlation computation. Moreover, a dual correlation module to learn the hard and soft correlations was developed by the authors. This module combines the correlation information between the prototype and unlabelled features in both the original and learnable feature spaces, aiming to produce a comprehensive cross‐correlation between the prototypes and unlabelled features. Using both FA and cross‐attention modules, our model can maintain informative class features and capture important shared features for classification. Experimental results on three few‐shot classification benchmarks show that the proposed method outperformed related methods and resulted in a 3% performance boost in the 1‐shot setting by inserting the proposed module into the related methods.