Local spatial alignment network for few-shot learning

Abstract

Few-Shot Learning (FSL) aims at recognizing the novel classes with extremely limited samples via transferring the learned knowledge from some base classes. Most of the existing metric-based approaches focus on measuring the instance-level feature similarity but neglect the spatial alignment between different instances, which would lead to poor adaptation with inaccurate local spatial matching, especially for he fine-grained classes. In this paper, we propose a model called Local Spatial Alignment Network (LSANet) to measure the instance-to-class similarity via aligning the local spatial regions in a traversal scanning way. Specifically, the local spatial alignment is achieved by continuously sampling the local patches from the query feature map, where each local patch performs as a kernel to filter the most similar local patches from the support feature maps, obtaining the patch-level similarities between the query instance and the support classes. Then, we propose an information aggregation module to aggregate the patch-level similarities into the class prediction score, where the important patches are highlighted and the backgrounds are diluted. In this way, our model is able to both align the local spatial patches and capture the discriminative information, which benefits in adapting for the novel few-shot classes. To evaluate the effectiveness of the proposed model, we conduct extensive experiments on both coarse-grained and fine-grained datasets. The experimental results show that the proposed LSANet performs competitively on the FSL benchmarks of different granularities.