Hierarchical few-shot learning with feature fusion driven by data and knowledge

Abstract

Few-shot learning (FSL) aims to use only a few samples to learn a model and utilizes that model to identify unseen classes. Recent, metric-based feature fusion methods mainly focus on the fusion of inter-layer features and show superior performance in solving FSL problems. However, due to the data scarcity in FSL, existing methods still face severe challenges in obtaining high-quality sample features for the improvement of classification performance. In this paper, we propose a hierarchical metric FSL model with comprehensive feature fusion driven by data and knowledge (HFFDK), which is based on intra-layer channel-feature and hierarchical class structure perspectives. First, we utilize the network hierarchy to construct an intra-layer channel feature fusion, which transfers the intra-layer fused features of the higher layer to the lower layer. The model can extract high-quality sample features in a data-driven manner. Moreover, we focus on different levels of granularity to obtain various levels of information, while hierarchical class structures can provide both coarse- and fine-grained information in a knowledge-driven manner. Then, we utilize the coarse-grained information to assist fine-grained recognition. Finally, we optimize hierarchical FSL with coarse- and fine-grained relational constraints and similarity measures among samples. Experiments on four benchmark datasets show that HFFDK achieves state-of-the-art performance.