Multi-semantic hypergraph neural network for effective few-shot learning

Abstract

Recently, Graph-based Few-Shot Learning (FSL) methods exhibit good generalization by mining relations among few samples with Graph Neural Networks. However, most Graph-based FSL methods consider only binary relations and ignore the multi-semantic information of the global context knowledge. We propose a framework of Multi-Semantic Hypergraph for FSL (MSH-FSL) to explore complex latent high-order multi-semantic relations among the few samples. By mining the complex relationship structure of multi-node and multi-semantics, more refined feature representation can be learned, which yields better classification robustness. Specifically, we first construct a novel Multi-Semantic Hypergraph by obtaining associated instances with different semantic features via orthogonal mapping. With the constructed hypergraph, we then develop the Hyergraph Neural Network along with a novel multi-generation hypergraph message passing so as to better leverage the complex latent semantic relations among samples. Finally, after a number of generations, the hyper-node representations embedded in the learned hypergraph become more accurate for obtaining few-shot prediction. In the 5-way 1-shot task of ResNet-12 on mini-Imagenet dataset, the multi-semantic hypergraph outperforms single-semantic graph by 3.1%, and with the proposed semantic-distribution message passing, the improvement can further reach 6.1%.