From patch, sample to domain: Capture geometric structures for few-shot learning

Abstract

Few-shot learning aims to recognize novel concepts with only few samples by using prior knowledge learned from the seen concepts. In this paper, we address the problem of few-shot learning under domain shifts. Traditional few-shot learning methods are not directly applicable to cross-domain scenarios due to the large discrepancy of feature distributions across domains. To this end, we propose a novel Hierarchical Optimal Transport network with Attention (HOTA) for cross-domain few-shot learning. The underlying idea is to learn the transferable and discriminative embeddings by taking advantage of the hierarchical geometric structures among image data, ranging from patch, sample to domain. The HOTA framework utilizes a hierarchical optimal transport network to smooth the domain shifts by domain alignment while enhancing the discrimination and the transferability of the embeddings by aligning the patches of images. To further enhance the transferability, HOTA conducts a mix-up data augmentation based on cross-domain attention to capture the relationships of samples in different domains. The extensive experiments on a variety of few-shot benchmark scenarios demonstrate that HOTA outperforms the state-of-the-art methods under both supervised and unsupervised conditions.