Meta-learning of feature distribution alignment for enhanced feature sharing

Abstract

Meta-learning aims to learn common feature representations between tasks to quickly adapt and perform well when facing new tasks. However, current meta-learning algorithms ignore the differences in feature distributions among tasks. The significant differences in task features may lead to an over-fitting of the model to specific features of certain tasks during the learning process, thereby causing a decrease in adaptability to other tasks. In this paper, we propose a novel method for feature distribution alignment. Specifically, we employ knowledge distillation (KD) to minimize the Kullback–Leibler (KL) divergence between task features, thus promoting greater similarity in the extracted feature distributions. This incentivizes the model to prioritize learning shared parameters and general features across tasks during the training process, rather than task-specific features. Furthermore, we also use an adaptive dropout strategy, which adjusts the training process for the next batch based on the feature distribution of the current task. This method reduces the impact of feature distribution differences during the training iterations. Extensive experiments and analyses demonstrate that our method improves the performance of various meta-learning models in few-shot classification, few-shot fine-grained classification, cross-domain few-shot classification, and non-mutually exclusive few-shot classification.