T[formula omitted]L: Trans-transfer Learning for few-shot fine-grained visual categorization with extended adaptation

Abstract

Fine-grained visual categorization requires the ability to distinguish categories with subtle differences, which is also a problem constantly burdened by collecting and labeling massive volume of samples. While transfer learning is extensively used in fine-grained visual categorization, these approaches generally does not work under few-shot regime. As apposed to meta-learning which suffers limitations such as shallow adaptation, and traditional transfer learning which is not task-oriented, we propose a novel transfer learning approach which we refer to as Trans-transfer Learning (T2L). Firstly, a two-phase learning framework is proposed to facilitate task orientation and deep adaptation. Secondly, a novel explainable-learning based procedure is integrated into the second phase to reconfigure the network for training on few samples, after which a deep adaptation procedure is conducted by simultaneously optimizing the feature extractor and the classification boundaries. It is motivated by the fact that a large factor of the over-fitting under few-shot regime, comes from noises that appear in every layers of the activation. The reconfigured model can train solely based on inter-class differences, which not only alleviates over-fitting, but also rediscovers more discriminating features. The proposed approach is comparatively evaluated with state-of-the-art approaches in this field and demonstrates better performances consistently.