Hierarchical multiple proxy loss for deep metric learning

Abstract

Recent studies on deep metric learning have taken efforts to learn a common feature representation for each category, which is conflict to the intrinsic intra-class variance caused by the fact that latent attributes of the subcategories enlarge the variability of samples from the same class, leading to difficulties of inferring more fine-gained similarity retrieval. This paper proposes a hierarchical multi-proxy representation to establish both the common feature representation of the main-proxy and the complementary representations of their sub-proxies. Specifically, the main-proxy represents the intrinsic characteristics of a certain category for inter-class distinction. Meanwhile, sub-proxies belonging to each main-proxy provide additional capacity for describing intra-class discrepancy. Therefore, a hierarchical multi-proxy representation naturally captures both inter-class and intra-class diversity, yielding a more complete and distinguishable data representation. Significantly, the instances from distinct categories with similar main-proxy attributes can be further separated by the multiple proxy representation due to distinguishable difference among sub-proxy attributes. Correspondingly, we propose Hierarchical Multi-proxy loss as a reliable guidance for deep metric learning. Performance improvement of around 0.5% in precision on the benchmarks as well as the theoretical analysis demonstrate the effectiveness of the proposed method, especially on datasets with high intra-class variance.