Leveraging Fine-Grained Labels to Regularize Fine-Grained Visual Classification

Abstract

Fine-grained visual categorization (FGVC) is challenging mainly due to the large intra-class confusion and small inter-class variance in terms of shape, pose, and appearance. We propose the concept of fine-grained label and that any given label can be further classified into some sub-classes as fine-grained labels, and thus samples of each original label are classifed into several sub-classes in which only more familiar samples are given the same fine-grained label. The samples of fine-grained labels have less intra-class confusion and bigger inter-class variance. Besides, fine-grained labels can be obtained through unsupervised means without any domain knowledge or annotations. Instead of training on the fine-grained labels directly, we utilize these free labels as an auxiliary task to regularize the training of the deep learning model. In the test phase, as sub-classes of the original label, the predicted fine-grained labels are used for integration with original labels to get the final classification results. Experiments on the popular CUB-200-2011 dataset demonstrate that employing the proposed fine-grained labels in CNN model improves performance from both training and test phases.