Weakly labeled fine-grained classification with hierarchy relationship of fine and coarse labels

Abstract

The current work of fine-grained classification generally depends on a large number of fine labels of images. However, these fine labels are much more difficult to annotate than the coarse labels, which generalize fine labels based on the hierarchy of categories. In this paper, we propose to make fine labels prediction under a weakly supervised setting where a subset of training data is labeled with fine labels and the others only have coarse labels. We aim to explore the hierarchy relationship between coarse classes and fine classes to achieve a better performance on fine-grained classification and meanwhile reduce the heavy dependence on fine labels. To this end, we use convolutional block attention module and multi-scale convolution kernel based feature fusion to generate more effective features from multi-scale convolution kernels and multi-level features. Besides, an adaptive classification module exploits the hierarchy relationship of categories to learn the fine-grained classifier automatically according to the available labels of the training data. Comprehensive experiments on the CIFAR100 dataset, a subset of ImageNet and CUB-200-2011 dataset demonstrate the better fine-grained classification performance of our model.