Coarse-grained classification and classification at different hierarchy levels are important in many engineering applications. Conventional approaches include post-inference aggregation of fine-grained predictions or multi-task learning with its disadvantages: necessity to incorporate post-prediction mechanisms, possible mistakes between unrelated categories when generic fine-grained models are used, and the need of training multi-task models with joint procedures. To overcome them, this paper proposes a novel method for extracting coarse-grained classifiers from large Convolutional Neural Networks (CNNs) trained on fine-grained tasks with no additional training. Leveraging the knowledge transfer capabilities of CNNs and acknowledging that a model proficient in a challenging task should inherently address easier, hierarchically related problems, the approach finds and aggregates available representatives of coarse-grained classes within existing fine-grained models. Unlike conventional approaches, the proposed technique aggregates the fine-grained model’s weights and not its predictions. As a result, a new coarse-grained model is obtained with the same feature extractor as the original model. The original and new models can be used to create a multi-output model without the multi-task learning. The method eliminates additional post-prediction mechanisms and limits the mistakes between unrelated categories. It can be used with three strategies to find the sets of fine-grained classes for weight aggregation. They are based on external WordNet knowledge or internal network knowledge. Two of the methods leverage the concepts of semantic and visual similarity. Experimental results with MobileNetV2, InceptionV3, ResNet50V2, EfficientNetV2B0 and ConvNeXt-Small trained on ImageNet demonstrate up to 98% accuracy in the coarse-grained classification, which is superior to the post-prediction aggregation.
Read full abstract