Knowledge distillation for object detection based on Inconsistency-based Feature Imitation and Global Relation Imitation

Abstract

Knowledge distillation (KD) is a method that transfers information from a larger network (teacher) to a smaller network (student) to obtain stronger performance without extra computational load. It has made great success in image classification, but only attains trivial improvement in object detection. In this paper, we propose a knowledge distillation scheme for object detection with Inconsistency-Based Feature Imitation (IBFI) and Global Relation Imitation (GRI) schemes. IBFI calculates the difference between the outputs of classification head and regression head to balance the classification and localization abilities of the detector. GRI enables the student to mimic the teacher’s relational information. Extensive experiments have been conducted on popular datasets including MS COCO and PASCAL VOC to validate the effectiveness of our scheme. We achieve 39.9% mAP on MS COCO by using RetinaNet + ResNet-50, which surpasses the baseline by 2.5%.