Research on Knowledge Distillation Algorithm of Object Detection

Abstract

The Algorithms of object detection are usually difficult to deploy on low-end devices due to the large amount of computation, but knowledge distillation can solve this problem by training small models to learn the already trained complex network models, realizing model compression, and effectively reducing the amount of computation. How to transfer rich knowledge from teachers to students is a key step in the knowledge distillation. To solve this problem, this paper uses the knowledge of the teacher to guide the student network training in feature extraction, target classification and frame prediction, and proposes a distillation algorithm based on multi-scale attention mechanism, which uses attention mechanism to integrate different scale features. The correlation of features between different channels is learned by assigning weights to the features of each channel. The distillation algorithm proposed in this paper is based on YOLOv4, so it can strengthen the student network to learn the key knowledge of the teacher network, and make the knowledge of the teacher network How to the student network better. Experimental analysis shows that it can effectively improve the detection accuracy of the student network. The size of the model is only 6.4% of the teacher network, but the speed is increased by 3 times, and mAP is 5.7% higher than the original student network and 2.1% lower than the teacher network.