Semantic Segmentation Optimization Algorithm Based on Knowledge Distillation and Model Pruning

Abstract

This paper proposes a semantic segmentation optimization method combining knowledge distillation and model pruning to reduce the total calculation and volume of deep learning models. Compared with traditional pruning, the unstructured channel pruning of semantic segmentation neural networks combined with knowledge distillation consumes less total training in model compression and less inference time in prediction. The sparse training method of knowledge distillation proposed in this paper improves the whole pruning process while keeping the accuracy of the model basically unchanged. This method improves the speed of the entire pruning process, and reduces the total number of parameters and the total amount of calculations of the model significantly.