A Channel-level Neural Network Compression Method Based on K-order Statistics

Abstract

At present, deep neural networks (DNNs) have been widely used, and the deployment of DNNs to resource-constrained devices becomes a popular trend, which leads to the problem of compression of deep neural networks. In this paper, we propose a channel-level deep neural network compression method, which aims to remove unimportant channels in the network, reduce the number of neural network parameters, and improve the performance of the compressed neural network. Specifically, to reduce channel redundancy more effectively, our approach introduces K-order statistics in the Batch Normalization (BN) layer, identifies and removes channels with low statistical values to generate a compact network, and improves the accuracy of the compressed network by fine-tuning. Our approach does not change the DNN architecture and does not require special hardware and software accelerators for the generated compression network. Our method was tested on CIFAR-10 image classification public dataset with various DNN models. By comparing with other model compression methods, the effectiveness of our method has been demonstrated.