Filter pruning via separation of sparsity search and model training

Abstract

Network pruning has been widely used in the field of model compression and inference acceleration for convolutional neural networks(CNN). Existing methods generally follow a “training-pruning-retraining” paradigm, known as a three-stage pipeline. However, it cannot play an effective role in a pre-trained model with a larger pruning rate. In addition, prevailing methods usually set pruning rates as super parameters, which fail to consider the sensitivity of different convolution layers. In this paper, a novel pruning approach, based on the separation of sparsity search and model training(SST), is proposed to solve the above problems. Specifically, an evolutionary algorithm is introduced into the process of searching for the most suitable number of pruned filters for every layer. After obtaining the best sparsity structure, a new pruning strategy, called the one-pruning pipeline, is utilized to prune the pre-trained model. Experiments on multiple advanced CNN architectures show that SST can greatly improve the pruning rate with a slight loss of accuracy, which is found to universally reduce more than 60% FLOPs on CIFAR-10. Notably, on ILSVRC-2012, pruning based on ResNet18 reduces FLOPs by 42.8%, while top-1 and top-5 accuracy only lose 1.19% and 0.62%, respectively.