Comparison Analysis for Pruning Algorithms of Neural Networks

Abstract

Deep learning has been widely applied in many fields, such as big data analysis, natural language processing, and image processing, etc. However, the number of parameters in the deep learning model is often too large that the model would be difficult to apply portable devices. Therefore, the researches about pruning algorithms for optimizing the deep learning model has great significance. This paper conducts an in-depth analysis of the pruning algorithm in the model compression method. Besides, we classify and summarize the structure, scheduling, and scoring of the pruning algorithms. And then, the compression effects of some mainstream pruning algorithms on some deep neural networks including accuracy are compared and analyzed. Finally, the future development of the pruning algorithm is promising. Regardless of the different pruning approaches, the mainstream pruning algorithms can all effectively compress the size of the network model while maintaining basic accuracy. However, one pruning algorithm has a different performance when it is applied to different network models and datasets. The future pruning algorithm can be designed in terms of network sparsity structure, scheduling, and parameter ranking criteria. Moreover, the datasets and models are also required to be considered when the corresponding high-performance pruning strategies are used.