Structure Characteristic-Aware Pruning Strategy for Convolutional Neural Networks

Abstract

Convolutional Neural Networks have received considerable attention over the past few years, and they are widely used in various fields. However, the computational complexity and excessive storage space caused by over-parameterized of these networks are also being serious. This will limit further development of deep learning in resource-constrained devices such as the mobile terminal. It has been proposed that pruning is a useful method to optimize networks, whereas there is no contribution focus on utilizing the traits of the network to guide pruning. The primary purpose of this paper is to research the different effects of layers with varying depth and the different number of parameters during the pruning process. By taking advantage of the characteristics of network structure itself, we propose a new pruning strategy with three variants to effectively prune CNNs which can automatically determine the most appropriate sparsity for each conv-layer. Using iterative pruning and retraining, we evaluated our strategy on InceptionV3 and ResNet50. We analyzed our three variants and made a comparison between each other.