RGP: Neural Network Pruning Through Regular Graph With Edges Swapping.

Abstract

Deep learning technology has found a promising application in lightweight model design, for which pruning is an effective means of achieving a large reduction in both model parameters and float points operations (FLOPs). The existing neural network pruning methods mostly start from the consideration of the importance of model parameters and design parameter evaluation metrics to perform parameter pruning iteratively. These methods were not studied from the perspective of network model topology, so they might be effective but not efficient, and they require completely different pruning for different datasets. In this article, we study the graph structure of the neural network and propose a regular graph pruning (RGP) method to perform a one-shot neural network pruning. Specifically, we first generate a regular graph and set its node-degree values to meet the preset pruning ratio. Then, we reduce the average shortest path-length (ASPL) of the graph by swapping edges to obtain the optimal edge distribution. Finally, we map the obtained graph to a neural network structure to realize pruning. Our experiments demonstrate that the ASPL of the graph is negatively correlated with the classification accuracy of the neural network and that RGP has a strong precision retention capability with high parameter reduction (more than 90%) and FLOPs reduction (more than 90%) (the code for quick use and reproduction is available at https://github.com/Holidays1999/Neural-Network-Pruning-through-its-RegularGraph-Structure).