APRS: Automatic pruning ratio search using Siamese network with layer-level rewards

Abstract

Structured pruning is still a mainstream model compression technique, for its merit of easy to implement and no reliance on specific hardware supporting library. In most previous works, the layer-wise channel pruning ratios were determined empirically. In this paper, we propose an Automatic Pruning Ratio Search (APRS) algorithm that can find the layer-wise optimal pruning ratio within the deep reinforcement learning framework. To solve the coarse-granularity reward problem existing in some previous works like AMC and CACP, a novel layer-level reward function is designed based on the Siamese network architecture for the fine-granularity agent-environment interaction purpose. We use a computationally efficient way to evaluate the effect of pruning action on each single layer. The incurred “backwardness disadvantage” problem has also been analyzed and addressed. The experiments are performed using the VGG-16, and MobileNet-v1 on the CIFAR10/100 and UC Merced Land-use datasets. The results verified that our method can better reveal the underlying sparse sensitivities of different layers in both high redundancy networks and compact networks, so that resulting a higher network accuracy after pruning compared to the traditional methods.