BAP: Bilateral asymptotic pruning for optimizing CNNs on image tasks

Abstract

Pruning facilitates the acquisition of efficient convolutional neural networks (CNNs) tailored for resource-limited environments. General pruning strategies for CNNs are typically based on the characteristics of filters or their generated feature maps. These strategies assume that the similarity between these elements remains highly consistent. Nevertheless, our analysis has shown that this assumption does not always hold, potentially causing pruning methods to misjudge parameter redundancy. It may erroneously remove crucial network parameters, leading to severe performance loss after network compression. This paper proposes a novel method termed Bilateral Asymptotic Pruning (BAP) for channel optimization. BAP conducts density-based clustering on filters and corresponding feature maps separately to identify redundancy. The resulting sets of channel numbers are then fused for bilateral collaborative pruning. Additionally, we present an exponential moving average method with a logarithmic reduction to mitigate the effects of pruning errors on performance. Finally, the compressing rate of networks gradually increases during the asymptotic process, thereby allowing for the precise generation of pruned networks of different sizes in a single experiment. Extensive experiments demonstrate that our BAP outperforms many state-of-the-art methods. For instance, BAP reduces 51.56% parameters and 59.90% FLOPs1 of ResNet-50 with only 0.5% TOP-5 accuracy loss on ImageNet.