Feature independent Filter Pruning by Successive Layers analysis

Abstract

Convolutional neural networks (CNNs) have become deeper and wider over time. However due to low computational power, mobile devices or embedded systems cannot use very deep models. Filter pruning solves this by eliminating redundant filters. Pruning can be performed in a feature dependent or independent manner. Feature dependent methods require extensive time to determine the filter importance as these methods require generation and processing of feature maps for each example. Additionally, in iterative pruning, filter importance is computed several times based on the current state. This increases algorithm execution time further. However, existing feature independent methods are fast, but they perform poor as they compute importance using only current layer filter weights. However, our analysis suggests that both the current and succeeding layer filters are crucial to determine filter importance. We propose ‘Filter Pruning by Successive Layers analysis’ (FPSL), a novel feature independent algorithm, that considers the effect of pruning a filter on the generation of feature maps for the first time. Moreover, FPSL does not require layer-wise retraining, rigorous hyperparameter search for fine-tuning, or human intervention to set the pruning percentage per layer. These make FPSL extremely fast, efficient, and adaptive. Thus it follows iterative pruning and retraining. FPSL outperforms the state-of-the-art (SOTA) methods on extensive experiments with different datasets (CIFAR, ImageNet) and architectures (VGG, ResNet, MobileNet). It decreases the computational burden of VGG16 by half but improves CIFAR10 and CIFAR100 accuracy. Even for ImageNet, FPSL reduces 42.7% floating point operations (FLOPs) while maintaining top-1 accuracy for ResNet50.