Abstract
With the recent development of GPUs, the depth of convolutional neural networks (CNNs) has increased, and its structure has become complex. Hence, it is challenging to deploy them into a hardware device owing to its immense computational cost and memory for storage parameters. We propose a method of pruning a filter located near the density peak, which grasps the density of the filter space for each layer to overcome this problem. The density is calculated in the filter space based on the number of neighboring filters within a certain distance around the filter and the distance to a denser space. Moreover, we do not remove all filters at once, but use a method of pruning a certain number iteratively, so that filters can be evenly pruned in multiple locations with high density inside the filter space. After that, we fine-tune the pruned network to restore their performance. The experimental results show the effectiveness of the proposed method with respect to the other methods using CIFAR-10, and ImageNet dataset on VGGNet and ResNet architecture. Notably, on CIFAR-10, our method reduces 60.8% of FLOPs on ResNet56 with 0.31% validation accuracy improvement. Moreover, we achieve up to 51.9% FLOPs reduction with a little accuracy drop on ImageNet for ResNet34.
Highlights
Convolutional neural networks (CNNs) have been successful in various computer vision tasks such as image segmentation [1], object detection [2], and image generation [3] owing to the advancement of GPUs in the past few years
METHOD we present in detail the novel density peak filter pruning (DPFP) method
In this paper, we propose a method of pruning filters for density peaks by exploring the density of the filter space of each layer
Summary
Convolutional neural networks (CNNs) have been successful in various computer vision tasks such as image segmentation [1], object detection [2], and image generation [3] owing to the advancement of GPUs in the past few years. To address the problems mentioned above, we propose a method to prune the filters located in the density-peak by analyzing the density of the filter space of each layer. The proposed method works with the following framework: 1) We find the number of neighboring filters for each filter of every layer of the pretrained network, calculat the distance to the nearest filter having more neighboring filters than its own, and multiply the obtained two values. The main contributions of this work are as follows: (1) We propose a novel method for pruning filters located at density-peak by calculating the density of the filter space. The proposed method performed as many iterations as desired while pruning only the pre-defined number of filters for each layer in one iteration. 13: Ci+1 × ppi filters with large γi values are selected
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
