Abstract
Most network pruning methods rely on rule-of-thumb for human experts to prune the unimportant channels. This is time-consuming and can lead to suboptimal pruning. In this paper, we propose an effective SuperPruner algorithm, which aims to find optimal pruned structure instead of pruning unimportant channels. We first train a VerifyNet, a kind of super network, which is able to roughly evaluate the performance of any given network structure. The particle swarm optimization algorithm is then used to search for optimal network structure. Lastly, the weights in the VerifyNet are used as the initial weights of the optimal pruned structure to make fine-tuning. VerifyNet is a network performance evaluation; our algorithm can quickly prune the network under any hardware constraints. Our algorithm can be applied in multiple fields such as object recognition and semantic segmentation. Extensive experiment results demonstrate the effectiveness of SuperPruner. For example, on CIFAR-10, the pruned VGG16 achieves 93.18% Top-1 accuracy and reduces 74.19% of FLOPs and 89.25% of parameters. Compared with state-of-the-art methods, our algorithm can achieve higher pruned ratio with less accuracy cost.
Highlights
IntroductionDeep neural networks have achieved remarkable results in various fields (including object recognition [1,2,3,4], object detection [5,6,7], semantic segmentation [8, 9], and autonomous driving [10])
In recent years, deep neural networks have achieved remarkable results in various fields
Liu et al [27] and Wang et al [28] believed that the essence of network pruning is pruning the network structure, rather than pruning unimportant filters. erefore, we propose the SuperPruner algorithm, which automatically prunes the model by finding the optimal network structure
Summary
Deep neural networks have achieved remarkable results in various fields (including object recognition [1,2,3,4], object detection [5,6,7], semantic segmentation [8, 9], and autonomous driving [10]). In order to alleviate this problem, the researchers proposed several CNN compression techniques, including low-rank decomposition [11, 12], parameter quantification [13,14,15], network pruning [16,17,18,19,20,21], and knowledge distillation [22, 23]. Network pruning is widely concerned as a simple and efficient method. E traditional network pruning method [16, 17, 24] consists of three steps: (1) pretraining, (2) filter pruning, and (3) fine-tuning. The pruning rate of each layer (pruning rate affects the network structure) requires a lot of experiments to determine. The pruning results are highly dependent on human experts, which often lead to suboptimal pruning
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
