Improvement of Efficiency in Evolutionary Pruning

Abstract

Filter pruning has been widely applied on compression of deep convolutional neural networks. Especially, the filter pruning approach using evolutionary algorithms has exhibited powerful capabilities of automatic model compression and global optimal solution search. However, the exponential increasing of search space brought by the extremely large scale of deep networks makes it difficult to achieve a well-compressed model with limited time and energy. In this paper, we design an effective method aiming at improving the efficiency in evolutionary pruning, called IEEPruner. Multiple pruning criteria are first utilized to initialize an efficient search space, and then a novel encoding scheme is proposed to further reduce the search space. After that, we formulate the search of the optimal pruned structure as a multiobjective optimization problem, and integrate a multiobjective evolutionary algorithm to achieve a good balance between model size and accuracy in an automatic manner. Thus, we automatically search for the optimal pruned model in this search space. Extensive experiments on widely used convolutional neural networks including VGGNet, ResNet, and LeNet have demonstrated the efficiency of the proposed evolutionary pruning method.