Soft independence guided filter pruning

Abstract

Filter pruning (FP) is an effective method for reducing the computational costs of convolutional neural networks, and herein, the most critical task involves evaluating the significance of each convolutional filter and eliminating the less important ones while minimizing performance degradation. Most existing FP methods consider only local information, which may prevent them from accurately recognizing the most important filters. To address this limitation, we propose the soft filter independence (SFI) method, which leverages global information to identify the most important filters using their magnitude and correlation information in different functional layers. The SFI criterion measures the replaceability of filters from a global perspective in a network. Filters with low independence can be represented effectively by others, so their information can be accurately conveyed by other filters. In addition, we introduce a novel SFI-based asymptotic pruning ratio, which improves training and pruning stability. Compared to the most advanced FP methods, our method enables CNNs to achieve higher pruning rates and better classification performance.