Robust filter pruning guided by deep frequency-features for edge intelligence

Abstract

Listen

Translate article

With the popularity of mobile applications, smart terminals require the deployment of lightweight deep neural networks (DNNs). However, those lightweight DNNs deployed on resource-constrained edge devices, have serious security vulnerabilities to adversarial examples. Pruning methods have been widely used to obtain lightweight DNNs, which mainly focus on improving classification accuracy, but ignore robustness. To this end, we present a Fourier space-based robust pruning algorithm (FSRP). We find that a robust network pays more attention to the low-frequency region of the adversarial examples’ feature maps, but ignores the high-frequency region. According to this finding, we design a filter robust indicator, a ratio of low-frequency components to high-frequency components of the feature map, to guide the pruning process. With this new robust pruning criterion, we adopt a strategy of local pruning that removes the filters with low robustness layer by layer in the model. Extensive experiments on CIFAR10/CIFAR100 and ImageNet show that the robust pruning accuracy is significantly improved under the FSRP robust pruning criterion. On CIFAR10, a pruning ratio of 90% on the VGG16 network still shows a 14.1% improvement in robust accuracy under auto attacks (AA).