Abstract
Convolutional neural networks (CNNs) have achieved tremendous success in solving complex classification problems. Motivated by this success, there have been proposed various compression methods for downsizing the CNNs to deploy them on resource-constrained embedded systems. However, a new type of vulnerability of compressed CNNs known as the adversarial examples has been discovered recently, which is critical for security-sensitive systems because the adversarial examples can cause malfunction of CNNs and can be crafted easily in many cases. In this paper, we proposed a compression framework to produce compressed CNNs robust against such adversarial examples. To achieve the goal, our framework uses both pruning and knowledge distillation with adversarial training. We formulate our framework as an optimization problem and provide a solution algorithm based on the proximal gradient method, which is more memory-efficient than the popular ADMM-based compression approaches. In experiments, we show that our framework can improve the trade-off between adversarial robustness and compression rate compared to the existing state-of-the-art adversarial pruning approach.
Highlights
In the past few years, convolutional neural networks (CNNs) have achieved great success in many applications including image classification and object detection
We focused on the original accuracy of the early stage of the optimization to show how well Adversarial Pruning with Distillation (APD) preserved the original accuracy of the baseline model during the adversarial pruning
The adversarial robustness of the compressed CNNs is essential for deploying them to the real-world embedded systems
Summary
In the past few years, convolutional neural networks (CNNs) have achieved great success in many applications including image classification and object detection. The excessively large amount of learning parameters and the vulnerability for the adversarial examples [1,2,3,4,5,6,7,8] are making it difficult to deploy CNNs especially on resource-constrained environments such as smartphones, automobiles, and wearable devices To overcome this drawback, various model compression methods have been proposed, where many are based on weight pruning [9,10,11,12,13,14,15,16,17]. We found that consistently providing information about the pretrained original network during adversarial training can improve the robustness of the resulting compressed network With this intuition, we propose a novel robust pruning framework that jointly uses pruning and knowledge distillation [21] within the adversarial training procedure. Our method showed a better trade-off between adversarial robustness and compression rate compared to the state-of-the-art methods [15,19,22]
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
