Abstract
With the widespread adoption of edge computing, compressing deep neural networks (DNNs) via knowledge distillation (KD) has emerged as a popular technique for resource-limited scenarios. Among various KD methods, feature-based KD, which leverages the feature representations from intermediate layers of the teacher model to supervise the training of the student model, has shown superior performance and enjoyed wide application. However, users often overlook potential backdoor threats when using knowledge distillation (KD) to extract knowledge. To address the issue, this paper mainly contributes to three aspects: (1) we try the first step of exploring the security risks in feature-based KD, where implanted backdoors in teacher models can survive and transfer to student models. (2) We propose a backdoor attack method targeting feature distillation, achieved by encoding backdoor knowledge into specific neuron activation layers. Specifically, we optimize triggers to induce consistent feature map values in the teacher model and transfer the backdoor knowledge to student models through these features. We also design an adaptive defense method against this attack. (3) Extensive experiments on four common datasets and six sets of different teacher and student models validate that our attack outperforms the state-of-the-art (SOTA) baselines, with an average attack success rate of (∼×1.5). Additionally, we discuss effective defense methods against such backdoor attacks.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.