Rethinking the validity of perturbation in single-step adversarial training

Abstract

The neural network model has the drawback of making incorrect predictions under the influence of slight adversarial perturbations. Single-step adversarial training (AT) is an effective tool to bring adversarial robustness for the model to resist such attack. From the perspective of perturbation setting in training, we identify a conflict between the pursuit of greater robustness and the need to prevent catastrophic overfitting within the AT framework. To get out of this dilemma, we delve into the impact of perturbations on human visual perception. Our analysis reveals that examples containing more misleading features should be assigned a smaller perturbation magnitude to preserve subtle yet significant features. Conversely, examples encompassing more relevant features should be assigned a larger perturbation magnitude, enabling the model to adapt to stronger attacks effectively. Motivated by these insights, we propose a concise refinement to the AT framework to unleash its full potential for single-step AT. Instead of employing a fixed perturbation magnitude, we introduce a “band” of magnitudes, allowing each example to select an appropriate magnitude based on its visual characteristic. Through extensive experiments conducted on three datasets, we demonstrate the efficacy of our proposed strategy. Our approach not only improves the model’s robustness and prevents catastrophic overfitting but also effectively mitigates robust overfitting—an issue that has remained unresolved in the context of single-step AT, marking a significant advancement in the field.