Optimized L2 Norm Loss for Adversarial Robustness

Abstract

Although adversarial training is the most common method to make models obtain better adversarial robustness, its drawback of leading to reduced accuracy has been plaguing the academic community. In recent years, many articles have pointed out that good Lipschitz continuity helps models obtain better robustness and standard accuracy, and argued that models that are both robust and accurate exist. However, many methods still perform less well with models even with the addition of Lipschitz continuity constraints. Therefore, we discuss the drawbacks of existing Lipschitz continuity metric in deep learning in terms of Lipschitz continuity, and propose a counteracting Lipschitz continuity metric that is more suitable for deep learning. We demonstrate theoretically and experimentally that Mixup can significantly enhance the local Lipschitz continuity of the model. Using this property, we generate a large number of mix confrontation samples using Target attack to fill the entire neighborhood space. Our method gives the model a smoother localization and significantly improves the adversarial robustness of the model beyond most existing adversarial training methods.