Abstract
Adversarial training (AT) is a promising method to improve the robustness against adversarial attacks. However, its performance is not still satisfactory in practice compared with standard training. To reveal the cause of the difficulty of AT, we analyze the smoothness of the loss function in AT, which determines the training performance. We reveal that nonsmoothness is caused by the constraint of adversarial attacks and depends on the type of constraint. Specifically, the L∞ constraint can cause nonsmoothness more than the L2 constraint. In addition, we found an interesting property for AT: the flatter loss surface in the input space tends to have the less smooth adversarial loss surface in the parameter space. To confirm that the nonsmoothness causes the poor performance of AT, we theoretically and experimentally show that smooth adversarial loss by EntropySGD (EnSGD) improves the performance of AT.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
More From: IEEE transactions on neural networks and learning systems
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.