Smoothness Enforcing Projected Gradient Descent Adversarial Training

Abstract

Abstract The paper proposes a novel adversarial training method based on a smoothness enforcing framework called Smoothness Enforcing Adversarial Training (SEAT). It is known that the traditional adversarial training methods suffer from prohibitive computational overhead and mismatched class distributions. To this end, our method is combined with smoothness enforcing methods in adversarial training iterations to achieve better robustness and generalization. SEAT reuses gradient information computed when updating model parameters. Meanwhile, random initialization and output smearing are integrated into the process to optimize the mismatched problems with better generalization performance. Furthermore, the temporal ensembling serves as an implicit self-ensemble of the adversarial information which benefits from a longer memory. In different iterations, the smoothness constraints are imposed to enforce smoothness. Unlike existing adversarial training methods, our method is free from arbitrarily complicated distributions and expensive generation of adversarial examples. Extensive experiments validate the effectiveness of SEAT in comparison with state-of-the-art adversarial training methods.