Rethinking maximum-margin softmax for adversarial robustness

Abstract

Learning discriminative features with adversarial behaviors can be extremely challenging to build a robust learning model. This is partly evidenced by the difficulties in training robust maximum-margin models (e.g., ArcFace and CosFace) that cannot discriminate decision boundaries between perturbed samples perfectly. One potential approach is to design a loss function that achieves robust generalization by learning high-density features in the latent space to discriminate between adversarial and legitimate samples effectively. Therefore, we propose an Ensemble Maximum-Margin Softmax (EMMS) method to construct a robust generalization that yields reliable models. Specifically, EMMS is designed to address the limitation in maximum-margin methods and induce high-density discriminative features for clean and adversarial settings. The empirical experiments using the CIFAR and SVHN datasets show that EMMS is more robust in terms of accuracy and error rates than other peer techniques. The outcomes suggest that EMMS could improve the robustness of the model compared with ArcFace and CosFace under various types of attacks.