Research on Effectiveness of Contrastive Learning with Adversarial Examples for Different Downstream Tasks

Abstract

Contrastive learning is a new self-supervised representation learning technique, which is considered to have great potential to improve the performance of downstream learning tasks. Recently, some researchers have conducted experiments on some datasets and found that contrastive learning based on adversarial examples can improve the performance of some image classification tasks. However, does contrastive learning with adversarial examples really benefit the performance of various downstream tasks? At present, there is still a lack of in-depth research on this issue. In this paper, we introduce a contrastive learning method with adversarial examples based on different adversarial perturbations intensities, and establish a variety of downstream image classification tasks. Experiments on different types of downstream tasks show that contrastive learning based on adversarial examples may not improve the performance of downstream tasks. We also find that the adversarial perturbation intensity and the proportion of the adversarial perturbation loss in contrastive loss are important factors affecting the above performance.