Domain adaptation with contrastive and adversarial oriented transferable semantic augmentation

Abstract

Recent studies indicate that explicit domain adaptation methods that directly align the source and target domains will distort the original target feature distributions and cause misalignment. Implicit domain adaptation using feature augmentation bridges the domain gap and mitigates the above problems. However, such methods have the following challenges: (1) insufficient intrinsic discrimination of augmented features may cause negative transfer; (2) most methods ignore learning the domain-invariant features between source and augmented samples. To address these issues, we propose a Contrastive and Adversarial oriented transferable semantic Augmentation Domain Adaptation method (CAADA). We embed the augmentation process into our contrastive loss implicitly, minimizing it to increase the similarity between each augmented sample and the source centroid, thereby improving the augmented features’ intrinsic discriminativeness. Moreover, the prediction distribution difference between the source and augmented samples is used as the loss, and adversarial learning is employed between the classifier and feature extractor to refine domain-invariant features. The proposed losses can be integrated into other domain adaptation methods as lightweight modules to enhance each other’s performance. Experiments indicate that our proposed model yields state-of-the-art results on benchmarks, particularly resulting in 0.7% and 0.8% improvements on Office-Home and ImageCLEF-DA, respectively.