Non-local Patch Mixup for Unsupervised Domain Adaptation

Abstract

Unsupervised Domain Adaptation (UDA) aims to ease the discrepancy between two domains, thereby transferring knowledge learned from a labeled source domain to a completely unlabeled target domain. Mainstream UDA approaches mainly focus on learning domain-invariant representations in the latent space, while neglecting the discrepancy brought about by rich domain-specific characteristics in the input space, resulting in insufficiently aligned or even misaligned representations. To address this issue, in this paper, we propose to build a unified input space for both the source and target domains, on which the source domain knowledge can be easily transferred to the target domain. Towards this goal, we propose a Non-local Cross-domain Patch Mixup (NCPM) approach that can adapt to Vision Transformer (ViT) architectures to generate intermediate examples to support the unified space. Through NCPM, we further design a three-stage training strategy to gradually increase the searching range of the source domain until it covers the target domain, progressively generating intermediate examples from source-like to target-like to bridge the domain gap. Moreover, to suppress the negative impact of noisy labels on the intermediate examples, we propose a high-order scoring aggregation method that explicitly leverages the intrinsic structure of the target data to refine its output score. Finally, we conduct comprehensive experiments on four benchmark datasets to verify the superiority of our proposed approach. Our code is available at https://github.com/BetterTMrR/NCPM-pytorch-implementation.