Consistency regularization-based mutual alignment for source-free domain adaptation

Abstract

Unsupervised domain adaptation (UDA) is used to extend the model working on well-annotated source data to unlabeled target data. However, in practice, due to privacy and storage issues, we can only obtain the well-trained source model. In this paper, we focus on this scenario named source-free domain adaptation (SFDA). At present, nearest neighbors-based SFDA methods assume that the target features extracted by the source model can form clear clusters, and align samples with their neighbors. However, due to the domain discrepancy, adjacent features may belong to different categories. We propose consistency regularization-based mutual alignment (CRMA) to address this problem. Firstly, we randomly augment each target sample. Due to the domain discrepancy, it may lead to negative transfer if we align them directly. Therefore, secondly, we leverage the information maximization loss to all target and augmented samples, improving the performance of mutual alignment. Finally, we mutually align original samples and augmented samples. It improves the ability of the model and increases the variety of samples to alleviate the phenomenon that incorrectly aligning samples when aligning with neighbors. CRMA achieves state-of-the-art performance on 3 popular cross-domain benchmarks. Compared with the original method, CRMA has improvements of 0.4% up to 89.4%, 1.9% up to 72.2%, and 1.9% up to 85.9% on 3 datasets respectively. At the last, we verify the effectiveness of each part of CRMA through ablation experiments and use a series of experiments to analyze CRMA in detail.