Learning a target-dependent classifier for cross-domain semantic segmentation: Fine-tuning versus meta-learning

Abstract

Recently proposed domain adaptation arts have dominated the field of cross-domain semantic segmentation by operating domain manifolds alignment and learning an optimal joint hypothesis (joint-domain classifier) for both source and target domains. However, a joint-domain classifier can still violate the cluster assumption in the target domain in case domain manifolds are not fully aligned after domain adaptation. In this work, we raise the intractability of perfect domain alignment and turn to exploit a novel hypothesis: a target-dependent classifier, to efficiently adapt to the target domain clusters even given a certain degree of domain misalignment. Specifically, we first propose an unsupervised fine-tuning strategy, which optimizes the joint hypothesis of vanilla domain adaptation into a target-dependent hypothesis to better fit with the target domain clusters. Second, we connect the “learning to learn” concept of meta-learning with pixel-wise domain adaptation, which serves as a reliable hypothesis initialization, providing an alternative solution to learning a more generalized target-dependent classifier. The proposed learning method is general to conventional domain adaptation models. In experiments, we recycle the pre-trained conventional DA models and learn target-dependent classifiers with the proposed method. Experimental results on synthetic-to-real adaptation and cross-city adaptation benchmarks demonstrate that the target-dependent classifier leads over state-of-the-art performance.