A multi-grained unsupervised domain adaptation approach for semantic segmentation

Abstract

When transferring knowledge between different datasets, domain mismatch greatly hinders model’s performance. So domain adaption has been brought up to tackle the problem. Traditional methods focusing either on global or local alignment play a limited role in improving model’s performance. In this paper, we propose a multi-grained unsupervised domain adaptation approach (Muda) for semantic segmentation. Muda aims to enforce multi-grained semantic consistency between domains by aligning domains at both global and category level. Specifically, coarse-grained adaptation uses global adversarial learning on an image translation model and a main segmentation model, which respectively attempts to eliminate appearance differences and to get similar segmentation maps from two domains. While fine-grained adaptation employs an auxiliary model to adapt category information to refine pseudo labels of target data. Experiments and ablation studies are conducted on two synthetic-to-real benchmarks: GTA5 → Cityscapes and SYNTHIA → Cityscapes, which show that our model outperforms the state-of-the-art methods.