Simplified unsupervised image translation for semantic segmentation adaptation

Abstract

Image to image translation achieves superior performance with the advent of generative adversarial networks. In this paper, we propose a Simplified Unsupervised Image Translation (SUIT) model for domain adaptation on semantic segmentation. We adopt adversarial training for superior image generation, and design a novel semantic-content loss to enhance visual appearance preservation. Thus, the high-fidelity generated images with target-style can help the model generalize to the target domain. Besides, the semantic-content loss contains two components, which focus on label- and content-consistency, respectively. Both of them can be derived from existing modules of SUIT, which makes it simple yet suitable for domain adaptation on semantic segmentation tasks. Meanwhile, since the transformation network (generator) is decoupled from the segmentation network, the former can be easily transplanted to other semantic segmentation models. Extensive experimental results demonstrate that these translated images within SUIT can significantly improve performance of the model on the target domain, and our model with FCN8s-VGG16 architecture achieves around 13 percentage points improvement in terms of mIoU on multiple semantic segmentation adaptation benchmarks.