Synthetic-to-Real Domain Adaptation for Object Instance Segmentation

Abstract

Object instance segmentation can achieve preferable results, powered with sufficient labeled training data. However, it is time-consuming for manually labeling, leading to the lack of large-scale diversified datasets with accurate instance segmentation annotations. Exploiting the synthetic data is a very promising solution except for domain distribution mismatch between synthetic dataset and real dataset. In this paper, we propose a synthetic-to-real domain adaptation method for object instance segmentation. At first, this approach is trained to generate object detection and segmentation using annotated data from synthetic dataset. Then, a feature adaptation module (FAM) is applied to reduce data distribution mismatch between synthetic dataset and real dataset. The FAM performs domain adaptation from three different aspects: global-level base feature adaptation module, local-level instance feature adaptation module, and subtle-level mask feature adaptation module. It is implemented based on novel discriminator networks with adversarial learning. The three modules of FAM have positive effects on improving the performance when adapting from synthetic to real scenes. We evaluate the proposed approach on Cityscapes dataset by adapting from Virtual KITTI and SYNTHIA datasets. The results show that it achieves a significantly better performance over the state-of-the-art methods.