Learning From Synthetic Data for Crowd Instance Segmentation in the Wild

Abstract

Crowd understanding has widespread applications, including video surveillance, crowd monitoring. Unlike existing coarse-grained crowd understanding methods(e.g., counting people in images), crowd instance segmentation can provide more precise results (pixel-wise segmentation for each person in images). However, crowd instance segmentation demands a considerable amount of pixel-wise labeled data, which is very time-consuming and challenging to annotate accurate human instance masks in the crowd scene. In this paper, we propose a data generator and labeler to automatically generate synthetic crowd instance segmentation data. Then based on it, we build a large-scale synthetic crowd instance segmentation dataset called "GCIS Dataset". Besides, we demonstrate two approaches that utilize the synthetic GCIS dataset to advance the performance of crowd instance segmentation: 1)supervised crowd instance segmentation: pretrain crowd instance segmentation models on GCIS dataset, then finetune on other real data. It can remarkably boost the model’s real-world performance; 2) crowd instance segmentation via domain adaption: transfer the synthetic GCIS dataset to photo-realistic images, then train the model together with transformed data and real data, which shows better performance when tested on real-world data. Extensive experiments show the validity of the synthetic GCIS dataset for crowd instance segmentation. The dataset and source code will be released online.