Uncertainty-weighted prototype active learning in domain adaptive semantic segmentation

Abstract

Generalizing segmentation networks that adapt to unseen target domains is practical to the real-world utility. Recent self-training-based approaches compute the pseudo labels of highly imbalanced target image that are typically biased to the majority classes and basically noisy. To avoid the model falling into an error-prone and suboptimal state, we propose an uncertainty-weighted prototype active learning method to deal with domain adaptation regarding the semantic segmentation task. For assisting active learning to select reliable samples, multiple prototypical anchors strategy is employed to build the target image distribution model and assign accurate pixel-level pseudo labels, which consists of two different stages. In the initial training stage, this model defines each class as a set of non-learnable prototypes, and performs online clustering to identify target instances for annotating in the prototype-based feature space. In the fine-tuning stage, entropy criterion is adopted to generate representative class prototypes set of target domain, then compute the uncertainty as the clustering weight of prototype-level features via a predictive entropy-based strategy. By mixing these latent representations of the target samples and developing feature alignment loss function, our method can achieve more accurate segmentation performance. Extensive experiments on two benchmarks, GTAV → Cityscapes and SYNTHIA → Cityscapes, demonstrate that the proposed model outperforms other state-of-the-art approaches, and ablation study verifies the effectiveness of each key component. We hope our work will open a new view for future research in the field of domain adaptation. The code will be released soon at https://github.com/zihaodong/UPAL.