Considering representation diversity and prediction consistency for domain generalization semantic segmentation

Abstract

Listen

Translate article

Existing domain generalization semantic segmentation (DGSS) methods aim to learn domain-invariant representation from single or multiple domains by using the consistency constraint or normalization strategy. Although these methods can improve the generalization capability of the models, the representation capability may be inevitably weakened since some effective information is eliminated, which may lead to the low discrimination capability of models. To simultaneously preserve the generalization and discrimination capability of models, we provide a novel perspective that encourages to learn the distinct representation while keeping the consistent prediction. Based on this, we propose a novel approach for domain generalization semantic segmentation by simultaneously considering the representation diversity and prediction consistency. The proposed approach consists of an adaptive style randomization (ASR) module, a representation diversity (RD) constraint, and a prediction consistency (PC) constraint. Specifically, first, since diverse stylized images are beneficial for the RD constraint to learn the distinct representation, the ASR module is proposed to leverage global and local style randomization to generate diverse stylized images which have the same content as the source domain images. Then, the distinct representation is learned from the source domain images and the stylized images by using the RD constraint to improve the discrimination capability of the model. Finally, the distinct representation is expected to generate consistent prediction by using the PC constraint for preserving the generalization capability of the model. Extensive experiments demonstrate that our approach achieves superior performance over current approaches on several benchmarks.