Unsupervised Domain Adaptation for Semantic Segmentation of Urban Street Scenes Reflected by Convex Mirrors

Abstract

Reflective convex mirrors are often used on street corners or as passenger-side mirrors on cars to obtain scene information by reflecting blind spots in the field of view, which can provide safety for pedestrians and drivers on roads, driveways, and alleys that lack of visibility. In recent years, deep learning based scene understanding methods (e.g., semantic segmentation) have been rapidly developed. However, due to gaps in the geometric domain, models trained on normal images are not directly applicable to scenes with convex mirror reflections. In this paper, we propose a novel framework to reduce the domain gap between normal images and convex mirror reflection images. In particular, we geometrically model convex mirrors to obtain a differentiable convex mirror simulation layer, CMSL. With the help of CMSL, we perform adversarial domain adaptation on edges in the input space and semantic boundaries in the output space to reduce the geometric appearance gap between the synthetic and real images. To verify the effectiveness of our algorithm, we construct the first convex mirror reflection scene dataset CMR1K, which contains 268 images with fine annotations. Extensive experimental results show that our algorithm can significantly outperform the baseline and previous methods. For example, our method surpasses the baseline and AdvEnt by 10% and 3% in mIoU, respectively.