A polar-edge context-aware (PECA) network for mirror segmentation

Abstract

This paper presents Polar-Edge Contrast-Aware Network (PECA) for mirror instance segmentation in images of indoor scenes. General instance segmentation methods typically rely on the surface appearance of the object to identify the foreground from background. However, these approaches do not directly apply to mirrors as the mirror surfaces are less reliable due to reflections of the surroundings. On the other hand, the existing saliency-based mirror segmentation methods are prone to predict false positives in images with no mirrors, especially for the indoor scenes that have mirror-shape objects, such as doors and windows. In this work, we propose a novel boundary-based mirror localization method PECA that achieves both high segmentation accuracy on mirrors and low false positive rate on negative samples. PECA uses a context-aware module to extract features along the instance contour and in this way incorporates boundary information for improving mirror detection. The predicted mirror candidates are further refined with a local contrast module for the final mirror instance segmentation. PECA achieves IoU 80.29% on the benchmark Mirror Segmentation dataset (MSD), outperforming the state-of-the-art method MirrorNet (IoU = 78.95%) by 1.34%. It also produces a significantly smaller false positive rate (43.37%) than existing methods (91.39%) on our challenging Negative Mirror Dataset (NMD) without retraining. After training on both MSD and NMD training sets, our model further reduces the false positive rate to 0.08% on NMD testing set, while keeping IoU of 73.07% on MSD, enabling realistic real-world applications.