Rethinking mask heads for partially supervised instance segmentation

Abstract

We focus on partially supervised instance segmentation where only a subset of categories are mask-annotated (seen) and the model is expected to generalize to unseen categories for which only box annotations are provided to eliminate laborious mask annotations. Many recent studies train a class-agnostic segmentation network to distinguish foreground areas in each proposal. However, class-agnostic models behave poorly in complex contexts when the foreground object overlaps with other irreverent objects. Identifying specific object categories is simpler than distinguishing foreground from background since the definition of the foreground is ambiguous even for a human. However, training class-specific model is unfeasible under the partially supervised setting since the mask annotations of unseen categories are absent during training. To overcome this issue, we put forward a teacher-student architecture where the teacher learns general yet comprehensive knowledge and the students, guided by the teacher, delve deeper into specific categories. Concretely, the teacher learns to segment foreground from proposals and the student is devoted to segmenting objects of specific categories. Extensive experiments on the challenging COCO dataset demonstrate our method consistently improve the performance of several recent state-of-the-art methods for the partially setting. Especially, for overlapped objects, our method significantly outperforms the competitors with a clear margin, demonstrating the superiority of our method.