Learning panoptic segmentation through feature discriminability

Abstract

Panoptic segmentation has attracted increasing attention as a joint task of semantic and instance segmentation. However, previous works have not noticed that the different requirements for semantic and instance segmentation can lead to conflict of feature discriminability. Instance segmentation mainly focuses on the central area of each instance in things regions, while semantic segmentation focuses on the whole region of a specific class. To resolve it, we propose: 1) a Dual-FPN framework which separates the shared Feature Pyramid Network (FPN) in previous works to reduce the conflict of receptive field and meet different requirements of the two tasks; 2) a Region Refinement Module which leverages the prediction of semantic segmentation to refine the result of instance segmentation and resolves the conflict between the things regions and the stuff regions. Experimental results on Cityscapes dataset and Mapillary Vistas dataset show that our proposed method can improve the result of both things and stuff and obtain state-of-the-art performance.