Panoptic Segmentation with Convex Object Representation

Abstract

Abstract The accurate representation of objects holds pivotal significance in the realm of panoptic segmentation. Presently, prevalent object representation methodologies, including box-based, keypoint-based and query-based techniques, encounter a challenge known as the ‘representation confusion’ issue in specific scenarios, often resulting in the mislabeling of instances. In response, this paper introduces Convex Object Representation (COR), a straightforward yet highly effective approach to address this problem. COR leverages a CNN-based Euclidean Distance Transform to convert the target instance into a convex heatmap. Simultaneously, it offers a parallel embedding method for encoding the object. Subsequently, COR characterizes objects based on the distinctive embedding vectors of their convex vertices. This paper seamlessly integrates COR into a state-of-the-art query-based panoptic segmentation framework. Experimental findings validate that COR successfully mitigates the representation confusion predicament, enhancing segmentation accuracy. The COR-augmented methods exhibit notable improvements of +1.3 and +0.7 points in PQ on the Cityscapes validation and MS COCO panoptic 2017 validation datasets, respectively.