Dual-CNN fusion panoptic segmentation based on edge optimization

Abstract

As a newly proposed research topic in recent years, panoptic segmentation is the combination of semantic segmentation and instance segmentation. The difficulties of panoptic segmentation include not only the common problems in traditional segmentation tasks, such as small object segmentation and low edge segmentation accuracy, but also the two-way fusion and the determination of conflicts. Considering the defect of edge segmentation, we use the edge optimization module SegFix based on edge detection and direction prediction for edge optimization, which reaches higher accuracy and shortens the calculation time by 3 times compared with Dense CRF. Based on the dual CNN fusion, we select EfficientNet as the baseline, which is more efficient, with depthwise separable convolutions and Mask R-CNN to achieve two-way segmentation. In addition, we use SegFix in multiple panoptic segmentation models to verify its versatility in panoptic segmentation. Finally, our PQ on the Cityscapes validation set reaches 65.5, which achieves the state-of-the-art result with all other panoptic segmentation models under the same experimental conditions, and we confirm that the edge optimization algorithm we use is universal for panoptic segmentation, and its consequence is better than other edge optimization algorithms.