MFNet: Panoptic segmentation network based on multiscale feature weighted fusion and frequency domain attention mechanism

Abstract

Existing panoptic segmentation networks based on multiscale methods do not distinguish shared features according to different scales, which leads to a loss of feature information and suboptimal results. To solve this problem, the authors improve on UPSNet (a unified panoptic segmentation network) by introducing a multiscale fusion module cascaded between layers into the semantic segmentation head of UPSNet. On the one hand, this enriches the feature information for differently scaled features by cascading the feature information of adjacent scales, and on the other hand, the feature information at each scale can be combined adaptively to achieve a better segmentation effect. Furthermore, a frequency domain attention mechanism is introduced into the backbone network of UPSNet to improve the feature extraction ability of the backbone network by learning more frequency domain feature information. The experimental results of the authors’ improved network on the Common Objects in Context (COCO) and Cityscapes data sets show that compared with UPSNet, the performance is significantly improved: the panoptic quality is improved by 0.9% and 1.6%, respectively.