CSF: Closed-mask-guided semantic fusion method for semantic perception of unknown scenes

Abstract

• A mask-guided semantic fusion method for optimizing the results of semantic segmentation. • A closed mask generation (CEG) module for post-processing of the edge detection algorithm. • A semantic confidence fusion (SCF) module to fuse the semantic segmentation results. • This method does not need to obtain the target data in advance. • This method has been verified in two scenarios. Though many high-precision semantic segmentation models have been proposed, how to improve the generalization ability of these models is still an urgent problem. Recently, a great number of unsupervised domain adaptation (UDA) algorithms around domain adaptation problems have been studied in semantic segmentation. These methods require labeled source domain data and unlabeled target domain data. In this paper, we propose a closed-mask-guided semantic fusion method (CSF) to improve the semantic segmentation results of unknown scenes, where the target domain data is not obtained in advance. First, a Closed Mask Generation (CMG) module is designed to convert the edge detection result into a mask that can segment the image into several image blocks. Then, a Semantic Confidence Fusion (SCF) module based on information entropy and voting method is introduced, which can select reliable semantic segmentation results for each image block by comparing the confidence of several semantic segmentation networks . In addition, the experimental results on both KITTI and COCO Stuff datasets validate the effectiveness of this method. The code is publicly available at https://github.com/tryhere/CSF .