Syncretic-NMS: A Merging Non-Maximum Suppression Algorithm for Instance Segmentation

Jun Chu,Lu Leng,Shaoming Li,Yiqing Zhang,Jun Miao

doi:10.1109/access.2020.3003917

Abstract

Instance segmentation is typically based on an object detection framework. Semantic segmentation is conducted on the bounding boxes that are returned by detectors. NMS (non-maximum suppression) is a common post-processing operation in instance segmentation and object detection tasks. It is typically used after bounding box regression to eliminate redundant bounding boxes. The evaluation criteria for object detection require that the bounding box be as close as possible to the ground truth, but they do not emphasize the integrity of the included object. However, sometimes the bounding boxes cannot contain the complete objects, and the parts beyond the bounding boxes cannot be correctly predicted in the subsequent semantic segmentation. To solve this problem, we propose the Syncretic-NMS algorithm. The algorithm takes traditional NMS as the first step and processes the bounding boxes obtained by traditional NMS, judges the neighboring bounding boxes of each bounding box, and combines the neighboring boxes that are strongly correlated with the corresponding bounding boxes. The coordinates of the merged box are the four coordinate extremes of the bounding box and the highly relevant neighboring box. The neighboring box with strong correlation is merged with the corresponding bounding box. Based on an analysis of the influences of corresponding factors, the criteria for correlation judgment are specified. Experimental results on the MS COCO dataset demonstrate that Syncretic-NMS can steadily increase the accuracy of instance segmentation, while experimental results on the Cityscapes dataset prove that the algorithm can adapt to application scenario changes. The computational complexity of Syncretic-NMS is the same as that of traditional NMS. Syncretic-NMS is easy to implement, requires no additional training, and can be easily integrated into the available instance segmentation framework.

Highlights

Instance segmentation is a multi-mission learning task that consists of object detection and semantic segmentation
Based on the above analysis, we propose Syncretic-NMS, which is a merging non-maximum suppression algorithm for instance segmentation that is based on traditional NMS
SYNCRETIC-NMS PIPELINE Similar to the traditional NMS algorithm, the Syncretic-NMS algorithm that is proposed in this paper accepts the bounding box list B and classification score S that are returned by the classifier as input, obtains the bounding box list D after one round of NMS, and conducts neighboring box correlation judgment and merge operations

Summary

INTRODUCTION

Instance segmentation is a multi-mission learning task that consists of object detection and semantic segmentation. Due to the conciseness of the judgment conditions, traditional NMS has extremely high efficiency, but some high-confidence bounding boxes may be filtered out by mistake, thereby resulting in the obtained bounding boxes not including complete objects Aiming at overcoming this problem, this paper proposes a new NMS algorithm that merges neighboring boxes: Syncretic-NMS. SYNCRETIC-NMS PIPELINE Similar to the traditional NMS algorithm, the Syncretic-NMS algorithm that is proposed in this paper accepts the bounding box list B and classification score S that are returned by the classifier as input, obtains the bounding box list D after one round of NMS, and conducts neighboring box correlation judgment and merge operations. Si is the original classification score of the bounding box, iou bi, bj is the IoU of the bounding box bi and the neighboring box bj, and Nc is the association judgment threshold

THRESHOLD SELECTION

INSTANCE SEGMENTATION ON THE MS COCO DATASET

CONCLUSIONS