Area in circle: A novel evaluation metric for object detection

Abstract

As computer vision technologies undergo extensive advancement in recent years, the significance of object detection has been valued to an unprecedented level in a variety of industries, such as manufacturing, healthcare, agriculture, and smart transportation. Even though Average Precision (AP) is widely employed as the evaluation metric for object detection, it typically suffers from some issues, e.g., the difficulties in explanation for non-professionals, the lack of clear guidance in determining thresholds, and the incomplete consideration in localization and classification accuracy. In this paper, we propose a novel evaluation metric for object detection, called Area in Circle (AIC). To be specific, we introduce the novel concepts of Weighted True Positive (WTP) and Weighted False Positive (WFP), which project the original Precision and Recall results onto a circular coordinate. After that, we calculate the corresponding area in the circular coordinate in terms of perfect and weak detectors, without determining any Intersection over Union (IoU) threshold. We conducted the experiments on the COCO 2017 dataset with 22 state-of-the-art object detection models. The results showed that our proposed metric can clearly and accurately interpret a model’s overall detection capacity. Furthermore, it can provide extra knowledge about the detection quality, and can be served as an effective alternative for the existing metrics.