Reasonable object detection guided by knowledge of global context and category relationship

Abstract

The mainstream object detectors usually treat each region separately, which overlooks the important global context information and the associations between object categories. Existing methods model global context via attention mechanism, which requires ad hoc design and prior knowledge. Some works combine CNN features with label dependencies learned from a pre-defined graph and word embeddings, which ignore the gap between visual features and textual corpus and are usually task-specific (depend on RoIPool/RoIAlign). In order to get rid of the previous specific settings, and enable different types of detectors to refine detection results with the help of prior knowledge, in this paper, we propose KROD (Knowledge-guided Reasonable Object Detection), which consists of the GKM (Global Category Knowledge Mining) module and CRM (Category Relationship Knowledge Mining) module, to improve detection performance by mimicking the processes of human reasoning. For a given image, GKM introduces global category knowledge into the detector by simply attaching a multi-label image classification branch to the backbone. Meanwhile, CRM input the raw detection outputs to the object category co-occurrence based knowledge graph to further refine the original results, with the help of GCN (Graph Convolutional Network). We also propose a novel loss-aware module to distinctively correct the classification probability of different detected boxes. Without bells and whistles, extensive experiments show that the proposed KROD can improve different baseline models (both anchor-based and anchor-free) by a large margin (1.2% ∼ 1.8% higher AP) with marginal loss of efficiency on MS COCO.