DG-FPN: Learning Dynamic Feature Fusion Based on Graph Convolution Network For Object Detection

Abstract

Feature Pyramid Network (FPN) is one of the most popular feature fusion methods to address the multi-scale issue in object detection. Current FPN-based methods are mostly designed manually, which cannot guarantee the optimal feature fusion. Besides, the predetermined methods generally provide the same strategy to various targets, which are not distinctive among targets with different scales. In this paper, we present a novel dynamic feature fusion method based on the graph convolution network (GCN), called DG-FPN. The proposed GCN-based method can dynamically transfer knowledge with learnable weights across all nodes, making it possible to learn the optimal feature fusion for detectors. Furthermore, the pixel-based adjacency matrix is proposed to offer customized fusion strategy for each target, achieving dynamic feature fusion. To optimize matrix-driven learning, semantic information is introduced to guide the process of fusion. Experiments show that DG-FPN significantly improves the performance of baseline networks on the challenging MS-COCO object benchmark, especially in small objects.