Dense Crosstalk Feature Aggregation for Classification and Localization in Object Detection

Abstract

The misalignment between classification and localization is a significant performance improvement point for object detection. To cope with the misalignment problem, more attempts have been made to separate different tasks (e.g., Classification, Bounding Box Regression) by introducing extra heads, which emphasizes the separation of multiple tasks to cope with their variability. In this paper, we consider that both separation and crosstalk are important between classification and localization. Considering that the two types of tasks are different and have different regions and features of interest, they are in conflict with each other and therefore need to be separated. However, they also need to be fused, because classification and localization are, after all, about understanding the same object. To realize this idea, we introduce bidirectional crosstalk detection head in a systematic manner to provide a full deep cross-fusion between classification and localization. To our best knowledge, it is the first time that full bidirectional crosstalk is introduced between classification and localization for one-stage detector. Extensive experiments are conducted to demonstrate the effectiveness of the proposed method. With a ResNet-50 backbone, our method can significantly improve the GFLV1 baseline by 2.0 AP with similar inference speed (18.5 fps vs. 18.3 fps) and further boost GFLV1 with a big margin (4.3 AP) by increasing our model size. Fair comparisons also show that the proposed head outperforms state-of-the-art heads (T-Head, DyHead) with comparable or faster inference speed under the same ATSS baseline model. With a Res2Net-DCN backbone, our model achieves 51.7 AP at single-model single-scale testing. The code and pretrained models will be made publicly available.