R4 Det: Refined single-stage detector with feature recursion and refinement for rotating object detection in aerial images

Abstract

The detection of objects with multi-orientations and multi-scales in aerial images is receiving increasing attention because of numerous useful applications in computer vision, image understanding, satellite remote sensing and surveillance. However, such detection can be exceedingly challenging because of a birds eye view, multi-scale rotating objects with large aspect ratios, dense distributions and extremely imbalanced categories. Despite the considerable progress that has been made, detection performance falls considerably below that required for real-world applications. In this paper, we propose an accurate and fast end-to-end detector to address the aforementioned challenges. Our contributions are threefold. First, inspired by the looking and thinking twice mechanism, recursive neural networks and the DetectoRS detector, we propose a novel encoder-decoder based architecture by introducing the recursive feature pyramid into a single-stage object detection framework. The improved backbone network can generate increasingly powerful multi-scale representations for classification and regression. Second, we propose a refined single-stage detector with feature recursion and refinement for rotating objects. Third, we use instance balance to improve focal loss, thereby optimizing the loss in the correct direction. Extensive experiments on two challenging aerial image object detection public datasets, DOTA and HRSC2016, show that the proposed R4Det detector achieves the state-of-the-art accuracy while running very fast. Moreover, further experiments show that our detector is more robust to adversarial image patch attacks than the previous state-of-art detector.