RoMP-transformer: Rotational bounding box with multi-level feature pyramid transformer for object detection

Abstract

This study proposes rotational bounding box with a multi-level feature pyramid transformer (RoMP-Transformer)—a fast and accurate one-stage deep neural network for object detection. The proposed RoMP-Transformer exhibits three characteristics. First, a rotational bounding box is utilized to minimize the effect of the background during the construction of feature maps, enhancing the robustness of the RoMP-Transformer. Second, the RoMP-Transformer employs a multi-level feature pyramid transformer by combining a multi-level feature pyramid network with a pyramid vision-transformer, effectively extracting high-quality features and achieving high accuracy. Third, the RoMP-Transformer executes bounding box optimization by minimizing the optimal intersection of union (IoU) loss by considering both the modified SKEW IoU and distance IoU. The modified SKEW IoU significantly accelerates the calculation, and the fused IoU calculation method improves prediction accuracy. Further, Bayesian optimization and weight lightening with half-tensor are performed to optimize the performance of the RoMP-Transformer for real-time applications. Experiments on three image sets—one on power transmission facilities, MSRA-TD500, and DOTA-v1.0—demonstrate that the proposed RoMP-Transformer outperforms other state-of-the-art neural networks in object detection in terms of accuracy, robustness, and calculation speed. Systematic analysis also reveals that the methods utilized by the RoMP-Transformer optimize object detection performance. The proposed architecture is expected to inspire further study of deep neural networks for object detection in real-world applications.