PETNet: A YOLO-based prior enhanced transformer network for aerial image detection

Abstract

Unmanned aerial vehicles (UAVs) have been applied to inspect in various scenarios due to their high efficiency, low cost, and excellent mobility. However, the objects in aerial images are much smaller and denser than general objects, causing it difficult for current object detection methods to achieve the expected results. To solve this issue, a prior enhanced Transformer network (PETNet) based on YOLO is proposed in this paper. Specifically, a novel prior enhanced Transformer (PET) module and a one-to-many feature fusion (OMFF) mechanism are proposed to embed into the network. Two additional detection heads are added to the shallow feature maps. In this work, PET is used to capture enhanced global information to improve the expressive ability of the network. The OMFF aims to fuse multi-type features to minimize the information loss of small objects. In addition, the added detection heads provide more possibility of detecting smaller-scale objects, and the extended multi-head parallel detection is more suitable for the multi-scale transformation of objects in aerial images. On the VisDrone-2021 and UAVDT databases, the proposed PETNet achieves state-of-the-art results with average precision (AP) of 35.3 and 21.5, respectively, which indicates that the proposed network is more suitable for aerial image detection and is of a great reference value.