Object Detection In Aerial Images Research Articles

Position information encoding FPN for small object detection in aerial images

Position information encoding FPN for small object detection in aerial images

Enhancing Small Object Detection in Aerial Images: A Novel Approach with PCSG Model

Generalized target detection algorithms perform well for large- and medium-sized targets but struggle with small ones. However, with the growing importance of aerial images in urban transportation and environmental monitoring, detecting small targets in such imagery has been a promising research hotspot. The challenge in small object detection lies in the limited pixel proportion and the complexity of feature extraction. Moreover, current mainstream detection algorithms tend to be overly complex, leading to structural redundancy for small objects. To cope with these challenges, this paper recommends the PCSG model based on yolov5, which optimizes both the detection head and backbone networks. (1) An enhanced detection header is introduced, featuring a new structure that enhances the feature pyramid network and the path aggregation network. This enhancement bolsters the model’s shallow feature reuse capability and introduces a dedicated detection layer for smaller objects. Additionally, redundant structures in the network are pruned, and the lightweight and versatile upsampling operator CARAFE is used to optimize the upsampling algorithm. (2) The paper proposes the module named SPD-Conv to replace the strided convolution operation and pooling structures in yolov5, thereby enhancing the backbone’s feature extraction capability. Furthermore, Ghost convolution is utilized to optimize the parameter count, ensuring that the backbone meets the real-time needs of aerial image detection. The experimental results from the RSOD dataset show that the PCSG model exhibits superior detection performance. The value of mAP increases from 97.1% to 97.8%, while the number of model parameters decreases by 22.3%, from 1,761,871 to 1,368,823. These findings unequivocally highlight the effectiveness of this approach.

Recent Advances for Aerial Object Detection: A Survey

Aerial object detection, as object detection in aerial images captured from an overhead perspective, has been widely applied in urban management, industrial inspection, and other aspects. However, the performance of existing aerial object detection algorithms is hindered by variations in object scales and orientations attributed to the aerial perspective. This survey presents a comprehensive review of recent advances in aerial object detection. We start with some basic concepts of aerial object detection and then summarize the five imbalance problems of aerial object detection, including scale imbalance, spatial imbalance, objective imbalance, semantic imbalance, and class imbalance. Moreover, we classify and analyze relevant methods and especially introduce the applications of aerial object detection in practical scenarios. Finally, the performance evaluation is presented on two popular aerial object detection datasets VisDrone-DET and DOTA, and we discuss several future directions that could facilitate the development of aerial object detection.

AFDet: alignment and focusing for aerial object detection

ABSTRACT Object detection in aerial images has become a focus in recent years due to the expansion of its application fields. Aerial objects have the characteristics of multi-scale, arbitrary angle and dense arrangement, which brings considerable challenges to the task. Based on the efficient one-stage detectors, most methods design modules to adaptively align the feature receptive field with the anchors to improve the accuracy of the bounding box regression and calculate loss according to parameter bias. Current methods are not adequately data-driven in feature optimization and the angle parameter regression faces boundary problems. To handle these problems, in this letter, we propose an Align-Focus detector (AFDet) for aerial image object detection. We introduce the ideology of deformable multi-head self-attention to feature optimization and design a new Align-Focus Module (AFM) which can sensitively focus feature encoding response to the real texture area of the objects. In addition, we apply KF-IOU Loss to solve the boundary problem. We conduct necessary experiments on DOTA and HRSC2016 datasets, and AFDet achieves the highest mAP performance compared to the existing methods.

FRED: Towards a Full Rotation-Equivariance in Aerial Image Object Detection

Rotation-equivariance is an essential yet challenging property in oriented object detection. While general object detectors naturally leverage robustness to spatial shifts due to the translation-equivariance of the conventional CNNs, achieving rotation-equivariance remains an elusive goal. Current detectors deploy various alignment techniques to derive rotation-invariant features, but still rely on high capacity models and heavy data augmentation with all possible rotations. In this paper, we introduce a Fully Rotation-Equivariant Oriented Object Detector (FRED), whose entire process from the image to the bounding box prediction is strictly equivariant. Specifically, we decouple the invariant task (object classification) and the equivariant task (object localization) to achieve end-to-end equivariance. We represent the bounding box as a set of rotation-equivariant vectors to implement rotation-equivariant localization. Moreover, we utilized these rotation-equivariant vectors as offsets in the deformable convolution, thereby enhancing the existing advantages of spatial adaptation. Leveraging full rotation-equivariance, our FRED demonstrates higher robustness to image-level rotation compared to existing methods. Furthermore, we show that FRED is one step closer to non-axis aligned learning through our experiments. Compared to state-of-the-art methods, our proposed method delivers comparable performance on DOTA-v1.0 and outperforms by 1.5 mAP on DOTA-v1.5, all while significantly reducing the model parameters to 16%.

DCEF2-YOLO: Aerial Detection YOLO with Deformable Convolution–Efficient Feature Fusion for Small Target Detection

Deep learning technology for real-time small object detection in aerial images can be used in various industrial environments such as real-time traffic surveillance and military reconnaissance. However, detecting small objects with few pixels and low resolution remains a challenging problem that requires performance improvement. To improve the performance of small object detection, we propose DCEF 2-YOLO. Our proposed method enables efficient real-time small object detection by using a deformable convolution (DFConv) module and an efficient feature fusion structure to maximize the use of the internal feature information of objects. DFConv preserves small object information by preventing the mixing of object information with the background. The optimized feature fusion structure produces high-quality feature maps for efficient real-time small object detection while maximizing the use of limited information. Additionally, modifying the input data processing stage and reducing the detection layer to suit small object detection also contributes to performance improvement. When compared to the performance of the latest YOLO-based models (such as DCN-YOLO and YOLOv7), DCEF 2-YOLO outperforms them, with a mAP of +6.1% on the DOTA-v1.0 test set, +0.3% on the NWPU VHR-10 test set, and +1.5% on the VEDAI512 test set. Furthermore, it has a fast processing speed of 120.48 FPS with an RTX3090 for 512 × 512 images, making it suitable for real-time small object detection tasks.

MS-YOLO: integration-based multi-subnets neural network for object detection in aerial images

MS-YOLO: integration-based multi-subnets neural network for object detection in aerial images

Enhance or Leave It: An Investigation of the Image Enhancement in Small Object Detection in Aerial Images

Recent years of object detection (OD), a fundamental task in computer vision, have witnessed the rise of numerous practical applications of this sub-field such as face detection, self-driving, security, and more. Although existing deep learning models show significant achievement in object detection, they are usually tested on datasets having mostly clean images. Thus, their performance levels were not measured on degraded images. In addition, images and videos in real-world scenarios often involve several natural artifacts such as noise, haze, rain, dust, and motion blur due to several factors such as insufficient light, atmospheric scattering, and faults in image sensors. This image acquisition-related problem becomes more severe when it comes to detecting small objects in aerial images. In this study, we investigate the small object identification performance of several state-of-the-art object detection models (Yolo 6/7/8) under three conditions (noisy, motion blurred, and rainy). Through this inspection, we evaluate the contribution of an image enhancement scheme so-called MPRNet. For this aim, we trained three OD algorithms with the original clean images of the VisDrone dataset. Followingly, we measured the detection performance of saved YOLO models against (1) clean, (2) degraded, and (3) enhanced counterparts. According to the results, MPRNet-based image enhancement promisingly contributes to the detection performance and YOLO8 outperforms its predecessors. We believe that this work presents useful findings for researchers studying aerial image-based vision tasks, especially under extreme weather and image acquisition conditions

Scale Enhancement Network for Object Detection in Aerial Images

The main challenge for object detection in aerial images is small object detection. Most existing methods use feature fusion strategies to enhance small object features in shallow layers but ignore the problem of inconsistent small object local region responses between feature layers, namely the semantic gap, which may lead to underutilization of small object information in multiple feature layers. To lift the above limitations, we propose a scale enhancement module that adaptively passes valuable small object features in different feature layers to shallow layers to alleviate the semantic gap problem. In particular, the module includes the novel fine-coarse self-attention mechanism, which captures global contextual information by performing strong interaction of pixel-level information at the local scale and weak interaction of region-level information at the global scale. In addition, the anchor assignment strategy based on the Intersection over Union (IoU) metric is not favorable for small objects as the IoU metric for small objects has a lower tolerance for position deviation compared to large ones. For this reason, we design the dynamic anchor assignment strategy with a scale-insensitive metric to assign adequate anchors to small objects. Extensive experiments on three aerial datasets demonstrate the effectiveness and adaptability of our method.

Hierarchical alignment network for domain adaptive object detection in aerial images

Domain Adaptive Object Detection (DAOD) alleviates the reliance on labeled data by transferring knowledge learned from labeled source domain to unlabeled target domain. Recent DAOD methods is modeled mainly based on ground-level images. Compared to ground-level images, aerial images suffer from scale variation and viewpoint diversity. This means that domain adaptive object detection in aerial images is a more challenging task. In this work, we address the domain shift in aerial images from two levels: 1) image-level shifts, such as weather, lighting, viewpoint, etc., 2) instance-level shifts, such as object appearance, scale, etc. Specifically, multiple domain-confusion classifiers are designed to learn image-level common knowledge of the source and target domains. Different levels of domain classifiers are further assigned adaptive weights to coordinate the transferability and discriminability of the adaptive detectors. Meanwhile, instance-level alignment is realized by forcing the intrinsic relationship between classes in both domains to be consistent. In addition, we perform instance-level alignment in different semantic-level feature layers to improve the scale awareness of the adaptation model. Extensive experimental results on VisDrone, UAVDT, DIOR and DOTA datasets demonstrate that our method achieves optimal detection performance in four domain adaption scenarios compared to other state-of-the-art methods, e.g., in Daytime → Night (VisDrone), the mAP50 is 23.5 %; in VisDrone → UAVDT, DIOR → UAVDT and DOTA → VisDrone, the AP50 of the car is 63.1 %, 46.6 % and 44.8 % respectively. Code will be available online (https://github.com/MaYou1997/HANet).

Semantic and geometric information propagation for oriented object detection in aerial images

Semantic and geometric information propagation for oriented object detection in aerial images

Cross-Grid Label Assignment for Arbitrary-Oriented Object Detection in Aerial Images

Cross-Grid Label Assignment for Arbitrary-Oriented Object Detection in Aerial Images

Class-Aware Prediction: A Solution to Center Point Collision for Anchor-Free Object Detection in Aerial Images

Class-Aware Prediction: A Solution to Center Point Collision for Anchor-Free Object Detection in Aerial Images

Effective Anchor Adaptation and Feature Enhancement Strategies for Tiny Object Detection in Aerial Images

Effective Anchor Adaptation and Feature Enhancement Strategies for Tiny Object Detection in Aerial Images

Task-Wise Sampling Convolutions for Arbitrary-Oriented Object Detection in Aerial Images.

Arbitrary-oriented object detection (AOOD) has been widely applied to locate and classify objects with diverse orientations in remote sensing images. However, the inconsistent features for the localization and classification tasks in AOOD models may lead to ambiguity and low-quality object predictions, which constrains the detection performance. In this article, an AOOD method called task-wise sampling convolutions (TS-Conv) is proposed. TS-Conv adaptively samples task-wise features from respective sensitive regions and maps these features together in alignment to guide a dynamic label assignment for better predictions. Specifically, sampling positions of the localization convolution in TS-Conv are supervised by the oriented bounding box (OBB) prediction associated with spatial coordinates, while sampling positions and convolutional kernel of the classification convolution are designed to be adaptively adjusted according to different orientations for improving the orientation robustness of features. Furthermore, a dynamic task-consistent-aware label assignment (DTLA) strategy is developed to select optimal candidate positions and assign labels dynamically according to ranked task-aware scores obtained from TS-Conv. Extensive experiments on several public datasets covering multiple scenes, multimodal images, and multiple categories of objects demonstrate the effectiveness, scalability, and superior performance of the proposed TS-Conv.

Position-sensitive Transformer aerial image object detection model

Position-sensitive Transformer aerial image object detection model

Expert teacher based on foundation image segmentation model for object detection in aerial images

Despite the remarkable progress of general object detection, the lack of labeled aerial images limits the robustness and generalization of the detector. Teacher–student learning is a feasible solution on natural image domain, but few works focus on unlabeled aerial images. Inspired by foundation models with the powerful generalization in computer vision field, we propose an expert teacher framework based on foundation image segmentation model called ET-FSM. Our approach provides the performance gains for the student detector by generating high-quality pseudo-labels for unlabeled aerial images. In the ET-FSM, we design the binary detector with expert guidance mechanism to sufficiently leverage the extra knowledge obtained from the foundation image segmentation model, which accurately detects object positions in the complex backgrounds. Also, we present the momentum contrast classification module to distinguish confused object categories in aerial images. To demonstrate the effectiveness of the proposed method, we construct an unlabeled aerial image dataset covering various scenes. The experiments are conducted on diverse types of student detectors. The results show that the proposed approach achieves superior performance compared to existing methods, and allows the student detector to achieve fully supervised performance with much less labeled aerial images. Our dataset and code are available at https://github.com/cq100/ET-FSM.

PERFORMANCE ASSESSMENT OF OBJECT DETECTION FROM MULTI SATELLITES AND AERIAL IMAGES

Abstract. Object detection in remote sensing imagery plays an important role in many applications, such as tracking and change detection. With the development of deep learning algorithms and advancement in hardware systems, improved accuracies have been achieved in the detection of various objects from remote sensing images. However, object detection across heterogeneous remote sensing imagery remains an important issue, particularly for satellite and aerial imagery. The colour variation for the same ground objects, variable resolutions, different platform heights, the parallax effect, and image distortion brought on by diverse shooting angles are the biggest hurdles in satellite-aerial detection applications. The research aims to obtain successful model for detecting aircrafts from satellite and aerial images and reduce cost and the gap of revisit time between sensors. The networks were tested using aerial, GF-2, Jilin-1 (JL-1) and Pleiades satellites test sets after being trained individually using the RGB high-resolution aerial set and panchromatic low-resolution GF-2 satellite set to validate the efficiency of the trained models. Also, the aerial-trained model and GF-2 satellite-trained model as dedicated models were compared with each other, and model trained by all dataset for Object Detection in Aerial Images (DOTA). It is observed that the anchor sizes and augmentation methods can enhance the performance of detection models. k-means algorithm and data augmentation were applied to produce better anchor box selection and avoid overfitting, atmospheric conditions problems, respectively. The accuracy assessment results demonstrate that the aerial-trained model outperforms the GF-2 satellite-trained model. In addition, the results of two dedicated detection models show improved accuracy compared to the DOTA-trained model.

Dense-and-Similar Object detection in aerial images

The general object detection performance has been improving significantly due to the prosperity of deep learning. When applied to aerial images, these algorithms perform poorly. There are, as we summarized, two practical reasons: (1) photographed from far above, objects are commonly small in size, dense in clusters, but sparsely scattered in the whole image; (2) some classes are naturally similar in appearance, not to mention the insufficiency of distinguishable visual features captured by cameras due to factors such as bad weather, various angles of view and long distances, etc. To address the two issues, we propose a dense-and-similar object detector with four key components: a coarse detector, an adaptive clustering procedure, a similar-class classifier, and a fine detector. At first, we input the original image into both the coarse and fine detectors. Then we cluster and patch the coarsely detected results adaptively to form a foreground region image. We feed the patched image into the coarse detector again and use the similar-class classifier to re-identify the labels of detected bounding boxes. At last, we put re-identified detections and outputs of the fine detector together, and obtain the final detections after non-maximal suppression. The approach proposed in this study achieved excellent results on both the VisDrone 2019 and DIOR datasets through experimental validation.

RA[formula omitted]DC-Net:A residual augment-convolutions and adaptive deformable convolution for points-based anchor-free orientation detection network in remote sensing images

With the rapid advancements in computer vision object detection networks, several detection networks based on remote sensing data have been proposed for oriented object detection. However, existing oriented networks have limitations in terms of feature extraction and oriented information extraction. To address these limitations, in this paper, we proposed an advanced adaptive points-based anchor-free oriented detection network called RA2DC-Net which includes a residual augment-convolutions (Res-AugConvs) module that uses residual structures and attention mechanisms to enhance feature focus information and minimize the loss of focus feature information. In addition, we proposed a method called adaptive deformable convolution (ADConv) to fine-tune custom convolution weights according to the features and generate high-quality adaptive offsets for accurate orientation information. Furthermore, we evaluated the performance of RA2DC-Net on two large-scale datasets: the Dataset for Object deTection in Aerial images (DOTA), the rotated object DetectIon in Optical Remote sensing images (DIOR-R) and the High-Resolution Ship Collection 2016 (HRSC2016). Experimental results demonstrated that RA2DC-Net achieved mAP values of 76.25%, 68.73% and 89.75% on the three datasets, respectively, with high AP values for various object classes on DOTA.