Object Detection In Aerial Images Research Articles

Class-Aware Prediction: A Solution to Center Point Collision for Anchor-Free Object Detection in Aerial Images

Class-Aware Prediction: A Solution to Center Point Collision for Anchor-Free Object Detection in Aerial Images

TIR-Net: Task Integration Based on Rotated Convolution Kernel for Oriented Object Detection in Aerial Images

TIR-Net: Task Integration Based on Rotated Convolution Kernel for Oriented Object Detection in Aerial Images

Enhanced Pre-processing for Robust Small Object Detection

Small object detection in aerial images poses a significant challenge due to their limited size and low resolution. Most existing studies only focus on the average precision (AP) for small object detection methods such as one-stage, two-stage, and anchored-free detection methods. Those studies lack speed analysis, and the state-of-the-art methods are hardly understandable. In this study, we present propose a novel data processing method to improve the all-class AP on DOTA-v1.5 datasets with the YOLOv8 model. Our approach leverages the YOLOv8 with darknet architecture as a backbone, which has shown promising results in object detection tasks. This study incorporates noise handling, data formatting, and then the data normalization method to enhance the representation of small objects, improving their visibility and discriminative power. To evaluate the performance of our proposed model, extensive experiments on the DOTA-v1.5 dataset are conducted. The results demonstrate that our model achieves significant improvement in all class AP compared to other methods for small object detection.

Effective Anchor Adaptation and Feature Enhancement Strategies for Tiny Object Detection in Aerial Images

Effective Anchor Adaptation and Feature Enhancement Strategies for Tiny Object Detection in Aerial Images

Task-Wise Sampling Convolutions for Arbitrary-Oriented Object Detection in Aerial Images.

Arbitrary-oriented object detection (AOOD) has been widely applied to locate and classify objects with diverse orientations in remote sensing images. However, the inconsistent features for the localization and classification tasks in AOOD models may lead to ambiguity and low-quality object predictions, which constrains the detection performance. In this article, an AOOD method called task-wise sampling convolutions (TS-Conv) is proposed. TS-Conv adaptively samples task-wise features from respective sensitive regions and maps these features together in alignment to guide a dynamic label assignment for better predictions. Specifically, sampling positions of the localization convolution in TS-Conv are supervised by the oriented bounding box (OBB) prediction associated with spatial coordinates, while sampling positions and convolutional kernel of the classification convolution are designed to be adaptively adjusted according to different orientations for improving the orientation robustness of features. Furthermore, a dynamic task-consistent-aware label assignment (DTLA) strategy is developed to select optimal candidate positions and assign labels dynamically according to ranked task-aware scores obtained from TS-Conv. Extensive experiments on several public datasets covering multiple scenes, multimodal images, and multiple categories of objects demonstrate the effectiveness, scalability, and superior performance of the proposed TS-Conv.

Position-sensitive Transformer aerial image object detection model

Position-sensitive Transformer aerial image object detection model

Expert teacher based on foundation image segmentation model for object detection in aerial images

Despite the remarkable progress of general object detection, the lack of labeled aerial images limits the robustness and generalization of the detector. Teacher–student learning is a feasible solution on natural image domain, but few works focus on unlabeled aerial images. Inspired by foundation models with the powerful generalization in computer vision field, we propose an expert teacher framework based on foundation image segmentation model called ET-FSM. Our approach provides the performance gains for the student detector by generating high-quality pseudo-labels for unlabeled aerial images. In the ET-FSM, we design the binary detector with expert guidance mechanism to sufficiently leverage the extra knowledge obtained from the foundation image segmentation model, which accurately detects object positions in the complex backgrounds. Also, we present the momentum contrast classification module to distinguish confused object categories in aerial images. To demonstrate the effectiveness of the proposed method, we construct an unlabeled aerial image dataset covering various scenes. The experiments are conducted on diverse types of student detectors. The results show that the proposed approach achieves superior performance compared to existing methods, and allows the student detector to achieve fully supervised performance with much less labeled aerial images. Our dataset and code are available at https://github.com/cq100/ET-FSM.

PERFORMANCE ASSESSMENT OF OBJECT DETECTION FROM MULTI SATELLITES AND AERIAL IMAGES

Abstract. Object detection in remote sensing imagery plays an important role in many applications, such as tracking and change detection. With the development of deep learning algorithms and advancement in hardware systems, improved accuracies have been achieved in the detection of various objects from remote sensing images. However, object detection across heterogeneous remote sensing imagery remains an important issue, particularly for satellite and aerial imagery. The colour variation for the same ground objects, variable resolutions, different platform heights, the parallax effect, and image distortion brought on by diverse shooting angles are the biggest hurdles in satellite-aerial detection applications. The research aims to obtain successful model for detecting aircrafts from satellite and aerial images and reduce cost and the gap of revisit time between sensors. The networks were tested using aerial, GF-2, Jilin-1 (JL-1) and Pleiades satellites test sets after being trained individually using the RGB high-resolution aerial set and panchromatic low-resolution GF-2 satellite set to validate the efficiency of the trained models. Also, the aerial-trained model and GF-2 satellite-trained model as dedicated models were compared with each other, and model trained by all dataset for Object Detection in Aerial Images (DOTA). It is observed that the anchor sizes and augmentation methods can enhance the performance of detection models. k-means algorithm and data augmentation were applied to produce better anchor box selection and avoid overfitting, atmospheric conditions problems, respectively. The accuracy assessment results demonstrate that the aerial-trained model outperforms the GF-2 satellite-trained model. In addition, the results of two dedicated detection models show improved accuracy compared to the DOTA-trained model.

Dense-and-Similar Object detection in aerial images

The general object detection performance has been improving significantly due to the prosperity of deep learning. When applied to aerial images, these algorithms perform poorly. There are, as we summarized, two practical reasons: (1) photographed from far above, objects are commonly small in size, dense in clusters, but sparsely scattered in the whole image; (2) some classes are naturally similar in appearance, not to mention the insufficiency of distinguishable visual features captured by cameras due to factors such as bad weather, various angles of view and long distances, etc. To address the two issues, we propose a dense-and-similar object detector with four key components: a coarse detector, an adaptive clustering procedure, a similar-class classifier, and a fine detector. At first, we input the original image into both the coarse and fine detectors. Then we cluster and patch the coarsely detected results adaptively to form a foreground region image. We feed the patched image into the coarse detector again and use the similar-class classifier to re-identify the labels of detected bounding boxes. At last, we put re-identified detections and outputs of the fine detector together, and obtain the final detections after non-maximal suppression. The approach proposed in this study achieved excellent results on both the VisDrone 2019 and DIOR datasets through experimental validation.

RA[formula omitted]DC-Net:A residual augment-convolutions and adaptive deformable convolution for points-based anchor-free orientation detection network in remote sensing images

With the rapid advancements in computer vision object detection networks, several detection networks based on remote sensing data have been proposed for oriented object detection. However, existing oriented networks have limitations in terms of feature extraction and oriented information extraction. To address these limitations, in this paper, we proposed an advanced adaptive points-based anchor-free oriented detection network called RA2DC-Net which includes a residual augment-convolutions (Res-AugConvs) module that uses residual structures and attention mechanisms to enhance feature focus information and minimize the loss of focus feature information. In addition, we proposed a method called adaptive deformable convolution (ADConv) to fine-tune custom convolution weights according to the features and generate high-quality adaptive offsets for accurate orientation information. Furthermore, we evaluated the performance of RA2DC-Net on two large-scale datasets: the Dataset for Object deTection in Aerial images (DOTA), the rotated object DetectIon in Optical Remote sensing images (DIOR-R) and the High-Resolution Ship Collection 2016 (HRSC2016). Experimental results demonstrated that RA2DC-Net achieved mAP values of 76.25%, 68.73% and 89.75% on the three datasets, respectively, with high AP values for various object classes on DOTA.

SenseLite: A YOLO-Based Lightweight Model for Small Object Detection in Aerial Imagery.

In the field of aerial remote sensing, detecting small objects in aerial images is challenging. Their subtle presence against broad backgrounds, combined with environmental complexities and low image resolution, complicates identification. While their detection is crucial for urban planning, traffic monitoring, and military reconnaissance, many deep learning approaches demand significant computational resources, hindering real-time applications. To elevate the accuracy of small object detection in aerial imagery and cater to real-time requirements, we introduce SenseLite, a lightweight and efficient model tailored for aerial image object detection. First, we innovatively structured the YOLOv5 model for a more streamlined structure. In the backbone, we replaced the original structure with cutting-edge lightweight neural operator Involution, enhancing contextual semantics and weight distribution. For the neck, we incorporated GSConv and slim-Neck, striking a balance between reduced computational complexity and performance, which is ideal for rapid predictions. Additionally, to enhance detection accuracy, we integrated a squeeze-and-excitation (SE) mechanism to amplify channel communication and improve detection accuracy. Finally, the Soft-NMS strategy was employed to manage overlapping targets, ensuring precise concurrent detections. Performance-wise, SenseLite reduces parameters by 30.5%, from 7.05 M to 4.9 M, as well as computational demands, with GFLOPs decreasing from 15.9 to 11.2. It surpasses the original YOLOv5, showing a 5.5% mAP0.5 improvement, 0.9% higher precision, and 1.4% better recall on the DOTA dataset. Compared to other leading methods, SenseLite stands out in terms of performance.

DATA AUGMENTATION ANALYSIS OF VEHICLE DETECTION IN AERIAL IMAGES

Drones are increasingly used in various application domains including surveillance, agriculture, delivery, search and rescue missions. Object detection in aerial images (captured by drones) gradually gains more interest in computer vision community. However, research activities are still very few in this area due to numerous challenges such as top-view angle, small-scale object, diverse directions, and data imbalance. In this paper, we investigate different data augmentation techniques. Furthermore, we propose combining data augmentation methods to further enhance the performance of the state-of-the-art object detection methods. Extensive experiments on two datasets, namely, AERIAU, and XDUAV, demonstrate that the combination of random cropped and vertical flipped data boosts the performance of object detectors on aerial images.

Long-Range Thermal Target Detection in Data-Limited Settings Using Restricted Receptive Fields.

Long-range target detection in thermal infrared imagery is a challenging research problem due to the low resolution and limited detail captured by thermal sensors. The limited size and variability in thermal image datasets for small target detection is also a major constraint for the development of accurate and robust detection algorithms. To address both the sensor and data constraints, we propose a novel convolutional neural network (CNN) feature extraction architecture designed for small object detection in data-limited settings. More specifically, we focus on long-range ground-based thermal vehicle detection, but also show the effectiveness of the proposed algorithm on drone and satellite aerial imagery. The design of the proposed architecture is inspired by an analysis of popular object detectors as well as custom-designed networks. We find that restricted receptive fields (rather than more globalized features, as is the trend), along with less downsampling of feature maps and attenuated processing of fine-grained features, lead to greatly improved detection rates while mitigating the model's capacity to overfit on small or poorly varied datasets. Our approach achieves state-of-the-art results on the Defense Systems Information Analysis Center (DSIAC) automated target recognition (ATR) and the Tiny Object Detection in Aerial Images (AI-TOD) datasets.

An Attentive Consensus Platform for Collaborative Reinforcement Learning Agents

This work integrates concurrent team learning into a deep reinforcement learning model for collaborative multiagent problems to reach a common goal. RL trains each agent to decide its best actions, and then the joint decision is made to accomplish the assigned team task based on the collaborative platform. Each agent learns individually and shares the observed feature with others. The collaborative platform makes the decision based on these features provided by agents. The collaborative platform consists of two attention mechanisms—soft and self-attention. The self-attention module is applied to distinguish local environmental features sensed by an individual agent for concurrent learning. The platform integrates collective information locally from agents through a soft attention interface. A termination network is introduced to the collaborative model to determine the number of elapsed sequences in collecting sufficient information to support the joint decision instead of a fixed number of iterations. Experiments on limit-sighted agent deployment for static image classifications, video game playing, and aerial image object detection of multiple airborne sensors have evaluated the performance of the proposed model. The experimental results show that the proposed model provides a more efficient collaborative mechanism for multiagent systems than the state-of-the-art methods.

Improved TPH for object detection in aerial images

ABSTRACT Deep learning has greatly enhanced the general object detection capability. However, when directly applied to aerial images, performance drops significantly due to: (1) Most objects in aerial images are dense and small; (2) UAV altitude variations cause diverse object scales. In this paper, we improve TPH algorithm for exceptional aerial object detection performance. Specifically, we introduce the SPD module to replace the strided convolution layers and pooling layers. And we improve the TPH backbone and neck networks so that large and small objects can be detected accurately. Experiments on VisDrone2019 and DOTA datasets validate the effectiveness of our method.

Satellite and Aerial Image Restoration Using Deep Reinforcement Learning

In this paper, we present a deep reinforcement learning-based method for effectively denoising satellite and aerial imagery data. Noise of various kinds and with varying noise levels contaminates satellite imagery data. The image’s quality and readability suffer when there is noise present. Therefore, it is crucial to create a network that can effectively and efficiently remove noise from the image while also preserving its quality and signal components. This paper evaluates the denoising capabilities of the deep reinforcement learning system. The proposed network is trained using the training set from the “dataset of object detection in aerial images (DOTA) dataset,” and its hyperparameters were adjusted for optimum performance. The training set from the aforementioned dataset was used to train the proposed network. The trained network was given the test set of unseen images for denoising. Statistical denoising, a common denoising technique, was used on the test dataset, and the outcomes were assessed. The same unseen images were also given to existing CNN-based denoising algorithms like denoising using CNN (DnCNN), U-shaped DnCNN (UDnCNN), and dilated U-shaped DnCNN (DUDnCNN), designed specifically for image denoising. Runtime and structural similarity index (SSIM) as well as peak signal-to-noise ratio have both been used as evaluation metrics to compare the effectiveness of various approaches. It is discovered that, when comparing the performance of various systems, the suggested system outperforms both statistical- and CNN-based denoising in terms of the evaluation metrics, PSNR and SSIM.

Efficient-Lightweight YOLO: Improving Small Object Detection in YOLO for Aerial Images.

The most significant technical challenges of current aerial image object-detection tasks are the extremely low accuracy for detecting small objects that are densely distributed within a scene and the lack of semantic information. Moreover, existing detectors with large parameter scales are unsuitable for aerial image object-detection scenarios oriented toward low-end GPUs. To address this technical challenge, we propose efficient-lightweight You Only Look Once (EL-YOLO), an innovative model that overcomes the limitations of existing detectors and low-end GPU orientation. EL-YOLO surpasses the baseline models in three key areas. Firstly, we design and scrutinize three model architectures to intensify the model's focus on small objects and identify the most effective network structure. Secondly, we design efficient spatial pyramid pooling (ESPP) to augment the representation of small-object features in aerial images. Lastly, we introduce the alpha-complete intersection over union (α-CIoU) loss function to tackle the imbalance between positive and negative samples in aerial images. Our proposed EL-YOLO method demonstrates a strong generalization and robustness for the small-object detection problem in aerial images. The experimental results show that, with the model parameters maintained below 10 M while the input image size was unified at 640 × 640 pixels, the APS of the EL-YOLOv5 reached 10.8% and 10.7% and enhanced the APs by 1.9% and 2.2% compared to YOLOv5 on two challenging aerial image datasets, DIOR and VisDrone, respectively.

MKIoU loss: toward accurate oriented object detection in aerial images

Oriented bounding box regression is crucial for oriented object detection. However, regression-based methods often suffer from boundary problems and the inconsistency between loss and evaluation metrics. A modulated Kalman intersection over union (IoU) loss of approximate SkewIoU is proposed, named MKIoU. To avoid boundary problems, we convert the oriented bounding box to Gaussian distribution then use the Kalman filter to approximate the intersection area. However, there exists significant difference between the calculated and actual intersection areas. Thus, we propose a modulation factor to adjust the sensitivity of angle deviation and width-height offset to loss variation, making the loss more consistent with the evaluation metric. Furthermore, the Gaussian modeling method avoids the boundary problem but causes the angle confusion of square objects simultaneously. Thus, the Gaussian angle loss (GA loss) is presented to solve this problem by adding a corrected loss for square targets. The proposed GA loss can be easily extended to other Gaussian-based methods. Experiments on three publicly available aerial image datasets, DOTA, UCAS-AOD, and HRSC2016, show the effectiveness of the proposed method.

Deep Spatial Feature Transformation for Oriented Aerial Object Detection

Object detection in aerial images has received extensive attention in the field of computer vision. Different from natural images, the aerial objects are usually distributed in any direction. Therefore, the existing detector usually needs more parameters to encode the direction information, resulting in a large number of redundant calculations. In addition, because an ordinary convolution neural network (CNN) does not effectively model the direction change, a large amount of the rotated data is required for the aerial detector. To solve these problems, we propose a Deep Spatial Feature Transformation Network (DSFTNet), which includes a Spatial Feature Extraction Module and a Feature Selection Module. Specifically, we add the rotation convolution kernel to the detector to extract the directional feature of the rotated target to accurately predict the direction of the model. Then we build a dual pyramid to separate the features in the classification and regression tasks. Finally, the polarization function is proposed to construct the critical features that are suitable for their respective tasks, achieving feature selection and more refined detection. Experiments on public remote sensing benchmarks (e.g. DOTA, HRSC2016, and UCASAOD) have proved the effectiveness of our detector.

Lightweight aerial image object detection algorithm based on improved YOLOv5s

YOLOv5 is one of the most popular object detection algorithms, which is divided into multiple series according to the control of network depth and width. To realize the deployment of mobile devices or embedded devices, the paper proposes a lightweight aerial image object detection algorithm (LAI-YOLOv5s) based on the improvement of YOLOv5s with a relatively small amount of calculation and parameter and relatively fast reasoning speed. Firstly, to better detect small objects, the paper replaces the minimum detection head with the maximum detection head and proposes a new feature fusion method, DFM-CPFN(Deep Feature Map Cross Path Fusion Network), to enrich the semantic information of deep features. Secondly, the paper designs a new module based on VoVNet to improve the feature extraction ability of the backbone network. Finally, based on the idea of ShuffleNetV2, the paper makes the network more lightweight without affecting detection accuracy. Based on the VisDrone2019 dataset, the detection accuracy of LAI-YOLOv5s on the mAP@0.5 index is 8.3% higher than that of the original algorithm. Compared with other series of YOLOv5 and YOLOv3 algorithms, LAI-YOLOv5s has the advantages of low computational cost and high detection accuracy.