DOTA Dataset Research Articles

Enhancing a You Only Look Once-Plated Detector via Auxiliary Textual Coding for Multi-Scale Rotating Remote Sensing Objects in Transportation Monitoring Applications

With the rapid development of intelligent information technologies, remote sensing object detection has played an important role in different field applications. Particularly in recent years, it has attracted widespread attention in assisting with food safety supervision, which still faces troubling issues between oversized parameters and low performance that are challenging to solve. Hence, this article proposes a novel remote sensing detection framework for multi-scale objects with a rotating status and mutual occlusion, defined as EYMR-Net. This proposed approach is established on the YOLO-v7 architecture with a Swin Transformer backbone, which offers multi-scale receptive fields to mine massive features. Then, an enhanced attention module is added to exploit the spatial and dimensional interrelationships among different local characteristics. Subsequently, the effective rotating frame regression mechanism via circular smoothing labels is introduced to the EYMR-Net structure, addressing the problem of horizontal YOLO (You Only Look Once) frames ignoring direction changes. Extensive experiments on DOTA datasets demonstrated the outstanding performance of EYMR-Net, which achieved an impressive mAP0.5 of up to 74.3%. Further ablation experiments verified that our proposed approach obtains a balance between performance and efficiency, which is beneficial for practical remote sensing applications in transportation monitoring and supply chain management.

RSDM: Research of Rotation and Small Object Detection Models for Remote Sensing Images

Abstract Instance objects captured in aerial imagery display a range of orientations and exhibit significant scaling differences in their dimensions. Recent researches mainly address the above problems in terms of label assignment strategies and extended backbone spatial receptive fields, however, the former may still introduce lower quality samples, while the latter usually introduces considerable background noise. In this paper, the proposed Multiscale Convolutional Kernels Combined with Bi-Level Routing Attention for backbone Feature Extraction Network (MKBANet) employs multi-scale convolutional kernels to extract feature information at different scales, capture local context information, and combine with a Bi-Level Routing Attention (BRA) to capture remote context information and reduce computational complexity. In addition, an adaptive constrained oriented point set strategy (ACOPSS) is proposed. The adaptive point set constraint strategy is introduced in training, and then the quality metric function is utilized to select the top k high-quality directed point set samples in training, and finally the outliers are penalized. 81.23% mAP on DOTA dataset, the efficacy of the introduced approach is demonstrated.

Deep convolutional neural networks for ship detection using refined DOTA and TGRS-HRRSD high-resolution image datasets

Automated identification of ships in satellite imagery is of utmost significance in many military and civilian applications, including monitoring maritime traffic and maintaining maritime security. However, ship detection from remotely sensed imagery is a challenging task because of their varying sizes, orientations, types, and various interferences. In recent years, deep learning algorithms have become a powerful tool in image processing applications, including image classification and object detection due to their strong feature representation capabilities. In this study, a refined ship dataset, containing a total of 13,735 instances and representing different ship types, was generated using state-of-the-art datasets for the ship detection task. In a comparative framework, the performance of five deep learning algorithms (Faster R-CNN, YOLOv8, CenterNet, EfficientDet, and SSD) in ship detection was investigated using the created dataset. The performance of the deep learning models was compared using MS COCO accuracy metrics. In addition, the generalization capabilities of the models were tested on an independent image acquired by Göktürk-1 having a mucilage effect that undermines the spectral discrimination. Results revealed that the YOLOv8 model had the greatest ship detection performance (AP@0.50 = 96.33 %), followed by CenterNet (AP@0.50 = 88.80 %), Faster R-CNN (AP@0.50 = 85.30 %), EfficientDet (AP@0.50 = 78.91 %) and SSD (AP@0.5 = 75.98 %), and obtained the highest accuracies in terms of other COCO metrics. In addition, generalization capabilities on the test image confirmed the superiority of the YOLOv8 model with AP@0.50 score of 63.93 %. Overall, the results of this study validated the effectiveness of the YOLOv8 model in identifying ship targets in remotely sensed images.

Lightweight Detection Algorithm for Air-to-Ground Threat Object

Abstract In air-to-ground threat object detection, the typically strong concealment, small size, and high movement speed of threat objects often lead to challenges such as missed detections, low accuracy, and an excessive number of network parameters. An efficient lightweight detection algorithm for air-to-ground threat objects is proposed, built upon the YOLOv7-tiny framework. To address the issue of large network parameters, lightweight modules ELAN-GS and Slim-Neck are employed. Additionally, to refine the detection precision for small objects, the NWD metric and NWD loss function are introduced. Results from experiments on the DOTA dataset highlight that, compared with the original network, the proposed algorithm achieves a 33.2% reduction in parameters and a 26.3% reduction in computational cost. The small object accuracy, mAPs@0.5:0.95, is 23.7%, which is a 4.4% enhancement relative to the original network. The significant reduction in parameters and computational cost, along with the substantial improvement in small object detection accuracy, demonstrate the effectiveness of the proposed algorithm.

YOLO V3 AND CCN FOR THE TRACKING AND CLASSIFICATION OF AERIAL OBJECT AND DRONES

The goal of this study is to give headways in aeronautical article ID that will help with making recognitions that are both more exact and more precise. Specifically, we revamp the meaning of the article recognition anchor enclose request to remember turns for expansion to level and width, and besides, we make it conceivable to have erratic four corner point structures. Furthermore, the consideration of new anchor boxes gives the model additional adaptability to address protests that are focused at a pivot of turn that gives a 45-degree point. By accomplishing these results, we can make an organization that considers negligible tradeoffs about speed and unwavering quality, while likewise giving more exact restrictions. The latest ways to deal with PC vision and article acknowledgment are for the most part dependent on brain organizations and different advances that utilize profound learning. This powerful field of study is utilized in various applications, including military and observation, aeronautical photography, independent driving, and airborne perception. To precisely locate the location of an item, contemporary object identification techniques make use of bounding boxes that are drawn over the object and have a rectangular form (horizontal and vertical). These orthogonal bounding boxes do not consider the posture of the object, which leads to a decrease in the amount of object localization and restricts subsequent tasks such as object comprehension and tracking. We have used the DOTA dataset to present all of the results, demonstrating the value of flexible object boundaries, particularly with rotated and non-rectangular objects. We have also achieved an accuracy of 98.47% for the detection and classification of aerial objects, with forty percent of the data being used for training and the remaining twenty percent being used for testing. There was a minimum of 2.8 seconds of processing time required for the whole program to be executed to categorize all of the aerial items that were parked on the base.

Turning a CLIP Model Into a Scene Text Spotter.

We exploit the potential of the large-scale Contrastive Language-Image Pretraining (CLIP) model to enhance scene text detection and spotting tasks, transforming it into a robust backbone, FastTCM-CR50. This backbone utilizes visual prompt learning and cross-attention in CLIP to extract image and text-based prior knowledge. Using predefined and learnable prompts, FastTCM-CR50 introduces an instance-language matching process to enhance the synergy between image and text embeddings, thereby refining text regions. Our Bimodal Similarity Matching (BSM) module facilitates dynamic language prompt generation, enabling offline computations and improving performance. FastTCM-CR50 offers several advantages: 1) It can enhance existing text detectors and spotters, improving performance by an average of 1.6% and 1.5%, respectively. 2) It outperforms the previous TCM-CR50 backbone, yielding an average improvement of 0.2% and 0.55% in text detection and spotting tasks, along with a 47.1% increase in inference speed. 3) It showcases robust few-shot training capabilities. Utilizing only 10% of the supervised data, FastTCM-CR50 improves performance by an average of 26.5% and 4.7% for text detection and spotting tasks, respectively. 4) It consistently enhances performance on out-of-distribution text detection and spotting datasets, particularly the NightTime-ArT subset from ICDAR2019-ArT and the DOTA dataset for oriented object detection.

Enhancing Sustainable Traffic Monitoring: Leveraging NanoSight–YOLO for Precision Detection of Micro-Vehicle Targets in Satellite Imagery

Satellite remote sensing technology significantly aids road traffic monitoring through its broad observational scope and data richness. However, accurately detecting micro-vehicle targets in satellite imagery is challenging due to complex backgrounds and limited semantic information hindering traditional object detection models. To overcome these issues, this paper presents the NanoSight–YOLO model, a specialized adaptation of YOLOv8, to boost micro-vehicle detection. This model features an advanced feature extraction network, incorporates a transformer-based attention mechanism to emphasize critical features, and improves the loss function and BBox regression for enhanced accuracy. A unique micro-target detection layer tailored for satellite imagery granularity is also introduced. Empirical evaluations show improvements of 12.4% in precision and 11.5% in both recall and mean average precision (mAP) in standard tests. Further validation of the DOTA dataset highlights the model’s adaptability and generalization across various satellite scenarios, with increases of 3.6% in precision, 6.5% in recall, and 4.3% in mAP. These enhancements confirm NanoSight–YOLO’s efficacy in complex satellite imaging environments, representing a significant leap in satellite-based traffic monitoring.

Direction-aware multi-branch attention and Gaussian label assignment for remote sensing aggregative object detection

ABSTRACT A large number of objects of different scales appear aggregative distribution in remote sensing images, which brings two challenges to remote sensing object detection. The first challenge arises from the presence of objects with large scale differences within the same scene, which complicates the detector’s ability to balance the detection performance between large and small objects. The second challenge is that the distribution of dense objects on remote sensing images with complex backgrounds can reduce the feature representation capability of the network, leading to false detections and missed detections of objects. To address the two challenges present in remote sensing object detection, we propose a new object detection network DA-GLANet (Direction-Aware Multi-Branch Attention and Gaussian Label Assignment Balance Net). For the first challenge, we designed a multi-scale feature fusion module to fully utilize semantic information at different scales, thereby improving the detection accuracy of multi-scale objects. Subsequently, to tackle the second challenge, we proposed a multi-branch attention module with direction-aware capabilities. This module can enhance the network’s feature representation of objects of interest by generating attention maps that are both direction-aware and position-sensitive. In addition, we propose a label assignment strategy based on Gaussian distribution to address the imbalance of positive and negative labels caused by the complexity of remote sensing images. These methods work synergistically to improve aggregative object detection accuracy. We have conducted extensive experiments on the DOTA dataset, Levir dataset, and WHU buildings change detection dataset to demonstrate the superiority of our method. On the DOTA dataset, our method improves mAP by 0.4%, 0.3%, and 0.2% compared to the top three methods. On the Levir dataset, our method improves mAP by 2.5%, 2.2%, and 1.6% compared to the top three methods.

BiF-DETR:Remote sensing object detection based on Bidirectional information fusion

Remote Sensing Object Detection(RSOD) is a fundamental task in the field of remote sensing image processing. The complexity of the background, the diversity of object scales and the locality limitation of Convolutional Neural Network (CNN) present specific challenges for RSOD. In this paper, an innovative hybrid detector, Bidirectional Information Fusion DEtection TRansformer (BiF-DETR), is proposed to mitigate the above issues. Specifically, BiF-DETR takes anchor-free detection network, CenterNet, as the baseline, designs the feature extraction backbone in parallel, extracts the local feature details using CNNs, and obtains the global information and long-term dependencies using Transformer branch. A Bidirectional Information Fusion (BIF) module is elaborately designed to reduce the semantic differences between different styles of feature maps through multi-level iterative information interactions, fully utilizing the complementary advantages of different detectors. Additionally, Coordination Attention(CA), is introduced to enables the detection network to focus on the saliency information of small objects. To address diversity insufficiency of remote sensing images in the training stage, Cascade Mixture Data Augmentation (CMDA), is designed to improve the robustness and generalization ability of the model. Comparative experiments with other cutting-edge methods are conducted on the publicly available DOTA and NWPU VHR-10 datasets. The experimental results reveal that the performance of proposed method is state-of-the-art, with mAP reaching 77.43% and 94.75%, respectively, far exceeding the other 25 competitive methods.

SA3Det: Detecting Rotated Objects via Pixel-Level Attention and Adaptive Labels Assignment

Remote sensing of rotated objects often encounters numerous small and dense objects. To tackle small-object neglect and inaccurate angle predictions in elongated objects, we propose SA3Det, a novel method employing Pixel-Level Attention and Adaptive Labels Assignment. First, we introduce a self-attention module that learns dense pixel-level relations between features extracted by the backbone and neck, effectively preserving and exploring the spatial relationships of potential small objects. We then introduce an adaptive label assignment strategy that refines proposals by assigning labels based on loss, enhancing sample selection during training. Additionally, we designed an angle-sensitive module that enhances angle prediction by learning rotational feature maps and incorporating multi-angle features. These modules significantly enhance detection accuracy and yield high-quality region proposals. Our approach was validated by experiments on the DOTA and HRSC2016 datasets, demonstrating that SA3Det achieves mAPs of 76.31% and 89.4%, respectively.

AFE-YOLOv8: A Novel Object Detection Model for Unmanned Aerial Vehicle Scenes with Adaptive Feature Enhancement

Object detection in unmanned aerial vehicle (UAV) scenes is a challenging task due to the varying scales and complexities of targets. To address this, we propose a novel object detection model, AFE-YOLOv8, which integrates three innovative modules: the Multi-scale Nonlinear Fusion Module (MNFM), the Adaptive Feature Enhancement Module (AFEM), and the Receptive Field Expansion Module (RFEM). The MNFM introduces nonlinear mapping by exploiting the property that deformable convolution can dynamically adjust the shape of the convolution kernel according to the shape of the target, and it effectively enhances the feature extraction capability of the backbone network by integrating multi-scale feature maps from different mapping branches. Meanwhile, the AFEM introduces an adaptive fusion factor, and through the fusion factor, it adaptively integrates the small-target features contained in the feature maps of different detection branches into the small-target detection branch, thus enhancing the expression of the small-target features contained in the feature maps of the small-target detection branch. Furthermore, the RFEM expands the receptive field of the feature maps of the large- and medium-scale target detection branches through stacked convolution, so as to make the model’s receptive field cover the whole target, and thereby learn more rich and comprehensive features of the target. The experimental results demonstrate the superior performance of the proposed model compared to the baseline in detecting objects of various scales. On the VisDrone dataset, the proposed model achieves a 4.5% enhancement in mean average precision (mAP) and a 5.45% improvement in average precision at an IOU threshold of 0.5 (AP50). Additionally, ablation experiments conducted on the challenging DOTA dataset showcase the model’s robustness and generalization capabilities.

Comparative Analysis of CNN-Based Smart Pre-Trained Models for Object Detection on DOTA

In this paper, we proposed comparative research on the classification of various objects in satellite images using some pre-trained models of CNN (VGG-16, Inception-V3, ResNet-50, EfficientNet-B7) and R-CNN. In this research work, we have used the DOTA dataset, which combines data from 14 classes. We have implemented above mentioned pre-trained models of CNN, and R-CNN to achieve optimal results for accuracy as well as productivity. To detect objects like ships, tennis courts, swimming pools, vehicles, and harbors from remotely accessed images. In this study, we have used a convolutional neural network (CNN) as the base model. The transfer learning mechanism is employed to speed up the results and for complex computations. We have discovered with the help of experimental analysis that R-CNN and Inception-V3 are performing best out of the five pre-trained models.

HVConv: Horizontal and Vertical Convolution for Remote Sensing Object Detection

Generally, the interesting objects in aerial images are completely different from objects in nature, and the remote sensing objects in particular tend to be more distinctive in aspect ratio. The existing convolutional networks have equal aspect ratios of the receptive fields, which leads to receptive fields either containing non-relevant information or being unable to fully cover the entire object. To this end, we propose Horizontal and Vertical Convolution, which is a plug-and-play module to address different aspect ratio problems. In our method, we introduce horizontal convolution and vertical convolution to expand the receptive fields in the horizontal and vertical directions, respectively, to reduce redundant receptive fields, so that remote sensing objects with different aspect ratios can achieve better receptive fields coverage, thereby achieving more accurate feature representation. In addition, we design an attention module to dynamically aggregate these two sub-modules to achieve more accurate feature coverage. Extensive experimental results on the DOTA and HRSC2016 datasets show that our HVConv achieves accuracy improvements in diverse detection architectures and obtains SOTA accuracy (mAP score of 77.60% with DOTA single-scale training and mAP score of 81.07% with DOTA multi-scale training). Various ablation studies were conducted as well, which is enough to verify the effectiveness of our model.

EB-Net: an efficient balanced network for accuracy and speed of remote sensing detection

ABSTRACT Remote sensing detection is a difficult task that requires not only identifying objects with complex background, quality, and angle issues, but also being lightweight enough to be carried by edge devices. In real remote sensing scenarios, achieving accurate, fast, and low-resource-consumption automated detection remains a significant challenge. Therefore, this paper proposes an efficient balanced network (EB-Net) for real remote sensing devices. First, a dynamic sparse attention (DSA) mechanism is proposed and has been proven with high performance via the complexity analysis. In addition, a new dynamic sparse transformer (DSFormer) is constructed using DSA, which enhances feature information and adapts to image resolution by self-attention and multi-headed attention, and achieves more flexible computation by random sampling. Then, three versions of discrete distribution IoU (DDIoU) are defined for adapting various scenarios and tasks in remote sensing, and this loss function makes the model achieve high accuracy and more lightweight. Finally, to make the model more lightweight, a cropped SPPF (CroSPPF) is presented, which significantly improves the computational efficiency by the lightweight of the sequence structure and activation function. Ablation experiments are conducted on the NWPU VHR-10 dataset, and demonstrate the effectiveness of the proposed methods. Numerous comparisons with state-of-the-art detectors are conducted on the NWPU VHR-10, RSOD and DOTA datasets. The experiments show that EB-Net outperforms state-of-the-art remote sensing detection models in comprehensive performance and achieves end-to-end accurate and lightweight detection.

Improving Oriented Object Detection by Scene Classification and Task-Aligned Focal Loss

Oriented object detection (OOD) can precisely detect objects with arbitrary direction in remote sensing images (RSIs). Up to now, the two-stage OOD methods have attracted more attention because of their high detection accuracy. However, the two-stage methods only rely on the features of each proposal for object recognition, which leads to the misclassification problem because of the intra-class diversity, inter-class similarity and clutter backgrounds in RSIs. To address the above problem, an OOD model combining scene classification is proposed. Considering the fact that each foreground object has a strong contextual relationship with the scene of the RSI, a scene classification branch is added to the baseline OOD model, and the scene classification result of input RSI is used to exclude the impossible categories. To focus on the hard instances and enhance the consistency between classification and regression, a task-aligned focal loss (TFL) which combines the classification difficulty with the regression loss is proposed, and TFL assigns lager weights to the hard instances and optimizes the classification and regression branches simultaneously. The ablation study proves the effectiveness of scene classification branch, TFL and their combination. The comparisons with 15 and 14 OOD methods on the DOTA and DIOR-R datasets validate the superiority of our method.

PARDet: Dynamic point set alignment for rotated object detection

Due to the inherent mismatch between rotated objects and horizontal features, feature point misalignment has been a challenge in the task of Rotated Object Detection (ROD). Specifically, considering the pattern of convolution, foreground features are often mixed with background noise, which can confuse the model and affect the model from feature point alignment during the training phase. To mitigate this issue, previous methods concentrate on fixed positions derived from predicted boxes by additionally introducing a refinement stage. However, merely learning fixed position priors during training can result in suboptimal alignment and inefficiency during inference. This paper introduces a dynamic point alignment detector to concurrently address issues associated with feature misalignment and inference inefficiency. The method is made up of two components: the fine-grained points generator (FPG) captures key information, and the point alignment module (PAM) derives precise feature representations. Both modules empower the detector with the capability to dynamically perceive rotated objects, extracting more comprehensive and reasoned feature contents from the feature maps. In general, our method enables the model to independently identify and prioritize the valuable features during the training process. Subsequently, during the inference stage, the results can be directly predicted without additional alignment operations. The experimental results demonstrate that our method can achieve competitive and superior results with average precision (AP) values of 79.33%, 95.73%, and 63.37% on the DOTA, HRSC2016, and DIOR-R datasets, respectively. Codes will be publicly available at https://github.com/Xuyihaoby/PARDet.

Discretization and decoupled knowledge distillation for arbitrary oriented object detection

In recent years, lightweight object detection networks have been increasingly applied to remote sensing platforms due to their fast inference speed and flexible deployment advantages. Knowledge distillation methods have been widely used to reduce the performance gap between large and small models, and many studies have combined knowledge distillation with object detection. However, existing knowledge distillation methods often overlook the transfer of localization knowledge. Therefore, this paper proposes a method called Discretized Position Knowledge Distillation (DPKD) to improve the use of knowledge distillation in object detection. Specifically, the DPKD method incorporates a Discretization Algorithm Module (DAM), which leverages both general probability distribution and cross-Gaussian distribution to transfer high-quality bounding box position and pose information. Additionally, the Position Knowledge Distillation (PKD) method splits the target and non-target bounding boxes to form the distillation loss function, addressing the issue of missing background knowledge transfer during the distillation process. To further enhance the learning of high-quality bounding boxes, a Region Weighting Module (RWM) based on EIoU is introduced in DPKD, assigning weights to the various bounding boxes in the teacher's output. The effectiveness of DPKD in the field of remote sensing image object detection in multi-modal scenarios was verified through multi-modal training on the publicly available DOTA and HRSID datasets.

AFDet: alignment and focusing for aerial object detection

ABSTRACT Object detection in aerial images has become a focus in recent years due to the expansion of its application fields. Aerial objects have the characteristics of multi-scale, arbitrary angle and dense arrangement, which brings considerable challenges to the task. Based on the efficient one-stage detectors, most methods design modules to adaptively align the feature receptive field with the anchors to improve the accuracy of the bounding box regression and calculate loss according to parameter bias. Current methods are not adequately data-driven in feature optimization and the angle parameter regression faces boundary problems. To handle these problems, in this letter, we propose an Align-Focus detector (AFDet) for aerial image object detection. We introduce the ideology of deformable multi-head self-attention to feature optimization and design a new Align-Focus Module (AFM) which can sensitively focus feature encoding response to the real texture area of the objects. In addition, we apply KF-IOU Loss to solve the boundary problem. We conduct necessary experiments on DOTA and HRSC2016 datasets, and AFDet achieves the highest mAP performance compared to the existing methods.

YOLO-RSA: A Multiscale Ship Detection Algorithm Based on Optical Remote Sensing Image

Currently, deep learning is extensively utilized for ship target detection; however, achieving accurate and real-time detection of multi-scale targets remains a significant challenge. Considering the diverse scenes, varied scales, and complex backgrounds of ships in optical remote sensing images, we introduce a network model named YOLO-RSA. The model consists of a backbone feature extraction network, a multi-scale feature pyramid, and a rotated detection head. We conduct thorough tests on the HRSC2016 and DOTA datasets to validate the proposed algorithm. Through ablation experiments, we assess the impact of each improvement component on the model. In comparative experiments, the proposed model surpasses other models in terms of Recall, Precision, and MAP on the HRSC2016 dataset. Finally, in generalization experiments, our proposed ship detection model exhibits excellent detection performance across various scenarios. The method can accurately detect multi-scale ships in the image and provide a basis for marine ship monitoring and port management.

Enhancing class-incremental object detection in remote sensing through instance-aware distillation

Object detection plays a important role within the field of remote sensing, boasting significant applications including intelligent monitoring and urban planning. However, traditional models are constrained by predefined classes and encounter a challenge known as catastrophic forgetting when attempting to learn new classes post-deployment. To address this problem, we propose a novel Instance-aware Distillation approach for Class-incremental Object Detection (IDCOD). Our approach capitalizes on the teacher model, a model from a previous stage, to serve as a guide during the training of the new model on novel data. This methodology facilitates the gradual acquisition of knowledge about new classes while simultaneously preserving the performance achieved on previously learned classes. Instance-aware distillation with masks of old and new classes aims to reduce forgetting and impact on new classes. Furthermore, we design a pseudo-label module to expand old class training data. Experiments on the challenging DOTA dataset, DIOR dataset, RTDOD dataset and PASCAL VOC dataset show that our method effectively detects old classes, incrementally detects new classes, and mitigates catastrophic forgetting.