DOTA Dataset Research Articles

MOON: A Subspace-Based Multi-Branch Network for Object Detection in Remotely Sensed Images

The effectiveness of training-based object detection heavily depends on the amount of sample data. But in the field of remote sensing, the amount of sample data is difficult to meet the needs of network training due to the non-cooperative imaging modes and complex imaging conditions. Moreover, the imbalance of the sample data between different categories may lead to the long-tail problem during the training. Given that similar sensors, data acquisition approaches, and data structures could make the targets in different categories possess certain similarities, those categories can be modeled together within a subspace rather than the entire space to leverage the amounts of sample data in different subspaces. To this end, a subspace-dividing strategy and a subspace-based multi-branch network is proposed for object detection in remotely sensed images. Specifically, a combination index is defined to depict this kind of similarity, a generalized category consisting of similar categories is proposed to represent the subspace, and a new subspace-based loss function is devised to address the relationship between targets in one subspace and across different subspaces to integrate the sample data from similar categories within a subspace and to balance the amounts of sample data between different subspaces. Furthermore, a subspace-based multi-branch network is constructed to ensure the subspace-aware regression. Experiments on the DOTA and HRSC2016 datasets demonstrated the superiority of our proposed method.

Oriented Object Detection in Remote Sensing Using an Enhanced Feature Pyramid Network

Object detection in remote sensing images is a critical task within the field of remote sensing image interpretation and analysis, serving as a fundamental foundation for military surveillance and traffic guidance. Recently, although many object detection algorithms have been improved to adapt to the characteristics of remote sensing images and have achieved good performance, most of them still use horizontal bounding boxes, which struggle to accurately mark targets with multiple angles and dense arrangements in remote sensing images. We propose an oriented bounding box optical remote sensing image object detection method based on an enhanced feature pyramid, and add an attention module to suppress background noise. To begin with, we incorporate an angle prediction module that accurately locates the detection target. Subsequently, we design an enhanced feature pyramid network, utilizing deformable convolutions and feature fusion modules to enhance the feature information of rotating targets and improve the expressive capacity of features at all levels. The proposed algorithm in this paper performs well on the public DOTA dataset and HRSC2016 dataset, compared with other object detection methods, and the detection accuracy AP values of most object categories are improved by at least three percentage points. The results show that our method can accurately locate densely arranged and dynamically oriented targets, significantly reducing the risk of missing detections, and achieving higher levels of target detection accuracy.

On-Board Multi-Class Geospatial Object Detection Based on Convolutional Neural Network for High Resolution Remote Sensing Images

Multi-class geospatial object detection in high-resolution remote sensing images has significant potential in various domains such as industrial production, military warning, disaster monitoring, and urban planning. However, the traditional process of remote sensing object detection involves several time-consuming steps, including image acquisition, image download, ground processing, and object detection. These steps may not be suitable for tasks with shorter timeliness requirements, such as military warning and disaster monitoring. Additionally, the transmission of massive data from satellites to the ground is limited by bandwidth, resulting in time delays and redundant information, such as cloud coverage images. To address these challenges and achieve efficient utilization of information, this paper proposes a comprehensive on-board multi-class geospatial object detection scheme. The proposed scheme consists of several steps. Firstly, the satellite imagery is sliced, and the PID-Net (Proportional-Integral-Derivative Network) method is employed to detect and filter out cloud-covered tiles. Subsequently, our Manhattan Intersection over Union (MIOU) loss-based YOLO (You Only Look Once) v7-Tiny method is used to detect remote-sensing objects in the remaining tiles. Finally, the detection results are mapped back to the original image, and the truncated NMS (Non-Maximum Suppression) method is utilized to filter out repeated and noisy boxes. To validate the reliability of the scheme, this paper creates a new dataset called DOTA-CD (Dataset for Object Detection in Aerial Images-Cloud Detection). Experiments were conducted on both ground and on-board equipment using the AIR-CD dataset, DOTA dataset, and DOTA-CD dataset. The results demonstrate the effectiveness of our method.

Remote sensing target detection algorithm based on perceptual extension and anchor frame best-fit matching

Aiming at the small imaging, complex background and crowded distribution of remote sensing image targets, a remote sensing image target detection algorithm (HQ-S2ANet) based on perceptual extension and anchor frame optimal matching is proposed by using the rotating target detection method S2ANet as a baseline network. Firstly, a cooperative attention(SEA) module is built to capture the relationship among the feature pixels when extending the model perception area to realize the relationship modeling between the target and the global. Secondly, the feature pyramid (FPN) feature fusion process is improved to form a perceptual extension feature pyramid module (HQFPN), which guarantees the low-level detail position information in the down sampling process when extending the perception area to enhance the model information capturing capability. Finally, a high-quality anchor frame is used to detect the target by using the high quality anchor frame as the baseline network. The high-quality anchor frame matching method (MaxIoUAssigner_HQ) is used to control the anchor frame truth value assignment by using a constant factor to ensure the recall rate while preventing the generation of low-quality anchor frame matching. The experimental results show that, under the DOTA dataset, the average accuracy(mAP) of HQ-S2ANet is improved by 3.1%, the parameters number increased by only 2.61M and the average recall(recall) is improved by 1.6% compared with the S2ANet algorithm, and the present algorithm effectively enhances the detection capability of the remote sensing image target.

Rotated Object Detection with Circular Gaussian Distribution

Rotated object detection is a challenging task due to the difficulties of locating the rotated objects and separating them effectively from the background. For rotated object prediction, researchers have explored numerous regression-based and classification-based approaches to predict a rotation angle. However, both paradigms are constrained by some flaws that make it difficult to accurately predict angles, such as multi-solution and boundary issues, which limits the performance upper bound of detectors. To address these issues, we propose a circular Gaussian distribution (CGD)-based method for angular prediction. We convert the labeled angle into a discrete circular Gaussian distribution spanning a single minimal positive period, and let the model predict the distribution parameters instead of directly regressing or classifying the angle. To improve the overall efficiency of the detection model, we also design a rotated object detector based on CenterNet. Experimental results on various public datasets demonstrated the effectiveness and superior performances of our method. In particular, our approach achieves better results than state-of-the-art competitors, with improvements of 1.92% and 1.04% in terms of AP points on the HRSC2016 and DOTA datasets, respectively.

ORCNN-X: Attention-Driven Multiscale Network for Detecting Small Objects in Complex Aerial Scenes

Currently, object detection on remote sensing images has drawn significant attention due to its extensive applications, including environmental monitoring, urban planning, and disaster assessment. However, detecting objects in the aerial images captured by remote sensors presents unique challenges compared to natural images, such as low resolution, complex backgrounds, and variations in scale and angle. Prior object detection algorithms are limited in their ability to identify oriented small objects, especially in aerial images where small objects are usually obscured by background noise. To address the above limitations, a novel framework (ORCNN-X) was proposed for oriented small object detection in remote sensing images by improving the Oriented RCNN. The framework adopts a multiscale feature extraction network (ResNeSt+) with a dynamic attention module (DCSA) and an effective feature fusion mechanism (W-PAFPN) to enhance the model’s perception ability and handle variations in scale and angle. The proposed framework is evaluated based on two public benchmark datasets, DOTA and HRSC2016. The experiments demonstrate its state-of-the-art performance in aspects of detection accuracy and speed. The presented model can also represent more objective spatial location information according to the feature visualization maps. Specifically, our model outperforms the baseline model by 1.43% mAP50 and 1.37% mAP12 on DOTA and HRSC2016 datasets, respectively.

CNTR-YOLO: Improved YOLOv5 Based on ConvNext and Transformer for Aircraft Detection in Remote Sensing Images

Aircraft detection in remote sensing images is an important branch of target detection due to the military value of aircraft. However, the diverse categories of aircraft and the intricate background of remote sensing images often lead to insufficient detection accuracy. Here, we present the CNTR-YOLO algorithm based on YOLOv5 as a solution to this issue. The CNTR-YOLO algorithm improves detection accuracy through three primary strategies. (1) We deploy DenseNet in the backbone to address the vanishing gradient problem during training and enhance the extraction of fundamental information. (2) The CBAM attention mechanism is integrated into the neck to minimize background noise interference. (3) The C3CNTR module is designed based on ConvNext and Transformer to clarify the target’s position in the feature map from both local and global perspectives. This module is applied before the prediction head to optimize the accuracy of prediction results. Our proposed algorithm is validated on the MAR20 and DOTA datasets. The results on the MAR20 dataset show that the mean average precision (mAP) of CNTR-YOLO reached 70.1%, which is a 3.3% improvement compared with YOLOv5l. On the DOTA dataset, the results indicate that the mAP of CNTR-YOLO reached 63.7%, which is 2.5% higher than YOLOv5l.

SOLARNet: A single stage regression based framework for efficient and robust object recognition in aerial images

Object recognition and localization play a crucial role in aerial images and their applications. The aerial images are challenging due to the large aspect ratio, arbitrary orientation, variation in scales, and non-uniform and cluttered object distribution. To address these challenges, we propose an efficient and robust model called the Simultaneous Object Localization and Recognition Network (SOLARNet), which is a fusion network that integrates two different sub-networks: PixelAttentionDetector (PD) and RotationDetector (RD). The PD considers features from different scales and cluttered objects, while RD handles rotation invariance, giving horizontal and oriented object detection results. The state-of-the-art model fails to improve accuracy when images are adversarially attacked. SOLARNet is not only efficient in terms of accuracy but also robust concerning to Fast Gradient Sign Method (FGSM), Basic Iterative Method (BIM), Jacobian Based Saliency Map (JSM) adversarial attacks, which is the crucial factor for any mission-critical system. We have performed experiments and achieved the accuracy on the publicly available DOTA dataset (75.00% mAP, 66.40% mAP) and DIOR dataset (88.60% mAP, 81.50% mAP) for horizontal and oriented object recognition tasks respectively while having high inference speed. Qualitative and quantitative results reported in this paper substantiate the superiority of SOLARNet over other state-of-the-art methodologies.

Gradient Structure Information-Guided Attention Generative Adversarial Networks for Remote Sensing Image Generation

A rich and effective dataset is an important foundation for the development of AI algorithms, and the quantity and quality of the dataset determine the upper limit level of the algorithms. For aerospace remote sensing datasets, due to the high cost of data collection and susceptibility to meteorological and airway conditions, the existing datasets have two problems: firstly, the number of datasets is obviously insufficient, and, secondly, there is large unevenness between different categories in datasets. One of the effective solutions is to use neural networks to generate fake data by learning from real data, but existing methods still find difficulty in generating remote sensing sample images with good texture detail and geometric distortion. To address the shortcomings of existing image generation algorithms, this paper proposes a gradient structure information-guided attention generative adversarial network (SGA-GAN) for remote sensing image generation, which contains two innovative initiatives: on the one hand, a learnable gradient structure information extraction branch network can be added to the generator network to obtain complex structural information in the sample image, thus alleviating the distortion of the sample geometric structure in remote sensing image generation; on the other hand, a multidimensional self-attention feature selection module is proposed to further improve the quality of the generated remote sensing images by connecting cross-attentive modules as well as spatial and channel attention modules in series to guide the generator to better utilize global information. The algorithm proposed in this paper outperformed other methods, such as StyleGAN-XL and FastGAN, in both the qualitative and quantitative evaluation, whereby the FID on the DOTA dataset decreased by 23.927 and the IS was improved by 2.351. The comparison experiments show that the method proposed in this paper can generate more realistic sample images, and images generated by this method can improve object detection metrics by increasing the number of single-category datasets and the number of targets in fewer categories in multi-category datasets, which means it can be effectively used in the field of intelligent processing of remote sensing images.

A remote sensing image rotation object detection approach for real-time environmental monitoring

For the problems of resources and environment faced in sustainable development, the use of remote sensing image object detection technology is an effective means to detect and analyze this major problem. Aiming at the problems of low accuracy and slow detection speed of existing remote sensing image rotation object detection algorithms implemented based on Transformer, a remote sensing image rotation object detection method based on dynamic position information Transformer is proposed. Firstly, to improve the detection accuracy, the cross-attention operation of the decoder is improved, and the obtained results are iteratively updated with the position information of the object queries and used as the initial object queries for the next decoder; secondly, to improve the robustness of the network for remote sensing image object detection, the data processing method of image pyramid is designed; Finally, a rotating IoU matching loss function more suitable for oriented object detection is introduced to improve the accuracy of matching the predicted boxed to the true boxed. The detection algorithm proposed in this paper is experimentally verified on DOTA and SSDD datasets, and the average detection accuracy is 73.70% and 90.3%, respectively, which effectively improves the average detection accuracy of Transformer-based rotating object detection algorithm in aerial remote sensing images and has better real-time detection performance.

Target detection for remote sensing based on the enhanced YOLOv4 with improved BiFPN

To solve problems for false detection, inadequate regression performance of anchor frames, and the inability to detect small targets in traditional multiscale target detection methods based on YOLOv4, we propose a novel target detection framework named as Enhanced YOLOv4. Firstly, our improved BiFPN replaced the original PANet as the feature fusion module, which can achieve multi-scale feature fusion by way of shared weights. Secondly, the channel attention mechanism (CAM) was embedded before the detection head to highlight the correlation between channels so that small targets can be get more attention. At last, to improve the anchor box regression effect and accelerate the training speed of YOLOv4, we improved the net training loss function, in which the original CIoU was replaced by CDIoU. The experimental results on the DOTA dataset validate our improvement. The mAP of our method is 90.88%, and the frame rate reached 58.76 FPS, at the same time, the speed of detection is not affected significantly.

Remote Sensing Image Segmentation Based on a Novel Gaussian Mixture Model and SURF Algorithm

This paper proposes a novel remote sensing image segmentation method based on Gaussian mixture model and SURF algorithm. Firstly, Gaussian mixture model is used for remote sensing image segmentation. Then the surf matching algorithm is adopted for eliminating misidentified areas. The determinant of Hession matrix (DoH) is used to detect key points in the image. The non-maximum suppression method and interpolation operation are utilized to search and locate the extreme points. The maximum likelihood method is used to estimate model parameters. Some remote sensing images in THE DOTA data set are selected for experimental verification, and the results show that the new algorithm has obvious improvement in segmentation effect and efficiency. In the background complex image segmentation, the improved algorithm has more obvious advantages compared than state-of-the-art segmentation methods.

Remote Sensing Image Object Detection Based on Improved YOLOv3 in Deep Learning Environment

A deep learning-based method, improved YOLOv3 algorithm is proposed in the deep learning environment to tackle challenges such as big scale, uneven distribution, largescale variation, and complicated background of small- and medium-sized remote sensing photos. This manufacture uses Densenet as the backbone, replacing Darknet-53 to realize feature reuse and make the feature extraction more effective; introduces the spatial pyramid pooling module into the feature pyramid part for increasing the receptive field and isolating the most prominent contextual features; adds SE attention module in the process of feature extraction and obtains richer features by learning more location information and channel information from the images. Under DOTA dataset, the final results are that the mean Average Precision value is 86.78%, which is 4.16% higher than the baseline YOLOv3 network. The model put forward makes it easier to extract information from the feature map and achieve higher detection accuracy without influencing the real-time performance of detection.

Object detection-oriented style transfer network for panchromatic remote sensing image

A panchromatic image is a remote sensing image that is imaged at the entire visible light band. It often has the highest spatial resolution and is widely used in the fields of resource detection, urban planning, geographic information systems, military, and national defense etc. However, the feature of single-band imaging determines that panchromatic images are usually displayed in the form of grayscale and result in some detailed differences between distinct types of ground objects that are indecipherable. It causes many difficulties for object detection applications. To address this problem, an object detection-oriented style transfer network for panchromatic remote sensing image is designed. In full consideration of the actual requirements of object detection tasks for panchromatic remote sensing image, a style transfer network based on feature fusion is designed, where a style transfer model is trained to transfer the grayscale panchromatic image into the corresponding color style. Further data preprocessing and postprocessing operations are designed to improve the quality of the transferred images and thus prevent negative transfer. DOTA dataset is used to verify the performance of the proposed algorithm. Results show that after the style transfer, the accuracy of object detection on the panchromatic remote sensing images has been significantly improved.

Recalibrating Features and Regression for Oriented Object Detection

The objects in remote sensing images are normally densely packed, arbitrarily oriented, and surrounded by complex backgrounds. Great efforts have been devoted to developing oriented object detection models to accommodate such data characteristics. We argue that an effective detection model hinges on three aspects: feature enhancement, feature decoupling for classification and localization, and an appropriate bounding box regression scheme. In this article, we instantiate the three aspects on top of the classical Faster R-CNN, with three novel components proposed. First, we propose a weighted fusion and refinement (WFR) module, which adaptively weighs multi-level features and leverages the attention mechanism to refine the fused features. Second, we decouple the RoI (region of interest) features for the subsequent classification and localization via a lightweight affine transformation-based feature decoupling (ATFD) module. Third, we propose a post-classification regression (PCR) module for generating the desired quadrilateral bounding boxes. Specifically, PCR predicts the precise vertex location on each side of a predicted horizontal box, by simply learning the following: (i) classify the discretized regression range of the vertex, and (ii) revise the vertex location with an offset. We conduct extensive experiments on the DOTA, DIOR-R, and HRSC2016 datasets to evaluate our method.

Adversarial patch attacks against aerial imagery object detectors

Although Deep Neural Networks (DNNs)-based object detectors are widely used in various fields, especially on aerial imagery object detections, it has been observed that a small elaborately designed patch attached to the images can mislead the DNNs-based detectors into producing erroneous output. However, the target detectors being attacked are quite simple, and the attack efficiency is relatively low in previous works, making it not practicable in real scenarios. To address these limitations, a new adversarial patch attack algorithm is proposed in this paper. Firstly, we designed a novel loss function using the intermediate outputs of the models rather than the model’s final outputs interpreted by the detection head to optimize adversarial patches. The experiments conducted on the DOTA, RSOD, and NWPU VHR-10 datasets demonstrate that our method can significantly degrade the performance of the detectors. Secondly, we conducted intensive experiments to investigate the impact of different outputs of the detection model on generating adversarial patches, demonstrating the class score is not as effective as the objectness score. Thirdly, we comprehensively analyzed the attack transferability across different aerial imagery datasets, verifying that the patches generated on one dataset are also effective in attacking another. Moreover, we proposed ensemble training to boost the attack’s transferability across models. Our work alarms the application of DNNs-based object detectors in aerial imagery.

Object Detection for UAV Aerial Scenarios Based on Vectorized IOU.

Object detection in unmanned aerial vehicle (UAV) images is an extremely challenging task and involves problems such as multi-scale objects, a high proportion of small objects, and high overlap between objects. To address these issues, first, we design a Vectorized Intersection Over Union (VIOU) loss based on YOLOv5s. This loss uses the width and height of the bounding box as a vector to construct a cosine function that corresponds to the size of the box and the aspect ratio and directly compares the center point value of the box to improve the accuracy of the bounding box regression. Second, we propose a Progressive Feature Fusion Network (PFFN) that addresses the issue of insufficient semantic extraction of shallow features by Panet. This allows each node of the network to fuse semantic information from deep layers with features from the current layer, thus significantly improving the detection ability of small objects in multi-scale scenes. Finally, we propose an Asymmetric Decoupled (AD) head, which separates the classification network from the regression network and improves the classification and regression capabilities of the network. Our proposed method results in significant improvements on two benchmark datasets compared to YOLOv5s. On the VisDrone 2019 dataset, the performance increased by 9.7% from 34.9% to 44.6%, and on the DOTA dataset, the performance increased by 2.1%.

SuperDet: An Efficient Single-Shot Network for Vehicle Detection in Remote Sensing Images

Vehicle detection in remote sensing images plays an important role for its wide range of applications. However, it is still a challenging task due to their small sizes. In this paper, we propose an efficient single-shot-based detector, called SuperDet, which achieves a combination of a super resolution algorithm with a deep convolutional neural network (DCNN)-based object detector. In SuperDet, there are two interconnected modules, namely, the super resolution module and the vehicle detection module. The super resolution module aims to recover a high resolution sensing image from its low resolution counterpart. With this module, the small vehicles will have a higher resolution, which is helpful for their detection. Taking the higher resolution image as input, the vehicle detection module extracts the features and predicts the location and category of the vehicles. We use a multi-task loss function to train the network in an end-to-end way. To assess the detection performance of SuperDet, we conducted experiments between SuperDet and the classical object detectors on both VEDAI and DOTA datasets. Experimental results indicate that SuperDet outperforms other detectors for vehicle detection in remote sensing images.

YOLOv4 with Deformable-Embedding-Transformer Feature Extractor for Exact Object Detection in Aerial Imagery.

The deep learning method for natural-image object detection tasks has made tremendous progress in recent decades. However, due to multiscale targets, complex backgrounds, and high-scale small targets, methods from the field of natural images frequently fail to produce satisfactory results when applied to aerial images. To address these problems, we proposed the DET-YOLO enhancement based on YOLOv4. Initially, we employed a vision transformer to acquire highly effective global information extraction capabilities. In the transformer, we proposed deformable embedding instead of linear embedding and a full convolution feedforward network (FCFN) instead of a feedforward network in order to reduce the feature loss caused by cutting in the embedding process and improve the spatial feature extraction capability. Second, for improved multiscale feature fusion in the neck, we employed a depth direction separable deformable pyramid module (DSDP) rather than a feature pyramid network. Experiments on the DOTA, RSOD, and UCAS-AOD datasets demonstrated that our method's average accuracy (mAP) values reached 0.728, 0.952, and 0.945, respectively, which were comparable to the existing state-of-the-art methods.

SRS-Net: Training object detectors from scratch for remote sensing images without pretraining

Most of the current object detection algorithms use pretrained models that are trained on ImageNet and then fine-tuned in the network, which can achieve good performance in terms of general object detectors. However, in the field of remote sensing image object detection, as pretrained models are significantly different from remote sensing data, it is meaningful to explore a train-from-scratch technique for remote sensing images. This paper proposes an object detection framework trained from scratch, SRS-Net, and describes the design of a densely connected backbone network to provide integrated hidden layer supervision for the convolution module. Then, two necessary improvement principles are proposed: studying the role of normalization in the network structure, and improving data augmentation methods for remote sensing images. To evaluate the proposed framework, we performed many ablation experiments on the DIOR, DOTA, and AS datasets. The results show that whether using the improved backbone network, the normalization method or training data enhancement strategy, the performance of the object detection network trained from scratch increased. These principles compensate for the lack of pretrained models. Furthermore, we found that SRS-Net could achieve similar to or slightly better performance than baseline methods, and surpassed most advanced general detectors.