Remote Sensing Object Detection Research Articles

A Refined and Efficient CNN Algorithm for Remote Sensing Object Detection

Remote sensing object detection (RSOD) plays a crucial role in resource utilization, geological disaster risk assessment and urban planning. Deep learning-based object-detection algorithms have proven effective in remote sensing image studies. However, accurate detection of objects with small size, dense distribution and complex object arrangement remains a significant challenge in the remote sensing field. To address this, a refined and efficient object-detection algorithm (RE-YOLO) has been proposed in this paper for remote sensing images. Initially, a refined and efficient module (REM) was designed to balance computational complexity and feature-extraction capabilities, which serves as a key component of the RE_CSP block. RE_CSP block efficiently extracts multi-scale information, overcoming challenges posed by complex backgrounds. Moreover, the spatial extracted attention module (SEAM) has been proposed in the bottleneck of backbone to promote representative feature learning and enhance the semantic information capture. In addition, a three-branch path aggregation network (TBPAN) has been constructed as the neck network, which facilitates comprehensive fusion of shallow positional information and deep semantic information across different channels, enabling the network with a robust ability to capture contextual information. Extensive experiments conducted on two large-scale remote sensing datasets, DOTA-v1.0 and SCERL, demonstrate that the proposed RE-YOLO outperforms state-of-the-art other object-detection approaches and exhibits a significant improvement in generalization ability.

SIA-Yolov5: improved Yolov5 based on smallness and imbalance-aware head for remote sensing object detection

SIA-Yolov5: improved Yolov5 based on smallness and imbalance-aware head for remote sensing object detection

Enhancing a You Only Look Once-Plated Detector via Auxiliary Textual Coding for Multi-Scale Rotating Remote Sensing Objects in Transportation Monitoring Applications

With the rapid development of intelligent information technologies, remote sensing object detection has played an important role in different field applications. Particularly in recent years, it has attracted widespread attention in assisting with food safety supervision, which still faces troubling issues between oversized parameters and low performance that are challenging to solve. Hence, this article proposes a novel remote sensing detection framework for multi-scale objects with a rotating status and mutual occlusion, defined as EYMR-Net. This proposed approach is established on the YOLO-v7 architecture with a Swin Transformer backbone, which offers multi-scale receptive fields to mine massive features. Then, an enhanced attention module is added to exploit the spatial and dimensional interrelationships among different local characteristics. Subsequently, the effective rotating frame regression mechanism via circular smoothing labels is introduced to the EYMR-Net structure, addressing the problem of horizontal YOLO (You Only Look Once) frames ignoring direction changes. Extensive experiments on DOTA datasets demonstrated the outstanding performance of EYMR-Net, which achieved an impressive mAP0.5 of up to 74.3%. Further ablation experiments verified that our proposed approach obtains a balance between performance and efficiency, which is beneficial for practical remote sensing applications in transportation monitoring and supply chain management.

SSN: Scale Selection Network for Multi-Scale Object Detection in Remote Sensing Images

The rapid growth of deep learning technology has made object detection in remote sensing images an important aspect of computer vision, finding applications in military surveillance, maritime rescue, and environmental monitoring. Nonetheless, the capture of remote sensing images at high altitudes causes significant scale variations, resulting in a heterogeneous range of object scales. These varying scales pose significant challenges for detection algorithms. To solve the scale variation problem, traditional detection algorithms compute multi-layer feature maps. However, this approach introduces significant computational redundancy. Inspired by the mechanism of cognitive scaling mechanisms handling multi-scale information, we propose a novel Scale Selection Network (SSN) to eliminate computational redundancy through scale attentional allocation. In particular, we have devised a lightweight Landmark Guided Scale Attention Network, which is capable of predicting potential scales in an image. The detector only needs to focus on the selected scale features, which greatly reduces the inference time. Additionally, a fast Reversible Scale Semantic Flow Preserving strategy is proposed to directly generate multi-scale feature maps for detection. Experiments demonstrate that our method facilitates the acceleration of image pyramid-based detectors by approximately 5.3 times on widely utilized remote sensing object detection benchmarks.

YOLO-Faster: An efficient remote sensing object detection method based on AMFFN.

As a pivotal task within computer vision, object detection finds application across a diverse spectrum of industrial scenarios. The advent of deep learning technologies has significantly elevated the accuracy of object detectors designed for general-purpose applications. Nevertheless, in contrast to conventional terrestrial environments, remote sensing object detection scenarios pose formidable challenges, including intricate and diverse backgrounds, fluctuating object scales, and pronounced interference from background noise, rendering remote sensing object detection an enduringly demanding task. In addition, despite the superior detection performance of deep learning-based object detection networks compared to traditional counterparts, their substantial parameter and computational demands curtail their feasibility for deployment on mobile devices equipped with low-power processors. In response to the aforementioned challenges, this paper introduces an enhanced lightweight remote sensing object detection network, denoted as YOLO-Faster, built upon the foundation of YOLOv5. Firstly, the lightweight design and inference speed of the object detection network is augmented by incorporating the lightweight network as the foundational network within YOLOv5, satisfying the demand for real-time detection on mobile devices. Moreover, to tackle the issue of detecting objects of different scales in large and complex backgrounds, an adaptive multiscale feature fusion network is introduced, which dynamically adjusts the large receptive field to capture dependencies among objects of different scales, enabling better modeling of object detection scenarios in remote sensing scenes. At last, the robustness of the object detection network under background noise is enhanced through incorporating a decoupled detection head that separates the classification and regression processes of the detection network. The results obtained from the public remote sensing object detection dataset DOTA show that the proposed method has a mean average precision of 71.4% and a detection speed of 38 frames per second.

DEDBNet: DoG-enhanced dual-branch object detection network for remote sensing object detection

With the improvement of spatial resolution of remote sensing images, object detection of remote sensing images has gradually become a difficult task. Extracted object features are usually hidden in a large amount of interference information in the background due to the complexity and large area of backgrounds, as well as the multi-scale nature of objects in remote sensing images. Still, many existing background weakening methods face difficulties in practical applications and are prone to high rates of false positives and false negatives. Therefore, remote sensing object detection has become increasingly challenging. To address these challenges, a novel background weakening method called Difference of Gaussian (DoG) to weaken background (DWB) module is proposed. Then, we develop a dual-branch network, named DoG-Enhanced Dual-Branch Object Detection Network (DEDBNet) for Remote Sensing Object Detection. The base branch network is responsible for detecting objects, while the DWB's branch network corrects the detected objects using feature-level attention. To combine the features of these branches, we propose two new methods Self-Mutual-Correcter with Detect heads (SMCD) for corrective learning and Map Channel Attention (MCA) for channel attention. Self-Corrector (SC) enables modification and integration of features, while the Mutual-Corrector (MC) enhances the features and further fuses them. We evaluate our proposed network, DEDBNet, through extensive experiments on four public datasets (DOTA with an mAP of 0.836, DIOR with an mAP of 0.871, NWPU VHR-10 with an mAP of 0.973, and RSOD with an mAP of 0.975). The results demonstrate that our method outperforms other state-of-the-art object detection methods significantly for remote sensing images.

DETR-ORD: An Improved DETR Detector for Oriented Remote Sensing Object Detection with Feature Reconstruction and Dynamic Query

Optical remote sensing images often feature high resolution, dense target distribution, and uneven target sizes, while transformer-based detectors like DETR reduce manually designed components, DETR does not support arbitrary-oriented object detection and suffers from high computational costs and slow convergence when handling large sequences of images. Additionally, bipartite graph matching and the limit on the number of queries result in transformer-based detectors performing poorly in scenarios with multiple objects and small object sizes. We propose an improved DETR detector for Oriented remote sensing object detection with Feature Reconstruction and Dynamic Query, termed DETR-ORD. It introduces rotation into the transformer architecture for oriented object detection, reduces computational cost with a hybrid encoder, and includes an IFR (image feature reconstruction) module to address the loss of positional information due to the flattening operation. It also uses ATSS to select auxiliary dynamic training queries for the decoder. This improved DETR-based detector enhances detection performance in challenging oriented optical remote sensing scenarios with similar backbone network parameters. Our approach achieves superior results on most optical remote sensing datasets, such as DOTA-v1.5 (72.07% mAP) and DIOR-R (66.60% mAP), surpassing the baseline detector.

SSFPN: Selective Spatial Feature Pyramid Network for Remote Sensing Object Detection

Abstract Object detection, particularly for oriented small objects, benefits greatly from the generation of multi-scale feature maps by the feature fusion module. Despite the success of series of Feature Pyramid Network models in general object detection, their application in remote sensing object detection has received limited attention. Small oriented objects require contextual information for detection, while various types of objects need different long-range context. In this paper, we propose an intuitive and simple fusion module to be added in Feature Pyramid Network called Selective Spatial Feature Pyramid Network (SSFPN) to address these challenges. SSFPN dynamically adjusts the spatial receptive field across multi-scale feature maps, enhancing the modeling of contextual variations among different objects in remote sensing scenarios. In extensive experiments, SSFPN has achieved competitive results, i.e., an improvement of 0.34% after adding SSFPN to the Oriented RCNN model. The codes are available at https://github.com/Atlantisming/SSFPN.

MSCANet: A multi-scale context-aware network for remote sensing object detection

MSCANet: A multi-scale context-aware network for remote sensing object detection

FFA: Foreground Feature Approximation Digitally against Remote Sensing Object Detection

FFA: Foreground Feature Approximation Digitally against Remote Sensing Object Detection

Enhancing Remote Sensing Object Detection with K-CBST YOLO: Integrating CBAM and Swin-Transformer

Enhancing Remote Sensing Object Detection with K-CBST YOLO: Integrating CBAM and Swin-Transformer

Direction-aware multi-branch attention and Gaussian label assignment for remote sensing aggregative object detection

ABSTRACT A large number of objects of different scales appear aggregative distribution in remote sensing images, which brings two challenges to remote sensing object detection. The first challenge arises from the presence of objects with large scale differences within the same scene, which complicates the detector’s ability to balance the detection performance between large and small objects. The second challenge is that the distribution of dense objects on remote sensing images with complex backgrounds can reduce the feature representation capability of the network, leading to false detections and missed detections of objects. To address the two challenges present in remote sensing object detection, we propose a new object detection network DA-GLANet (Direction-Aware Multi-Branch Attention and Gaussian Label Assignment Balance Net). For the first challenge, we designed a multi-scale feature fusion module to fully utilize semantic information at different scales, thereby improving the detection accuracy of multi-scale objects. Subsequently, to tackle the second challenge, we proposed a multi-branch attention module with direction-aware capabilities. This module can enhance the network’s feature representation of objects of interest by generating attention maps that are both direction-aware and position-sensitive. In addition, we propose a label assignment strategy based on Gaussian distribution to address the imbalance of positive and negative labels caused by the complexity of remote sensing images. These methods work synergistically to improve aggregative object detection accuracy. We have conducted extensive experiments on the DOTA dataset, Levir dataset, and WHU buildings change detection dataset to demonstrate the superiority of our method. On the DOTA dataset, our method improves mAP by 0.4%, 0.3%, and 0.2% compared to the top three methods. On the Levir dataset, our method improves mAP by 2.5%, 2.2%, and 1.6% compared to the top three methods.

BiF-DETR:Remote sensing object detection based on Bidirectional information fusion

Remote Sensing Object Detection(RSOD) is a fundamental task in the field of remote sensing image processing. The complexity of the background, the diversity of object scales and the locality limitation of Convolutional Neural Network (CNN) present specific challenges for RSOD. In this paper, an innovative hybrid detector, Bidirectional Information Fusion DEtection TRansformer (BiF-DETR), is proposed to mitigate the above issues. Specifically, BiF-DETR takes anchor-free detection network, CenterNet, as the baseline, designs the feature extraction backbone in parallel, extracts the local feature details using CNNs, and obtains the global information and long-term dependencies using Transformer branch. A Bidirectional Information Fusion (BIF) module is elaborately designed to reduce the semantic differences between different styles of feature maps through multi-level iterative information interactions, fully utilizing the complementary advantages of different detectors. Additionally, Coordination Attention(CA), is introduced to enables the detection network to focus on the saliency information of small objects. To address diversity insufficiency of remote sensing images in the training stage, Cascade Mixture Data Augmentation (CMDA), is designed to improve the robustness and generalization ability of the model. Comparative experiments with other cutting-edge methods are conducted on the publicly available DOTA and NWPU VHR-10 datasets. The experimental results reveal that the performance of proposed method is state-of-the-art, with mAP reaching 77.43% and 94.75%, respectively, far exceeding the other 25 competitive methods.

Remote sensing object detection with feature-associated convolutional neural networks

Neural networks have become integral to remote sensing data processing. Among neural networks, convolutional neural networks (CNNs) in deep learning offer numerous advanced algorithms for object detection in remote sensing imagery, which is pivotal in military and civilian contexts. CNNs excel in extracting features from training samples. However, traditional CNN models often lack specific signal assumptions tailored to remote sensing data at the feature level. In this paper, we propose a novel approach aimed at effectively representing and correlating information within CNNs for remote sensing object detection. We introduce object tokens and incorporate global information features in embedding layers, facilitating the comprehensive utilization of features across multiple hierarchical levels. Consideration of feature maps from images as two-dimensional signals, matrix image signal processing is employed to correlate features for diverse representations within the CNN framework. Moreover, hierarchical feature signals are effectively represented and associated during end-to-end network training. Experiments on various datasets demonstrate that the CNN model incorporating feature representation and association outperforms CNN models lacking these elements in object detection from remote sensing images. Additionally, integrating image signal processing enhances efficiency in end-to-end network training. Various signal processing approaches increase the process ability of the network, and the methodology could be transferred to other specific and well-defined task.

A visual knowledge oriented approach for weakly supervised remote sensing object detection

Weakly supervised learning poses significant challenges in remote sensing (RS) object detection due to the lack of precise instance annotations. This issue becomes particularly pronounced when dealing with complex backgrounds and dense target alignments in RS images. To address above limitations, we propose a visual knowledge oriented approach to leverage visual cues as pseudo labels, thereby enhancing the supervision for object detection. The visual knowledge is mainly explored from two perspectives: Firstly, recognizing that annotations are made solely at the image level, we address this limitation by aggregating objects of the same type across a group of images that share related semantic concepts. This approach allows us to infer instance-level annotations through collective knowledge. Secondly, due to the bird’s-eye view of RS images, certain object categories display distinctive visual patterns that are identifiable via expert knowledge. Specifically, with the multi-instance self-training framework as our base model, we establish the correlation among images sharing the same class labels, the co-saliency is utilized to extract the regions of common interests, thereby obtaining initial foregrounds in each image. Moreover, by leveraging the expert knowledge of class-specific visual patterns, we refine the pseudo labels and strength the foreground feature extraction by incorporating the low-level visual cues. To further stabilize the training process and address potential noise in object proposals, we incorporate a two-stage training strategy to refine initial predictions. We validate the effectiveness of our proposed approach on two benchmark datasets, i.e. NWPU VHR-10.v2 and DIOR, and achieve mAP of 84.25% and 27.5% on these datasets, respectively, which significantly outperform trending methods.

SREDet: Semantic-Driven Rotational Feature Enhancement for Oriented Object Detection in Remote Sensing Images

Significant progress has been achieved in the field of oriented object detection (OOD) in recent years. Compared to natural images, objects in remote sensing images exhibit characteristics of dense arrangement and arbitrary orientation while also containing a large amount of background information. Feature extraction in OOD becomes more challenging due to the diversity of object orientations. In this paper, we propose a semantic-driven rotational feature enhancement method, termed SREDet, to fully leverage the joint semantic and spatial information of oriented objects in the remote sensing images. We first construct a multi-rotation feature pyramid network (MRFPN), which leverages a fusion of multi-angle and multiscale feature maps to enhance the capability to extract features from different orientations. Then, considering feature confusion and contamination caused by the dense arrangement of objects and background interference, we present a semantic-driven feature enhancement module (SFEM), which decouples features in the spatial domain to separately enhance the features of objects and weaken those of backgrounds. Furthermore, we introduce an error source evaluation metric for rotated object detection to further analyze detection errors and indicate the effectiveness of our method. Extensive experiments demonstrate that our SREDet method achieves superior performance on two commonly used remote sensing object detection datasets (i.e., DOTA and HRSC2016).

OD-YOLO: Robust Small Object Detection Model in Remote Sensing Image with a Novel Multi-Scale Feature Fusion.

As remote sensing technology has advanced, the use of satellites and similar technologies has become increasingly prevalent in daily life. Now, it plays a crucial role in hydrology, agriculture, and geography. Nevertheless, because of the distinct qualities of remote sensing, including expansive scenes and small, densely packed targets, there are many challenges in detecting remote sensing objects. Those challenges lead to insufficient accuracy in remote sensing object detection. Consequently, developing a new model is essential to enhance the identification capabilities for objects in remote sensing imagery. To solve these constraints, we have designed the OD-YOLO approach that uses multi-scale feature fusion to improve the performance of the YOLOv8n model in small target detection. Firstly, traditional convolutions have poor recognition capabilities for certain geometric shapes. Therefore, in this paper, we introduce the Detection Refinement Module (DRmodule) into the backbone architecture. This module utilizes Deformable Convolutional Networks and the Hybrid Attention Transformer to strengthen the model's capability for feature extraction from geometric shapes and blurred objects effectively. Meanwhile, based on the Feature Pyramid Network of YOLO, at the head of the model framework, this paper enhances the detection capability by introducing a Dynamic Head to strengthen the fusion of different scales features in the feature pyramid. Additionally, to address the issue of detecting small objects in remote sensing images, this paper specifically designs the OIoU loss function to finely describe the difference between the detection box and the true box, further enhancing model performance. Experiments on the VisDrone dataset show that OD-YOLO surpasses the compared models by at least 5.2% in mAP50 and 4.4% in mAP75, and experiments on the Foggy Cityscapes dataset demonstrated that OD-YOLO improved mAP by 6.5%, demonstrating outstanding results in tasks related to remote sensing images and adverse weather object detection. This work not only advances the research in remote sensing image analysis, but also provides effective technical support for the practical deployment of future remote sensing applications.

Improving remote sensing object detection by using feature extraction and rotational equivariant attention

ABSTRACT In recent years, computer vision has witnessed significant attention in the research on object detection in remote-sensing images. Unlike objects in traditional natural images, those in remote sensing images are captured vertically by spacecraft, introducing arbitrary directionality, substantial scale variation, and a more complex background. We propose a rotation-equivariant detector enhanced with feature fusion and attention modules to address remote sensing image object detection challenges. Specifically, we introduce a Rotation-Enhanced Feature Extraction (REFE) module and a Rotational Equivariant Attention (REA) module. These enhancements empower the detector to extract object information more effectively from remotely sensed images, filtering out complex background information and improving detection accuracy and stability. Through extensive experiments on diverse and challenging remote sensing image datasets, our method outperforms the task of object detection. Remarkably, our network achieves 1.0, 2.91, and 1.11 mean average precision (mAP) improvements on the DOTA-v1.5, HRSC2016 and DIOR-R datasets. These experimental results robustly demonstrate the effectiveness and superiority of our proposed method.

C3TB-YOLOv5: integrated YOLOv5 with transformer for object detection in high-resolution remote sensing images

ABSTRACT In the realm of object detection from high-resolution remote sensing images (HRRSIs), the existing YOLOv5 methods encounter several challenges, including dense object arrangements, small object sizes, and complex backgrounds. To tackle these challenges, we propose a novel approach called C3TB-YOLOv5, which combines traditional YOLOv5 with the Transformer model to detect objects in HRRSIs. Unlike conventional YOLOv5 methods that primarily focus on capturing local information from remote sensing scenes, our C3TB-YOLOv5 method incorporates global information through the introduction of a new C3TB module. This module, based on the Transformer multi-head attention mechanism (AM), consists of two branches that extract local and global information from feature maps. By integrating these branches and establishing long-range relationships, our method successfully detects densely arranged small objects in HRRSIs. Furthermore, to improve the accuracy of tiny object detection, a novel detection head has been developed to effectively utilize the unused C3 module, thereby preventing the loss of fine-grained textures and positional features. In addition, we integrate an enhanced SimAM, namely Sim-GMP, into the model to adjust the focus across varying regions, effectively distinguishing the features of interested objects from complex backgrounds. Finally, to address the problem of sample imbalance in remote sensing object detection, the most recent Wise-IoU v3 loss function is employed to improve the accuracy of anchor box predictions for objects. To maintain a high object detection speed, the most critical C3 modules are substituted with the proposed C3TB module for the purpose of striking a good balance between object detection accuracy and model lightweight. Extensive experiments conducted on two remote sensing datasets of NWPU VHR-10 and VisDrone 2019 demonstrates that our method achieves superior object detection performance than state-of-the-art methods.

Semi-supervised Open-World Object Detection

Conventional open-world object detection (OWOD) problem setting first distinguishes known and unknown classes and then later incrementally learns the unknown objects when introduced with labels in the subsequent tasks. However, the current OWOD formulation heavily relies on the external human oracle for knowledge input during the incremental learning stages. Such reliance on run-time makes this formulation less realistic in a real-world deployment. To address this, we introduce a more realistic formulation, named semi-supervised open-world detection (SS-OWOD), that reduces the annotation cost by casting the incremental learning stages of OWOD in a semi-supervised manner. We demonstrate that the performance of the state-of-the-art OWOD detector dramatically deteriorates in the proposed SS-OWOD setting. Therefore, we introduce a novel SS-OWOD detector, named SS-OWFormer, that utilizes a feature-alignment scheme to better align the object query representations between the original and augmented images to leverage the large unlabeled and few labeled data. We further introduce a pseudo-labeling scheme for unknown detection that exploits the inherent capability of decoder object queries to capture object-specific information. On the COCO dataset, our SS-OWFormer using only 50% of the labeled data achieves detection performance that is on par with the state-of-the-art (SOTA) OWOD detector using all the 100% of labeled data. Further, our SS-OWFormer achieves an absolute gain of 4.8% in unknown recall over the SOTA OWOD detector. Lastly, we demonstrate the effectiveness of our SS-OWOD problem setting and approach for remote sensing object detection, proposing carefully curated splits and baseline performance evaluations. Our experiments on 4 datasets including MS COCO, PASCAL, Objects365 and DOTA demonstrate the effectiveness of our approach. Our source code, models and splits are available here https://github.com/sahalshajim/SS-OWFormer