Remote Sensing Object Detection Research Articles

Improving remote sensing object detection by using feature extraction and rotational equivariant attention

ABSTRACT In recent years, computer vision has witnessed significant attention in the research on object detection in remote-sensing images. Unlike objects in traditional natural images, those in remote sensing images are captured vertically by spacecraft, introducing arbitrary directionality, substantial scale variation, and a more complex background. We propose a rotation-equivariant detector enhanced with feature fusion and attention modules to address remote sensing image object detection challenges. Specifically, we introduce a Rotation-Enhanced Feature Extraction (REFE) module and a Rotational Equivariant Attention (REA) module. These enhancements empower the detector to extract object information more effectively from remotely sensed images, filtering out complex background information and improving detection accuracy and stability. Through extensive experiments on diverse and challenging remote sensing image datasets, our method outperforms the task of object detection. Remarkably, our network achieves 1.0, 2.91, and 1.11 mean average precision (mAP) improvements on the DOTA-v1.5, HRSC2016 and DIOR-R datasets. These experimental results robustly demonstrate the effectiveness and superiority of our proposed method.

C3TB-YOLOv5: integrated YOLOv5 with transformer for object detection in high-resolution remote sensing images

ABSTRACT In the realm of object detection from high-resolution remote sensing images (HRRSIs), the existing YOLOv5 methods encounter several challenges, including dense object arrangements, small object sizes, and complex backgrounds. To tackle these challenges, we propose a novel approach called C3TB-YOLOv5, which combines traditional YOLOv5 with the Transformer model to detect objects in HRRSIs. Unlike conventional YOLOv5 methods that primarily focus on capturing local information from remote sensing scenes, our C3TB-YOLOv5 method incorporates global information through the introduction of a new C3TB module. This module, based on the Transformer multi-head attention mechanism (AM), consists of two branches that extract local and global information from feature maps. By integrating these branches and establishing long-range relationships, our method successfully detects densely arranged small objects in HRRSIs. Furthermore, to improve the accuracy of tiny object detection, a novel detection head has been developed to effectively utilize the unused C3 module, thereby preventing the loss of fine-grained textures and positional features. In addition, we integrate an enhanced SimAM, namely Sim-GMP, into the model to adjust the focus across varying regions, effectively distinguishing the features of interested objects from complex backgrounds. Finally, to address the problem of sample imbalance in remote sensing object detection, the most recent Wise-IoU v3 loss function is employed to improve the accuracy of anchor box predictions for objects. To maintain a high object detection speed, the most critical C3 modules are substituted with the proposed C3TB module for the purpose of striking a good balance between object detection accuracy and model lightweight. Extensive experiments conducted on two remote sensing datasets of NWPU VHR-10 and VisDrone 2019 demonstrates that our method achieves superior object detection performance than state-of-the-art methods.

Semi-supervised Open-World Object Detection

Conventional open-world object detection (OWOD) problem setting first distinguishes known and unknown classes and then later incrementally learns the unknown objects when introduced with labels in the subsequent tasks. However, the current OWOD formulation heavily relies on the external human oracle for knowledge input during the incremental learning stages. Such reliance on run-time makes this formulation less realistic in a real-world deployment. To address this, we introduce a more realistic formulation, named semi-supervised open-world detection (SS-OWOD), that reduces the annotation cost by casting the incremental learning stages of OWOD in a semi-supervised manner. We demonstrate that the performance of the state-of-the-art OWOD detector dramatically deteriorates in the proposed SS-OWOD setting. Therefore, we introduce a novel SS-OWOD detector, named SS-OWFormer, that utilizes a feature-alignment scheme to better align the object query representations between the original and augmented images to leverage the large unlabeled and few labeled data. We further introduce a pseudo-labeling scheme for unknown detection that exploits the inherent capability of decoder object queries to capture object-specific information. On the COCO dataset, our SS-OWFormer using only 50% of the labeled data achieves detection performance that is on par with the state-of-the-art (SOTA) OWOD detector using all the 100% of labeled data. Further, our SS-OWFormer achieves an absolute gain of 4.8% in unknown recall over the SOTA OWOD detector. Lastly, we demonstrate the effectiveness of our SS-OWOD problem setting and approach for remote sensing object detection, proposing carefully curated splits and baseline performance evaluations. Our experiments on 4 datasets including MS COCO, PASCAL, Objects365 and DOTA demonstrate the effectiveness of our approach. Our source code, models and splits are available here https://github.com/sahalshajim/SS-OWFormer

Object Detection in Remote Sensing Images Based on Adaptive Multi-Scale Feature Fusion Method

Multi-scale object detection is critical for analyzing remote sensing images. Traditional feature pyramid networks, which are aimed at accommodating objects of varying sizes through multi-level feature extraction, face significant challenges due to the diverse scale variations present in remote sensing images. This situation often forces single-level features to span a broad spectrum of object sizes, complicating accurate localization and classification. To tackle these challenges, this paper proposes an innovative algorithm that incorporates an adaptive multi-scale feature enhancement and fusion module (ASEM), which enhances remote sensing image object detection through sophisticated multi-scale feature fusion. Our method begins by employing a feature pyramid to gather coarse multi-scale features. Subsequently, it integrates a fine-grained feature extraction module at each level, utilizing atrous convolutions with varied dilation rates to refine multi-scale features, which markedly improves the information capture from widely varied object scales. Furthermore, an adaptive enhancement module is applied to the features of each level by employing an attention mechanism for feature fusion. This strategy concentrates on the features of critical scale, which significantly enhance the effectiveness of capturing essential feature information. Compared with the baseline method, namely, Rotated FasterRCNN, our method achieved an mAP of 74.21% ( 0.81%) on the DOTA-v1.0 dataset and an mAP of 84.90% (+9.2%) on the HRSC2016 dataset. These results validated the effectiveness and practicality of our method and demonstrated its significant application value in multi-scale remote sensing object detection tasks.

A Lightweight Remote Sensing Aircraft Object Detection Network Based on Improved YOLOv5n

Due to the issues of remote sensing object detection algorithms based on deep learning, such as a high number of network parameters, large model size, and high computational requirements, it is challenging to deploy them on small mobile devices. This paper proposes an extremely lightweight remote sensing aircraft object detection network based on the improved YOLOv5n. This network combines Shufflenet v2 and YOLOv5n, significantly reducing the network size while ensuring high detection accuracy. It substitutes the original CIoU and convolution with EIoU and deformable convolution, optimizing for the small-scale characteristics of aircraft objects and further accelerating convergence and improving regression accuracy. Additionally, a coordinate attention (CA) mechanism is introduced at the end of the backbone to focus on orientation perception and positional information. We conducted a series of experiments, comparing our method with networks like GhostNet, PP-LCNet, MobileNetV3, and MobileNetV3s, and performed detailed ablation studies. The experimental results on the Mar20 public dataset indicate that, compared to the original YOLOv5n network, our lightweight network has only about one-fifth of its parameter count, with only a slight decrease of 2.7% in mAP@0.5. At the same time, compared with other lightweight networks of the same magnitude, our network achieves an effective balance between detection accuracy and resource consumption such as memory and computing power, providing a novel solution for the implementation and hardware deployment of lightweight remote sensing object detection networks.

OII: An Orientation Information Integrating Network for Oriented Object Detection in Remote Sensing Images

Oriented object detection for remote sensing images poses formidable challenges due to arbitrary orientation, diverse scales, and densely distributed targets (e.g., across terrain). Current investigations in remote sensing object detection have primarily focused on improving the representation of oriented bounding boxes yet have neglected the significant orientation information of targets in remote sensing contexts. Recent investigations point out that the inclusion and fusion of orientation information yields substantial benefits in training an accurate oriented object system. In this paper, we propose a simple but effective orientation information integrating (OII) network comprising two main parts: the orientation information highlighting (OIH) module and orientation feature fusion (OFF) module. The OIH module extracts orientation features from those produced by the backbone by modeling the frequency information of spatial features. Given that low-frequency components in an image capture its primary content, and high-frequency components contribute to its intricate details and edges, the transformation from the spatial domain to the frequency domain can effectively emphasize the orientation information of images. Subsequently, our OFF module employs a combination of a CNN attention mechanism and self-attention to derive weights for orientation features and original features. These derived weights are adopted to adaptively enhance the original features, resulting in integrated features that contain enriched orientation information. Given the inherent limitation of the original spatial attention weights in explicitly capturing orientation nuances, the incorporation of the introduced orientation weights serves as a pivotal tool to accentuate and delineate orientation information related to targets. Without unnecessary embellishments, our OII network achieves competitive detection accuracy on two prevalent remote sensing-oriented object detection datasets: DOTA (80.82 mAP) and HRSC2016 (98.32 mAP).

Remote Sensing Micro-Object Detection under Global and Local Attention Mechanism

With the rapid advancement of technology, satellite and drone technologies have had significant impacts on various fields, creating both opportunities and challenges. In areas like the military, urban planning, and environmental monitoring, the application of remote sensing technology is paramount. However, due to the unique characteristics of remote sensing images, such as high resolution, large-scale scenes, and small, densely packed targets, remote sensing object detection faces numerous technical challenges. Traditional detection methods are inadequate for effectively detecting small targets, rendering the accurate and efficient detection of objects in complex remote sensing images a pressing issue. Current detection techniques fall short in accurately detecting small targets compared to medium and large ones, primarily due to limited feature information, insufficient contextual data, and poor localization capabilities for small targets. In response, we propose an innovative detection method. Unlike previous approaches that often focused solely on either local or contextual information, we introduce a novel Global and Local Attention Mechanism (GAL), providing an in-depth modeling method for input images. Our method integrates fine-grained local feature analysis with global contextual information processing. The local attention concentrates on details and spatial relationships within local windows, enabling the model to recognize intricate details in complex images. Meanwhile, the global attention addresses the entire image’s global information, capturing overarching patterns and structures, thus enhancing the model’s high-level semantic understanding. Ultimately, a specific mechanism fuses local details with global context, allowing the model to consider both aspects for a more precise and comprehensive interpretation of images. Furthermore, we have developed a multi-head prediction module that leverages semantic information at various scales to capture the multi-scale characteristics of remote sensing targets. Adding decoupled prediction heads aims to improve the accuracy and robustness of target detection. Additionally, we have innovatively designed the Ziou loss function, an advanced loss calculation, to enhance the model’s precision in small target localization, thereby boosting its overall performance in small target detection. Experimental results on the Visdrone2019 and DOTA datasets demonstrate that our method significantly surpasses traditional methods in detecting small targets in remote sensing imagery.

Adaptive multimodal feature fusion with frequency domain gate for remote sensing object detection

ABSTRACT Fusing complementary information of visible and infrared radiation modalities can improve object detection performance for unmanned aerial vehicle (UAV) remote sensing images under insufficient illumination conditions. Although previous works have conducted some studies in this field, they have rarely considered the adaptive ability of multimodal feature fusion, which limits the performance improvement space for multimodal detectors. To this end, we propose an adaptive multimodal feature fusion method with a frequency domain gate based on DINO (detection transformer with improved denoising anchor boxes), called multimodal DINO. In our approach, a multimodal feature encoder with underlying feature sharing is designed, which efficiently extracts common and differential features through RGB-guided infrared radiation data transformation. Additionally, an adaptive frequency domain gate is introduced to dynamically learn the degree of dependence on frequency-filtered features of each modality when processing different samples. We evaluate the proposed method on the two multimodal detection remote sensing image datasets, VEDAI and DroneVehicle. Extensive experiments demonstrate that our approach achieves superior performance compared to basic detectors and existing multimodal detection methods. Our code is available at https://github.com/cq100/multimodalDINO.

SGR: An Improved Point-Based Method for Remote Sensing Object Detection via Dual-Domain Alignment Saliency-Guided RepPoints

With the advancement of deep neural networks, several methods leveraging convolution neural networks (CNNs) have gained prominence in the field of remote sensing object detection. Acquiring accurate feature representations from feature maps is a critical step in CNN-based object detection methods. Previously, region of interest (RoI)-based methods have been widely used, but of late, deformable convolution network (DCN)-based approaches have started receiving considerable attention. A significant challenge in the use of DCN-based methods is the inefficient distribution patterns of sampling points, stemming from a lack of effective and flexible guidance. To address this, our study introduces Saliency-Guided RepPoints (SGR), an innovative framework designed to enhance feature representation quality in remote sensing object detection. SGR employs a dynamic dual-domain alignment (DDA) training strategy to mitigate potential misalignment issues between spatial and feature domains during the learning process. Furthermore, we propose an interpretable visualization method to assess the alignment between feature representation and classification performance in DCN-based methods, providing theoretical analysis and validation for the effectiveness of sampling points. In this study, we assessed the proposed SGR framework through a series of experiments conducted on four varied and rigorous datasets: DOTA, HRSC2016, DIOR-R, and UCAS-AOD, all of which are widely employed in remote sensing object detection. The outcomes of these experiments substantiate the effectiveness of the SGR framework, underscoring its potential to enhance the accuracy of object detection within remote sensing imagery.

A Multiscale Information Fusion Network Based on PixelShuffle Integrated With YOLO for Aerial Remote Sensing Object Detection

A Multiscale Information Fusion Network Based on PixelShuffle Integrated With YOLO for Aerial Remote Sensing Object Detection

A Self-Supplementary and Revised Network for Remote Sensing Object Detection

A Self-Supplementary and Revised Network for Remote Sensing Object Detection

Parallel Space and Channel Attention for Stronger Remote Sensing Object Detection

Parallel Space and Channel Attention for Stronger Remote Sensing Object Detection

SPGC: Shape-Prior-Based Generated Content Data Augmentation for Remote Sensing Object Detection

SPGC: Shape-Prior-Based Generated Content Data Augmentation for Remote Sensing Object Detection

Relational Part-Aware Learning for Complex Composite Object Detection in High-Resolution Remote Sensing Images.

In high-resolution remote sensing images (RSIs), complex composite object detection (e.g., coal-fired power plant detection and harbor detection) is challenging due to multiple discrete parts with variable layouts leading to complex weak inter-relationship and blurred boundaries, instead of a clearly defined single object. To address this issue, this article proposes an end-to-end framework, i.e., relational part-aware network (REPAN), to explore the semantic correlation and extract discriminative features among multiple parts. Specifically, we first design a part region proposal network (P-RPN) to locate discriminative yet subtle regions. With butterfly units (BFUs) embedded, feature-scale confusion problems stemming from aliasing effects can be largely alleviated. Second, a feature relation Transformer (FRT) plumbs the depths of the spatial relationships by part-and-global joint learning, exploring correlations between various parts to enhance significant part representation. Finally, a contextual detector (CD) classifies and detects parts and the whole composite object through multirelation-aware features, where part information guides to locate the whole object. We collect three remote sensing object detection datasets with four categories to evaluate our method. Consistently surpassing the performance of state-of-the-art methods, the results of extensive experiments underscore the effectiveness and superiority of our proposed method.

Learning Remote Sensing Object Detection With Single Point Supervision

Learning Remote Sensing Object Detection With Single Point Supervision

A robust training method for object detectors in remote sensing image

With the development of convolutional neural networks (CNNs), remote sensing object detection has been made a great improvement. The CNNs-based detectors rely on accurate manually labeled training data. Due to the characteristics of remote sensing images and the professional requirements of annotation, the quality of data annotation is difficult to guarantee, and the labels of data will inevitably introduce noise. Several approaches have been developed to learn robust models from noisy labels, but most of them have focused on classification tasks. Training object detectors with noisy remote sensing data has been less investigated. In this article, we propose an object Co-teaching training strategy to train robust object detectors from noisy labels. Specifically, we train two detectors in a parallel manner, and let them teach each other to filter noise object instances in every given mini-batch. We also introduce a weighting factor to the object Co-teaching (named Reweighting object Co-teaching) which improves the performance of the detector on clean datasets without adding any complexity of the detector. Extensive experiments on DIOR and RSOD datasets demonstrate the effectiveness of the proposed method. Under 0.5 level noisy label, the object Co-teaching improves 7.63% and 9.57% mAP compared to the Baseline on the two datasets, respectively. Under clean label, the proposed Reweighting object Co-teaching improves 0.78% and 2.60% mAP on the two datasets without adding any complexity to the detector.

YOLO-SE: Improved YOLOv8 for Remote Sensing Object Detection and Recognition

Object detection remains a pivotal aspect of remote sensing image analysis, and recent strides in Earth observation technology coupled with convolutional neural networks (CNNs) have propelled the field forward. Despite advancements, challenges persist, especially in detecting objects across diverse scales and pinpointing small-sized targets. This paper introduces YOLO-SE, a novel YOLOv8-based network that innovatively addresses these challenges. First, the introduction of a lightweight convolution SEConv in lieu of standard convolutions reduces the network’s parameter count, thereby expediting the detection process. To tackle multi-scale object detection, the paper proposes the SEF module, an enhancement based on SEConv. Second, an ingenious Efficient Multi-Scale Attention (EMA) mechanism is integrated into the network, forming the SPPFE module. This addition augments the network’s feature extraction capabilities, adeptly handling challenges in multi-scale object detection. Furthermore, a dedicated prediction head for tiny object detection is incorporated, and the original detection head is replaced by a transformer prediction head. To address adverse gradients stemming from low-quality instances in the target detection training dataset, the paper introduces the Wise-IoU bounding box loss function. YOLO-SE showcases remarkable performance, achieving an average precision at IoU threshold 0.5 (AP50) of 86.5% on the optical remote sensing dataset SIMD. This represents a noteworthy 2.1% improvement over YOLOv8 and YOLO-SE outperforms the state-of-the-art model by 0.91%. In further validation, experiments on the NWPU VHR-10 dataset demonstrated YOLO-SE’s superiority with an accuracy of 94.9%, surpassing that of YOLOv8 by 2.6%. The proposed advancements position YOLO-SE as a compelling solution in the realm of deep learning-based remote sensing image object detection.

Remote Sensing Object Detection Meets Deep Learning: A metareview of challenges and advances

Remote Sensing Object Detection Meets Deep Learning: A metareview of challenges and advances

MSA R-CNN: A comprehensive approach to remote sensing object detection and scene understanding

Remote sensing scene understanding is crucial for extracting valuable information from high-resolution images, including object detection and classification. Traditional object detection methods face challenges in handling the diverse scales, orientations, and complex backgrounds present in remote sensing data. In this paper, we propose a novel remote sensing scene understanding system called multiscale-attention R-CNN (MSA R-CNN), which incorporates a super multiscale feature extraction network (SMENet) for enhanced feature extraction from multiscale images, an adaptive dynamic inner lateral (ADIL) connection module to tackle information loss in feature pyramid networks (FPN), and a distributed lightweight attention module (DLAM) to refine feature information processing. Furthermore, a new dataset combining the DIOR and DOTA datasets is introduced to extract the background information of detected objects and evaluate the proposed system’s performance. MSA R-CNN achieved an mAP of 74.37% on the DIOR dataset when the gamma value was set to 0.2 and 81.97% on the DOTA dataset when the gamma value was set to 0.1 with the same learning rate, outperforming state-of-the-art models on both datasets. The proposed system demonstrates significant improvements in both object detection and background information extraction, providing a comprehensive solution for remote sensing scene understanding.

HRRSODN: high-Resolution remote sensing object detection network fusion of road-adjacent relationships

Application of deep learning technology in remote sensing image target detection(RSITD)has made remarkable progress in recent years. However, existing methods usually fuse spatial context information at the pixel level while ignoring the mining of spatial relationships between ground objects. This paper proposes a high-resolution remote sensing object detection network fusion of road-adjacent relationships (HRRSODN) that integrates road proximity relations. Taking artificial features such as toll stations adjacent to roads as an example, the spatial relationship of road proximity is used to assist visual information and improve the extraction accuracy. The method optimizes detection by: Using the spatial relationship as prior information; fusing coordinate attention that captures the spatial relationship between objects; and adding the attention generated by the spatial relationship to the attention generated in the original X and Y directions. This enables the model to understand the spatial layout of objects in the image. The distance loss function concretizes the spatial relationship into a distance index. The distance loss is incorporated into the training process by measuring the spatial distance between feature targets. Thus, during training, the network focuses not only on the positional information of the targets but also pays more attention to the spatial layout of the features. Experimental results show that, compared with conventional methods, the proposed method has achieved significant improvements in detection performance and mining spatial relationships. This verifies the importance of considering the spatial relationship of ground objects in remote sensing target detection. Further, this paper proposes a new method for target detection in remote sensing images (RSIs), which effectively utilizes the spatial relationship information between targets, and provides a new idea for further improving target detection in RSI.