Object Detection In Remote Sensing Images Research Articles

Improving remote sensing object detection by using feature extraction and rotational equivariant attention

ABSTRACT In recent years, computer vision has witnessed significant attention in the research on object detection in remote-sensing images. Unlike objects in traditional natural images, those in remote sensing images are captured vertically by spacecraft, introducing arbitrary directionality, substantial scale variation, and a more complex background. We propose a rotation-equivariant detector enhanced with feature fusion and attention modules to address remote sensing image object detection challenges. Specifically, we introduce a Rotation-Enhanced Feature Extraction (REFE) module and a Rotational Equivariant Attention (REA) module. These enhancements empower the detector to extract object information more effectively from remotely sensed images, filtering out complex background information and improving detection accuracy and stability. Through extensive experiments on diverse and challenging remote sensing image datasets, our method outperforms the task of object detection. Remarkably, our network achieves 1.0, 2.91, and 1.11 mean average precision (mAP) improvements on the DOTA-v1.5, HRSC2016 and DIOR-R datasets. These experimental results robustly demonstrate the effectiveness and superiority of our proposed method.

Research on the Multiple Small Target Detection Methodology in Remote Sensing.

This study focuses on advancing the field of remote sensing image target detection, addressing challenges such as small target detection, complex background handling, and dense target distribution. We propose solutions based on enhancing the YOLOv7 algorithm. Firstly, we improve the multi-scale feature enhancement (MFE) method of YOLOv7, enhancing its adaptability and precision in detecting small targets and complex backgrounds. Secondly, we design a modified YOLOv7 global information DP-MLP module to effectively capture and integrate global information, thereby improving target detection accuracy and robustness, especially in handling large-scale variations and complex scenes. Lastly, we explore a semi-supervised learning model (SSLM) target detection algorithm incorporating unlabeled data, leveraging information from unlabeled data to enhance the model's generalization ability and performance. Experimental results demonstrate that despite the outstanding performance of YOLOv7, the mean average precision (MAP) can still be improved by 1.9%. Specifically, under testing on the TGRS-HRRSD-Dataset, the MFE and DP-MLP models achieve MAP values of 93.4% and 93.1%, respectively. Across the NWPU VHR-10 dataset, the three models achieve MAP values of 93.1%, 92.1%, and 92.2%, respectively. Significant improvements are observed across various metrics compared to the original model. This study enhances the adaptability, accuracy, and generalization of remote sensing image object detection.

Discretization and decoupled knowledge distillation for arbitrary oriented object detection

In recent years, lightweight object detection networks have been increasingly applied to remote sensing platforms due to their fast inference speed and flexible deployment advantages. Knowledge distillation methods have been widely used to reduce the performance gap between large and small models, and many studies have combined knowledge distillation with object detection. However, existing knowledge distillation methods often overlook the transfer of localization knowledge. Therefore, this paper proposes a method called Discretized Position Knowledge Distillation (DPKD) to improve the use of knowledge distillation in object detection. Specifically, the DPKD method incorporates a Discretization Algorithm Module (DAM), which leverages both general probability distribution and cross-Gaussian distribution to transfer high-quality bounding box position and pose information. Additionally, the Position Knowledge Distillation (PKD) method splits the target and non-target bounding boxes to form the distillation loss function, addressing the issue of missing background knowledge transfer during the distillation process. To further enhance the learning of high-quality bounding boxes, a Region Weighting Module (RWM) based on EIoU is introduced in DPKD, assigning weights to the various bounding boxes in the teacher's output. The effectiveness of DPKD in the field of remote sensing image object detection in multi-modal scenarios was verified through multi-modal training on the publicly available DOTA and HRSID datasets.

Object Detection in Remote Sensing Images Based on Adaptive Multi-Scale Feature Fusion Method

Multi-scale object detection is critical for analyzing remote sensing images. Traditional feature pyramid networks, which are aimed at accommodating objects of varying sizes through multi-level feature extraction, face significant challenges due to the diverse scale variations present in remote sensing images. This situation often forces single-level features to span a broad spectrum of object sizes, complicating accurate localization and classification. To tackle these challenges, this paper proposes an innovative algorithm that incorporates an adaptive multi-scale feature enhancement and fusion module (ASEM), which enhances remote sensing image object detection through sophisticated multi-scale feature fusion. Our method begins by employing a feature pyramid to gather coarse multi-scale features. Subsequently, it integrates a fine-grained feature extraction module at each level, utilizing atrous convolutions with varied dilation rates to refine multi-scale features, which markedly improves the information capture from widely varied object scales. Furthermore, an adaptive enhancement module is applied to the features of each level by employing an attention mechanism for feature fusion. This strategy concentrates on the features of critical scale, which significantly enhance the effectiveness of capturing essential feature information. Compared with the baseline method, namely, Rotated FasterRCNN, our method achieved an mAP of 74.21% ( 0.81%) on the DOTA-v1.0 dataset and an mAP of 84.90% (+9.2%) on the HRSC2016 dataset. These results validated the effectiveness and practicality of our method and demonstrated its significant application value in multi-scale remote sensing object detection tasks.

Fast fourier convolution based remote sensing image object detection for Earth observation

Fast fourier convolution based remote sensing image object detection for Earth observation

Few‐shot object detection based on global context and implicit knowledge decoupled head

AbstractThe acquisition cycle of remote sensing images is slow, and the labelling process encounters challenges, which have become prominent with the rapid development of remote sensing image object detection research. Therefore, this article provides a way to make the model better capture the diversity and contextual relationships in the data, and solve this problem by more than just data augmentation. Specifically, this method is a few‐shot object detection method for remote sensing based on global context combined with implicit knowledge decoupled head (GC‐IKDH). This method first uses a segmentation strategy to convert high‐resolution images into low‐resolution images and expands the sample size through a generative model. Secondly, GC attention is introduced to generate a GC vector by weighting and averaging the information of each position in the input sequence, which helps the model better understand the semantics of the input sequence. Finally, an IKDH is added to improve the model head, which is used to learn specific features in the data so that the model can better handle the diversity in the data. Experimental results show that GC attention and IKDH boosting provide a good performance boost to the baseline model. Compared with other few‐shot samples, this method achieves state‐of‐the‐art performance under different shot settings and highly competitive results on two benchmark datasets (NWPU VHR‐10 and DIOR).

A Parameter-Free Pixel Correlation-Based Attention Module for Remote Sensing Object Detection

Remote sensing image object detection is a challenging task in the field of computer vision due to the complex backgrounds and diverse arrangements of targets in remote sensing images, forming intricate scenes. To overcome this challenge, existing object detection models adopt rotated target detection methods. However, these methods often lead to a loss of semantic information during feature extraction, specifically regarding the insufficient consideration of element correlations. To solve this problem, this research introduces a novel attention module (EuPea) designed to effectively capture inter-elemental information in feature maps and generate more powerful feature maps for use in neural networks. In the EuPea attention mechanism, we integrate distance information and Pearson correlation coefficient information between elements in the feature map. Experimental results show that using either type of information individually can improve network performance, but their combination has a stronger effect, producing an attention-weighted feature map. This improvement effectively enhances the object detection performance of the model, enabling it to better comprehend information in remote sensing images. Concurrently, this also improves missed detections and false alarms in object detection. Experimental results obtained on the DOTA, NWPU VHR-10, and DIOR datasets indicate that, compared with baseline RCNN models, our approach achieves respective improvements of 1.0%, 2.4%, and 1.8% in mean average precision (mAP).

Fusing DCN and BBAV for Remote Sensing Image Object Detection

At the oriented object detection in aerial remote sensing images, the perceptual field boundaries of ordinary convolutional kernels are often not parallel to the boundaries of the objects to be detected, affecting the model precision. Therefore, an object detection model (DCN-BBAV) that fuses deformable convolution networks (DCNs) and box boundary-aware vectors (BBAVs) is proposed. Firstly, a BBAV is used as the baseline, replacing the normal convolution kernels in the backbone network with deformable convolution kernels. Then, the spatial attention module (SAM) and channel attention mechanism (CAM) are used to enhance the feature extraction ability for a DCN. Finally, the dot product of the included angles of four adjacent vectors are added to the loss function of the rotation frame parameter, improving the regression precision of the boundary vector. The DCN-BBAV model demonstrates notable performance with a 77.30% mean average precision (mAP) on the DOTA dataset. Additionally, it outperforms other advanced rotating frame object detection methods, achieving impressive results of 90.52% mAP on VOC07 and 96.67% mAP on VOC12 for HRSC2016.

YOLO-MFD: Remote Sensing Image Object Detection with Multi-Scale Fusion Dynamic Head

YOLO-MFD: Remote Sensing Image Object Detection with Multi-Scale Fusion Dynamic Head

M2-YOLOX: A Novel Method for Object Detection Based on an Improved YOLOX Algorithm Introducing a Global Attention Mechanism and a Feature Enhancement Module

Deep learning-based algorithms for detecting objects in remote sensing images have produced excellent results recently. However, the target recognition and classification process of remote sensing images has problems such as dense targets, uneven distribution, large-scale changes and complex backgrounds. In order to improve the effectiveness of existing detection methods, based on the YOLOX algorithm, a remote sensing image object detection algorithm introducing Global Attention Mechanism (GAM) and Feature Enhancement Module (FEM) proposed, named the M2-YOLOX(GAM+FEM+YOLOX) algorithm. First, a novel GAM module is developed that employs a sequential channel-space attention mechanism and redesigns the Convolutional Block Attention Module (CBAM), to address the issues of low effective information extraction and weak information representation of the feature map in the backbone network. CBAM is capable of amplifying global dimensional interaction features while reducing information dispersion as well. Second, the goal of the FEM is to improve the target feature extraction capabilities of the backbone feature extraction network by fusing numerous perceptual field features in lower-level feature maps. Then, the Flexible Rectified Linear Unit (FReLU) activation function is introduced under the action of feature fusion and global attention mechanism. Four-way feature map output in Feature Pyramid Networks (FPN) with Non-Maximum Suppression (NMS) and score filtering for object detection and output results. In comparison to the YOLOX algorithm, the experimental results show that the mAP value of the M2-YOLOX algorithm is improved by 0.0123, the LAMR value is decreased by 0.0150, the Precision rate value is increased by 0.105450, the Recall value is increased by 0.053250, and the mF1 value is increased by 0.0425.

Self-Training-Based Semantic-Balanced Network for Weakly Supervised Object Detection in Remote-Sensing Images

Self-Training-Based Semantic-Balanced Network for Weakly Supervised Object Detection in Remote-Sensing Images

A Review: Remote Sensing Image Object Detection Algorithm Based on Deep Learning

A Review: Remote Sensing Image Object Detection Algorithm Based on Deep Learning

YOLO-SE: Improved YOLOv8 for Remote Sensing Object Detection and Recognition

Object detection remains a pivotal aspect of remote sensing image analysis, and recent strides in Earth observation technology coupled with convolutional neural networks (CNNs) have propelled the field forward. Despite advancements, challenges persist, especially in detecting objects across diverse scales and pinpointing small-sized targets. This paper introduces YOLO-SE, a novel YOLOv8-based network that innovatively addresses these challenges. First, the introduction of a lightweight convolution SEConv in lieu of standard convolutions reduces the network’s parameter count, thereby expediting the detection process. To tackle multi-scale object detection, the paper proposes the SEF module, an enhancement based on SEConv. Second, an ingenious Efficient Multi-Scale Attention (EMA) mechanism is integrated into the network, forming the SPPFE module. This addition augments the network’s feature extraction capabilities, adeptly handling challenges in multi-scale object detection. Furthermore, a dedicated prediction head for tiny object detection is incorporated, and the original detection head is replaced by a transformer prediction head. To address adverse gradients stemming from low-quality instances in the target detection training dataset, the paper introduces the Wise-IoU bounding box loss function. YOLO-SE showcases remarkable performance, achieving an average precision at IoU threshold 0.5 (AP50) of 86.5% on the optical remote sensing dataset SIMD. This represents a noteworthy 2.1% improvement over YOLOv8 and YOLO-SE outperforms the state-of-the-art model by 0.91%. In further validation, experiments on the NWPU VHR-10 dataset demonstrated YOLO-SE’s superiority with an accuracy of 94.9%, surpassing that of YOLOv8 by 2.6%. The proposed advancements position YOLO-SE as a compelling solution in the realm of deep learning-based remote sensing image object detection.

A Novel Adaptive Edge Aggregation and Multiscale Feature Interaction Detector for Object Detection in Remote Sensing Images

Object detection (OD) in remote sensing (RS) images is an important task in the field of computer vision. OD techniques have achieved impressive advances in recent years. However, complex background interference, large-scale variations, and dense instances pose significant challenges for OD. These challenges may lead to misalignment between features extracted by OD models and the features of real objects. To address these challenges, we explore a novel single-stage detection framework for the adaptive fusion of multiscale features and propose a novel adaptive edge aggregation and multiscale feature interaction detector (AEAMFI-Det) for OD in RS images. AEAMFI-Det consists of an adaptive edge aggregation (AEA) module, a feature enhancement module (FEM) embedded in a context-aware cross-attention feature pyramid network (2CA-FPN), and a pyramid squeeze attention (PSA) module. The AEA module employs an edge enhancement mechanism to guide the network to learn spatial multiscale nonlocal dependencies and solve the problem of feature misalignment between the network’s focus and the real object. The 2CA-FPN employs level-by-level feature fusion to enhance multiscale feature interactions and effectively mitigate the misalignment between the scales of the extracted features and the scales of real objects. The FEM is designed to capture the local and nonlocal contexts as auxiliary information to enhance the feature representation of information interaction between multiscale features in a cross-attention manner. We introduce the PSA module to establish long-term dependencies between multiscale spaces and channels for better interdependency refinement. Experimental results obtained using the NWPU VHR-10 and DIOR datasets demonstrate the superior performance of AEAMFI-Det in object classification and localization.

Consistency- and dependence-guided knowledge distillation for object detection in remote sensing images

As one of the challenging tasks in the remote sensing (RS), object detection has been successfully applied in many fields. Convolution neural network (CNN) has recently attracted extensive attention and is widely used in the natural image processing. Nevertheless, RS images have cluttered scenes compared with natural images. As a result, the existing detectors perform poorly in RS images, especially with the complicated backgrounds. Moreover, the detection inference time and model volume of detectors in RS images often go unrecognized. To address the above issues, this study proposes a novel method for object detection in RS images, which is called the consistency- and dependence-guided knowledge distillation (CDKD). To this end, the spatial- and channel-oriented structure discriminative modules (SCSDM) are put forward to extract the discriminative spatial locations and channels to which the teacher model pays attention. SCSDM improves the feature representation of the student model by effectively eliminating the influence of noises and the complicated backgrounds. Then, the consistency and dependence of the features between the teacher model and the student model are constructed under the guidance of SCSDM. Experimental results over public datasets for RS images demonstrate that our CDKD method surpasses the state-of-the-art methods effectively. Most of all, on the RSOD dataset, our CDKD method achieves 92% mean average precision with 3.3M model volume and 588.2 frames per second.

CamoNet: A Target Camouflage Network for Remote Sensing Images Based on Adversarial Attack

Object detection algorithms based on convolutional neural networks (CNNs) have achieved remarkable success in remote sensing images (RSIs), such as aircraft and ship detection, which play a vital role in military and civilian fields. However, CNNs are fragile and can be easily fooled. There have been a series of studies on adversarial attacks for image classification in RSIs. However, the existing gradient attack algorithms designed for classification cannot achieve excellent performance when directly applied to object detection, which is an essential task in RSI understanding. Although we can find some works on adversarial attacks for object detection, they are weak in concealment and easily detected by the naked eye. To handle these problems, we propose a target camouflage network for object detection in RSIs, called CamoNet, to deceive CNN-based detectors by adding imperceptible perturbation to the image. In addition, we propose a detection space initialization strategy to maximize the diversity in the detector’s outputs among the generated samples. It can enhance the performance of the gradient attack algorithms in the object detection task. Moreover, a key pixel distillation module is employed, which can further reduce the modified pixels without weakening the concealment effect. Compared with several of the most advanced adversarial attacks, the proposed attack has advantages in terms of both peak signal-to-noise ratio (PSNR) and attack success rate. The transferability of the proposed target camouflage network is evaluated on three dominant detection algorithms (RetinaNet, Faster R-CNN, and RTMDet) with two commonly used remote sensing datasets (i.e., DOTA and DIOR).

Faster and Lightweight: An Improved YOLOv5 Object Detector for Remote Sensing Images

In recent years, the realm of deep learning has witnessed significant advancements, particularly in object detection algorithms. However, the unique challenges posed by remote sensing images, such as complex backgrounds, diverse target sizes, dense target distribution, and overlapping or obscuring targets, demand specialized solutions. Addressing these challenges, we introduce a novel lightweight object detection algorithm based on Yolov5s to enhance detection performance while ensuring rapid processing and broad applicability. Our primary contributions include: firstly, we implemented a new Lightweight Asymmetric Detection Head (LADH-Head), replacing the original detection head in the Yolov5s model. Secondly, we introduce a new C3CA module, incorporating the Coordinate Attention mechanism, strengthening the network’s capability to extract precise location information. Thirdly, we proposed a new backbone network, replacing the C3 module in the Yolov5s backbone with a FasterConv module, enhancing the network’s feature extraction capabilities. Additionally, we introduced a Content-aware Feature Reassembly (content-aware reassembly of features) (CARAFE) module to reassemble semantic similar feature points effectively, enhancing the network’s detection capabilities and reducing the model parameters. Finally, we introduced a novel XIoU loss function, aiming to improve the model’s convergence speed and robustness during training. Experimental results on widely used remote sensing image datasets such as DIOR, DOTA, and SIMD demonstrate the effectiveness of our proposed model. Compared to the original Yolov5s algorithm, we achieved a mean average precision (mAP) increase of 3.3%, 6.7%, and 3.2%, respectively. These findings underscore the superior performance of our proposed model in remote sensing image object detection, offering an efficient, lightweight solution for remote sensing applications.

Hardware Acceleration of Satellite Remote Sensing Image Object Detection Based on Channel Pruning

Real-time detection of satellite remote sensing images is one of the key technologies in the field of remote sensing, which requires not only high-efficiency algorithms, but also low-power and high-performance hardware deployment platforms. At present, the image processing hardware acceleration platform mainly uses an image processing unit (GPU), but the GPU has the problem of large power consumption, and it is difficult to apply to micro-nano satellites and other devices with limited volume, weight, computing power, and power consumption. At the same time, the deep learning algorithm model has the problem of too many parameters, and it is difficult to directly deploy it on embedded devices. In order to solve the above problems, we propose a YOLOv4-MobileNetv3 field programmable gate array (FPGA) deployment scheme based on channel layer pruning. Experiments show that the acceleration strategy proposed by us can reduce the number of model parameters by 91.11%, and on the aerial remote sensing dataset DIOR, the average accuracy of the design scheme in this paper reaches 82.61%, the FPS reaches 48.14, and the average power consumption is 7.2 W, which is 317.88% FPS higher than the CPU and reduces the power consumption by 81.91%. Compared to the GPU, it reduces power consumption by 91.85% and improves FPS by 8.50%. Compared with CPUs and GPUs, our proposed lightweight algorithm model is more energy-efficient and more real-time, and is suitable for application in spaceborne remote sensing image processing systems.

Oriented Object Detection in Remote Sensing Using an Enhanced Feature Pyramid Network

Object detection in remote sensing images is a critical task within the field of remote sensing image interpretation and analysis, serving as a fundamental foundation for military surveillance and traffic guidance. Recently, although many object detection algorithms have been improved to adapt to the characteristics of remote sensing images and have achieved good performance, most of them still use horizontal bounding boxes, which struggle to accurately mark targets with multiple angles and dense arrangements in remote sensing images. We propose an oriented bounding box optical remote sensing image object detection method based on an enhanced feature pyramid, and add an attention module to suppress background noise. To begin with, we incorporate an angle prediction module that accurately locates the detection target. Subsequently, we design an enhanced feature pyramid network, utilizing deformable convolutions and feature fusion modules to enhance the feature information of rotating targets and improve the expressive capacity of features at all levels. The proposed algorithm in this paper performs well on the public DOTA dataset and HRSC2016 dataset, compared with other object detection methods, and the detection accuracy AP values of most object categories are improved by at least three percentage points. The results show that our method can accurately locate densely arranged and dynamically oriented targets, significantly reducing the risk of missing detections, and achieving higher levels of target detection accuracy.

SFTN: Fast object detection for aerial images

AbstractThe task of remote sensing image object detection in low latency scenes is of great research significance. To address the problem that the current high‐precision object detection algorithm based on a feature pyramid network is slow due to a large number of parameters and complicated computation, a fast remote sensing image object detection method based on a Single‐scale Feature Transformation Network (SFTN) is proposed. Firstly, based on the single‐scale remote sensing image features, the new channel features are quickly generated by a linear transformation of the original features and convolution kernel clustering optimization using cosine similarity; secondly, in order to obtain multi‐scale receptive fields, a parallel residual hole convolution module is designed to cover multi‐category remote sensing object scales on the feature map; finally, angle variables are introduced and optimized using angle similarity to effectively improve the object orientation accuracy. The experimental results on different datasets show that the method in this paper improves the detection speed rapidly while ensuring the accuracy of remote sensing image object detection, which is better than many remote sensing image object detection methods. The results demonstrate the reliability and robustness of the method.