Oriented Object Detection Research Articles

Boosting Point Set-Based Network with Optimal Transport Optimization for Oriented Object Detection

When handling complex remote sensing scenarios, rotational angle information can improve detection accuracy and enhance algorithm robustness, providing support for fine-grained detection. Point set representation is one of the most commonly used methods in arbitrary-oriented object detection tasks, leveraging discrete feature points to represent oriented targets and achieve high accuracy in angle prediction. However, due to the inherent discreteness of point set representation, it is prone to significant impact from isolated points and representational ambiguity in harsh application scenarios, leading to inaccurate detection. To address this issue, an efficient aerial object detector named BE-Det is proposed, which uses the optimal transport (OT) strategy to constrain the positions of isolated points. Additionally, a candidate point set quality evaluation scheme is designed to effectively assess the quality of candidate point sets. Experimental results on two challenging aerial datasets demonstrate that the proposed method outperforms several advanced detection methods.

Feature Enhancement Based Oriented Object Detection in Remote Sensing Images

Since objects in remote sensing imagery often have arbitrary orientations and high densities, the features of small objects are inclined to be contaminated by the background and other instances. To address the issues, we propose a new oriented object detection framework where a series of feature enhancement schemes are implemented so as to improve robustness and accuracy of the detector. Firstly, we design a weighted bidirectional feature pyramid network, which can be used to fuse both high-level semantic features and low-level detail features for effectively handling with multi-scale objects. Accordingly, we apply the convolutional block attention module that exploits both spatial- and channel-wise attention in our detector, and study how to effectively integrate it into the framework for adaptive feature refinement. In the meanwhile, we present a semantic segmentation guided module to generate naive mask, which is used to multiple with pyramid features to filter out background noise and improve feature representation for small objects. The experimental results on two public datasets, i.e., UCAS-AOD and DOTA, validate the effective performance of the proposed method for oriented object detection in remote sensing images.

On Boundary Discontinuity in Angle Regression Based Arbitrary Oriented Object Detection.

With vigorous development e.g., in autonomous driving and remote sensing, oriented object detection has gradually been featured. The majority of existing methods directly perform regression on the rotation angle, which we argue has fundamental limitations of boundary discontinuity (even if using Gaussian or RotatedIoU-based losses). In this paper, a novel angle coder named phase-shifting coder (PSC) is proposed to address this issue. Different from another well-explored alternative i.e., angle classification, PSC achieves boundary-discontinuity-free in a continuous and differentiable manner and thus can work together with Gaussian or RotatedIoU-based methods to further boost their performance. Moreover, by rethinking the boundary discontinuity of elongated and square-like objects as rotational symmetry of different cycles, a dual-frequency version (PSCD) is proposed to accurately predict the orientation of both types of objects. Visual analysis and extensive experiments on several popular backbone detectors and datasets demonstrate the effectiveness and the potentiality of our approach. When facing scenarios requiring high-quality bounding boxes, the proposed methods are expected to give a competitive performance.

Consistency-based semi-supervised learning for oriented object detection

Semi-Supervised Object Detection (SSOD) has emerged as a potent framework that leverages unlabeled data to reduce annotation costs while enhancing model performance. Nevertheless, existing SSOD methods primarily concentrate on the conventional localization of horizontal objects, overlooking the common arbitrary-oriented objects in aerial scenes. Extending semi-supervised learning to oriented object detection still encounters two major challenges: (1) There is an inconsistency between classification and localization in oriented object detectors, which means that pseudo-labels selected solely based on classification scores cannot properly represent the localization quality. (2) The static and monotonic pseudo-label selection process fails to dynamically filter pseudo-labels for different tasks and categories throughout the iteration process, inevitably introducing accumulated noise and bias into the supervisory signals for the student model. To mitigate this problems, we proposed a novel consistency-based semi-supervised oriented object detection framework, named CSLO-Det. Specifically, Task Alignment Learning (TAL) is proposed to improve the consistency in terms of both features and learning objectives, thereby facilitating a more robust selection of pseudo-labels. Noise-Resistant Pseudo-label Mining (NR-PM) is introduced to separately and dynamically exploits positives for the classification and localization task. Moreover, diverging from the conventional semi-supervised learning frameworks that utilize pseudo-boxes, our method applies pixel-level dense supervision during unsupervised training, which improves detection performance. Comprehensive experiment results reveal that the proposed CSLO-Det attains state-of-the-art performance under multiple semi-supervised datasets.

DETR-ORD: An Improved DETR Detector for Oriented Remote Sensing Object Detection with Feature Reconstruction and Dynamic Query

Optical remote sensing images often feature high resolution, dense target distribution, and uneven target sizes, while transformer-based detectors like DETR reduce manually designed components, DETR does not support arbitrary-oriented object detection and suffers from high computational costs and slow convergence when handling large sequences of images. Additionally, bipartite graph matching and the limit on the number of queries result in transformer-based detectors performing poorly in scenarios with multiple objects and small object sizes. We propose an improved DETR detector for Oriented remote sensing object detection with Feature Reconstruction and Dynamic Query, termed DETR-ORD. It introduces rotation into the transformer architecture for oriented object detection, reduces computational cost with a hybrid encoder, and includes an IFR (image feature reconstruction) module to address the loss of positional information due to the flattening operation. It also uses ATSS to select auxiliary dynamic training queries for the decoder. This improved DETR-based detector enhances detection performance in challenging oriented optical remote sensing scenarios with similar backbone network parameters. Our approach achieves superior results on most optical remote sensing datasets, such as DOTA-v1.5 (72.07% mAP) and DIOR-R (66.60% mAP), surpassing the baseline detector.

Relevance Pooling Guidance and Class-Balanced Feature Enhancement for Fine-Grained Oriented Object Detection in Remote Sensing Images

Fine-grained object detection in remote sensing images is highly challenging due to class imbalance and high inter-class indistinguishability. The strategies employed by most existing methods to resolve these two challenges are relatively rudimentary, resulting in suboptimal model performance. To address these issues, we propose a fine-grained object-oriented detection method based on relevance pooling guidance and class balance feature enhancement. Firstly, we propose a global attention mechanism that dynamically retains spatial features pertinent to objects through relevance pooling during down-sampling, thereby enabling the model to acquire more discriminative features. Next, a class balance correction module is proposed to alleviate the class imbalance problem. This module employs feature translation and a learnable reinforcement coefficient to highlight the boundaries of tail class features while maintaining their distinctiveness. Furthermore, we present an enhanced contrastive learning strategy. By dynamically adjusting the contribution of inter-class samples and intra-class similarity measures, this strategy not only constrains inter-class feature distances but also facilitates tighter intra-class clustering, making it more suitable for imbalanced datasets. Evaluation on the FAIR1M and MAR20 datasets demonstrates that our method is superior compared to other methods in object detection and achieves 46.44 and 85.05% mean average precision, respectively.

DOUNet: Dynamic Optimization and Update Network for Oriented Object Detection

DOUNet: Dynamic Optimization and Update Network for Oriented Object Detection

S$^{2}$O-Det: A Semisupervised Oriented Object Detection Network for Remote Sensing Images

S$^{2}$O-Det: A Semisupervised Oriented Object Detection Network for Remote Sensing Images

Turning a CLIP Model Into a Scene Text Spotter.

We exploit the potential of the large-scale Contrastive Language-Image Pretraining (CLIP) model to enhance scene text detection and spotting tasks, transforming it into a robust backbone, FastTCM-CR50. This backbone utilizes visual prompt learning and cross-attention in CLIP to extract image and text-based prior knowledge. Using predefined and learnable prompts, FastTCM-CR50 introduces an instance-language matching process to enhance the synergy between image and text embeddings, thereby refining text regions. Our Bimodal Similarity Matching (BSM) module facilitates dynamic language prompt generation, enabling offline computations and improving performance. FastTCM-CR50 offers several advantages: 1) It can enhance existing text detectors and spotters, improving performance by an average of 1.6% and 1.5%, respectively. 2) It outperforms the previous TCM-CR50 backbone, yielding an average improvement of 0.2% and 0.55% in text detection and spotting tasks, along with a 47.1% increase in inference speed. 3) It showcases robust few-shot training capabilities. Utilizing only 10% of the supervised data, FastTCM-CR50 improves performance by an average of 26.5% and 4.7% for text detection and spotting tasks, respectively. 4) It consistently enhances performance on out-of-distribution text detection and spotting datasets, particularly the NightTime-ArT subset from ICDAR2019-ArT and the DOTA dataset for oriented object detection.

Dynamic weighting label assignment for oriented object detection

Dynamic weighting label assignment for oriented object detection

Learning Oriented Object Detection via Naive Geometric Computing.

Detecting oriented objects along with estimating their rotation information is one crucial step for image analysis, especially for remote sensing images. Despite that many methods proposed recently have achieved remarkable performance, most of them directly learn to predict object directions under the supervision of only one (e.g., the rotation angle) or a few (e.g., several coordinates) groundtruth (GT) values individually. Oriented object detection would be more accurate and robust if extra constraints, with respect to proposal and rotation information regression, are adopted for joint supervision during training. To this end, we propose a mechanism that simultaneously learns the regression of horizontal proposals, oriented proposals, and rotation angles of objects in a consistent manner, via naive geometric computing, as one additional steady constraint. An oriented center prior guided label assignment strategy is proposed for further enhancing the quality of proposals, yielding better performance. Extensive experiments on six datasets demonstrate the model equipped with our idea significantly outperforms the baseline by a large margin and several new state-of-the-art results are achieved without any extra computational burden during inference. Our proposed idea is simple and intuitive that can be readily implemented. Source codes are publicly available at: https://github.com/wangWilson/CGCDet.git.

FADL-Net: Frequency-Assisted Dynamic Learning Network for Oriented Object Detection in Remote Sensing Images

FADL-Net: Frequency-Assisted Dynamic Learning Network for Oriented Object Detection in Remote Sensing Images

Improved YOLOv5 Algorithm for Oriented Object Detection of Aerial Image

Improved YOLOv5 Algorithm for Oriented Object Detection of Aerial Image

SREDet: Semantic-Driven Rotational Feature Enhancement for Oriented Object Detection in Remote Sensing Images

Significant progress has been achieved in the field of oriented object detection (OOD) in recent years. Compared to natural images, objects in remote sensing images exhibit characteristics of dense arrangement and arbitrary orientation while also containing a large amount of background information. Feature extraction in OOD becomes more challenging due to the diversity of object orientations. In this paper, we propose a semantic-driven rotational feature enhancement method, termed SREDet, to fully leverage the joint semantic and spatial information of oriented objects in the remote sensing images. We first construct a multi-rotation feature pyramid network (MRFPN), which leverages a fusion of multi-angle and multiscale feature maps to enhance the capability to extract features from different orientations. Then, considering feature confusion and contamination caused by the dense arrangement of objects and background interference, we present a semantic-driven feature enhancement module (SFEM), which decouples features in the spatial domain to separately enhance the features of objects and weaken those of backgrounds. Furthermore, we introduce an error source evaluation metric for rotated object detection to further analyze detection errors and indicate the effectiveness of our method. Extensive experiments demonstrate that our SREDet method achieves superior performance on two commonly used remote sensing object detection datasets (i.e., DOTA and HRSC2016).

Linear Gaussian bounding box representation and ring-shaped rotated convolution for oriented object detection

In oriented object detection, current representations of oriented bounding boxes (OBBs) often suffer from the boundary discontinuity problem. Methods of designing continuous regression losses do not essentially solve this problem. Although Gaussian bounding box (GBB) representation avoids this problem, directly regressing GBB is susceptible to numerical instability. We propose linear GBB (LGBB), a novel OBB representation. By linearly transforming the elements of GBB, LGBB avoids the boundary discontinuity problem and has high numerical stability. In addition, existing convolution-based rotation-sensitive feature extraction methods only have local receptive fields, resulting in slow feature aggregation. We propose ring-shaped rotated convolution (RRC), which adaptively rotates feature maps to arbitrary orientations to extract rotation-sensitive features under a ring-shaped receptive field, rapidly aggregating features and contextual information. Experimental results demonstrate that LGBB and RRC achieve state-of-the-art performance. Furthermore, integrating LGBB and RRC into various models effectively improves detection accuracy.

Improving Oriented Object Detection by Scene Classification and Task-Aligned Focal Loss

Oriented object detection (OOD) can precisely detect objects with arbitrary direction in remote sensing images (RSIs). Up to now, the two-stage OOD methods have attracted more attention because of their high detection accuracy. However, the two-stage methods only rely on the features of each proposal for object recognition, which leads to the misclassification problem because of the intra-class diversity, inter-class similarity and clutter backgrounds in RSIs. To address the above problem, an OOD model combining scene classification is proposed. Considering the fact that each foreground object has a strong contextual relationship with the scene of the RSI, a scene classification branch is added to the baseline OOD model, and the scene classification result of input RSI is used to exclude the impossible categories. To focus on the hard instances and enhance the consistency between classification and regression, a task-aligned focal loss (TFL) which combines the classification difficulty with the regression loss is proposed, and TFL assigns lager weights to the hard instances and optimizes the classification and regression branches simultaneously. The ablation study proves the effectiveness of scene classification branch, TFL and their combination. The comparisons with 15 and 14 OOD methods on the DOTA and DIOR-R datasets validate the superiority of our method.

Single-stage oriented object detection via Corona Heatmap and Multi-stage Angle Prediction

Oriented-object detection is a pivotal research subject within computer vision. Nevertheless, extant-oriented detection methodologies frequently grapple with boundary problems when handling targets with large aspect ratios and arbitrary orientations. This study introduces a Single-stage oriented-object detection via Corona Heatmap and Multi-stage Angle Prediction (SOD) to address the above challenges. Specifically, the angle is conceptualised as a multi-stage classification prediction; the initial stage forecasts the approximate rotational direction of the target. Based on the outcome of the first stage, the subsequent stage uses Gaussian smoothing encoding to predict the precise angle, mitigating discrepancies caused by floating-point representation. Moreover, this study proposes a spatially constrained corona heatmap to rectify the issue of central position prediction drift in the Gaussian heatmap when dealing with targets with large aspect ratios. The post-processing methodology is refined based on the above approach, effectively mitigating the deviation of the predicted target’s central position of the predicted target. Visual results and experimental analyses conducted on multiple challenging datasets corroborate the efficacy and competitiveness of the proposed method.

Discretization and decoupled knowledge distillation for arbitrary oriented object detection

In recent years, lightweight object detection networks have been increasingly applied to remote sensing platforms due to their fast inference speed and flexible deployment advantages. Knowledge distillation methods have been widely used to reduce the performance gap between large and small models, and many studies have combined knowledge distillation with object detection. However, existing knowledge distillation methods often overlook the transfer of localization knowledge. Therefore, this paper proposes a method called Discretized Position Knowledge Distillation (DPKD) to improve the use of knowledge distillation in object detection. Specifically, the DPKD method incorporates a Discretization Algorithm Module (DAM), which leverages both general probability distribution and cross-Gaussian distribution to transfer high-quality bounding box position and pose information. Additionally, the Position Knowledge Distillation (PKD) method splits the target and non-target bounding boxes to form the distillation loss function, addressing the issue of missing background knowledge transfer during the distillation process. To further enhance the learning of high-quality bounding boxes, a Region Weighting Module (RWM) based on EIoU is introduced in DPKD, assigning weights to the various bounding boxes in the teacher's output. The effectiveness of DPKD in the field of remote sensing image object detection in multi-modal scenarios was verified through multi-modal training on the publicly available DOTA and HRSID datasets.

Task-Aligned Oriented Object Detection in Remote Sensing Images

Oriented object detection (OOD) can recognize and locate various objects more precisely than horizontal object detection; however, two problems have not been satisfactorily resolved so far. Firstly, the absence of interactions between the classification and regression branches leads to inconsistent performance in the two tasks of object detection. Secondly, the traditional convolution operation cannot precisely extract the features of objects in extremely aspect ratio in remote sensing images (RSIs). To address the first problem, the task-aligned detection module (TADM) and the task-aligned loss function (TL) are proposed in this paper. On the one hand, a spatial probability map and a spatial offset map are inferred from the shared features in the TADM and separately incorporated into the classification and regression branches to obtain consistency in the two tasks. On the other hand, the TL combines employing the generalized intersection over union (GIoU) metric with classification loss to further enhance the consistency in the two tasks. To address the second problem, a two-stage detection framework based on alignment convolution (TDA) is proposed. The features extracted from the backbone network are refined through alignment convolution in the first stage, and the final OOD results are inferred from refined features in the second stage. The ablation study verifies the effectiveness of the TADM, TL, and TDA. The comparisons with other advanced methods, on two RSI benchmarks, demonstrate the overall effectiveness of our method.

FRED: Towards a Full Rotation-Equivariance in Aerial Image Object Detection

Rotation-equivariance is an essential yet challenging property in oriented object detection. While general object detectors naturally leverage robustness to spatial shifts due to the translation-equivariance of the conventional CNNs, achieving rotation-equivariance remains an elusive goal. Current detectors deploy various alignment techniques to derive rotation-invariant features, but still rely on high capacity models and heavy data augmentation with all possible rotations. In this paper, we introduce a Fully Rotation-Equivariant Oriented Object Detector (FRED), whose entire process from the image to the bounding box prediction is strictly equivariant. Specifically, we decouple the invariant task (object classification) and the equivariant task (object localization) to achieve end-to-end equivariance. We represent the bounding box as a set of rotation-equivariant vectors to implement rotation-equivariant localization. Moreover, we utilized these rotation-equivariant vectors as offsets in the deformable convolution, thereby enhancing the existing advantages of spatial adaptation. Leveraging full rotation-equivariance, our FRED demonstrates higher robustness to image-level rotation compared to existing methods. Furthermore, we show that FRED is one step closer to non-axis aligned learning through our experiments. Compared to state-of-the-art methods, our proposed method delivers comparable performance on DOTA-v1.0 and outperforms by 1.5 mAP on DOTA-v1.5, all while significantly reducing the model parameters to 16%.