Detection In Aerial Images Research Articles

Focus-and-Detect: A small object detection framework for aerial images

Despite recent advances, object detection in aerial images is still a challenging task. Specific problems in aerial images makes the detection problem harder, such as small objects, densely packed objects, objects in different sizes and with different orientations. To address small object detection problem, we propose a two-stage object detection framework called “Focus-and-Detect”. The first stage which consists of an object detector network supervised by a Gaussian Mixture Model, generates clusters of objects constituting the focused regions. The second stage, which is also an object detector network, predicts objects within the focal regions. Incomplete Box Suppression (IBS) method is also proposed to overcome the truncation effect of region search approach. Results indicate that the proposed two-stage framework achieves an AP score of 42.06 on VisDrone validation dataset, surpassing all other state-of-the-art small object detection methods reported in the literature, to the best of authors’ knowledge.

CAFC-Net: A Critical and Align Feature Constructing Network for Oriented Ship Detection in Aerial Images.

Ship detection is one of the fundamental tasks in computer vision. In recent years, the methods based on convolutional neural networks have made great progress. However, improvement of ship detection in aerial images is limited by large-scale variation, aspect ratio, and dense distribution. In this paper, a Critical and Align Feature Constructing Network (CAFC-Net) which is an end-to-end single-stage rotation detector is proposed to improve ship detection accuracy. The framework is formed by three modules: a Biased Attention Module (BAM), a Feature Alignment Module (FAM), and a Distinctive Detection Module (DDM). Specifically, the BAM extracts biased critical features for classification and regression. With the extracted biased regression features, the FAM generates high-quality anchor boxes. Through a novel Alignment Convolution, convolutional features can be aligned according to anchor boxes. The DDM produces orientation-sensitive feature and reconstructs orientation-invariant features to alleviate inconsistency between classification and localization accuracy. Extensive experiments on two remote sensing datasets HRS2016 and self-built ship datasets show the state-of-the-art performance of our detector.

Combined and optimized 2-steps ratio map for shadow detection in aerial images

Combined and optimized 2-steps ratio map for shadow detection in aerial images

ARFP: A novel adaptive recursive feature pyramid for object detection in aerial images

ARFP: A novel adaptive recursive feature pyramid for object detection in aerial images

GANsformer: A Detection Network for Aerial Images with High Performance Combining Convolutional Network and Transformer

There has been substantial progress in small object detection in aerial images in recent years, due to the extensive applications and improved performances of convolutional neural networks (CNNs). Typically, traditional machine learning algorithms tend to prioritize inference speed over accuracy. Insufficient samples can cause problems for convolutional neural networks, such as instability, non-convergence, and overfitting. Additionally, detecting aerial images has inherent challenges, such as varying altitudes and illuminance situations, and blurred and dense objects, resulting in low detection accuracy. As a result, this paper adds a transformer backbone attention mechanism as a branch network, using the region-wide feature information. This paper also employs a generative model to expand the input aerial images ahead of the backbone. The respective advantages of the generative model and transformer network are incorporated. On the dataset presented in this study, the model achieves 96.77% precision, 98.83% recall, and 97.91% mAP by adding the Multi-GANs module to the one-stage detection network. These three indices are enhanced by 13.9%, 20.54%, and 10.27%, respectively, when compared to the other detection networks. Furthermore, this study provides an auto-pruning technique that may achieve 32.2 FPS inference speed with a minor performance loss while responding to the real-time detection task’s usage environment. This research also develops a macOS application for the proposed algorithm using Swift development technology.

Classification of objects in images with distortions based on a two-stage topological analysis

The authors propose a method for automatic classification of spatial objects in images under conditions of a limited data set. The stability of the method to distortions appearing in images due to natural phenomena and partial overlap of urban infrastructure objects is investigated. High classification accuracy, when using existing approaches, requires a large training sample, including data sets with distortions, which significantly increases computational complexity. The paper proposes a method for a two-step topological analysis of images. Topological features are initially extracted by analyzing the image in the brightness range from 0 to 255, and then from 255 to 0. These features complement each other and reflect the topological structure of the object. Under certain deformations and distortions, the object preserves its structure in the form of extracted features. The advantage of the method is a small number of patterns, which reduces the computational complexity of training compared to neural networks. The proposed method is investigated and compared with the modern neural network approach. The study was performed on a DOTA dataset (Dataset for Object deTection in Aerial images) containing images of spatial objects of several classes. In the absence of distortion in the image, the neural network approach showed a classification accuracy of over 98 %, while the proposed method achieved about 82 %. Further distortions such as 90 degree rotation, 50 % narrowing and 50 % edge truncation and their combinations were applied in the experiment. The proposed method showed its robustness and outperformed the neural network approach. In the most difficult combination of the test, the decrease in classification accuracy of the neural network was 46 %, while the described method showed 12 %. The proposed method can be applied in cases with a high probability of distortion in the images. Such distortions arise in the field of geoinformatics when analyzing objects of various scales, under different weather conditions, partial overlap of one object with another, in the presence of shadows, etc. It is possible to use the proposed method in vision systems of industrial enterprises for automatic classification of the parts that belong to superimposed objects.

MSODANet: A network for multi-scale object detection in aerial images using hierarchical dilated convolutions

The object detection in aerial images is one of the most commonly used tasks in the wide-range of computer vision applications. However, the object detection is more challenging due to the following issues: (a) the pixel occupancy vary among the different scales of objects, (b) the distribution of objects is not uniform in aerial images, (c) the appearance of an object varies with different view-points and illumination conditions, and (d) the number of objects, even though they belong to same type, vary across the images. To address these issues, we propose a novel network for multi-scale object detection in aerial images using hierarchical dilated convolutions, called as mSODANet. In particular, we probe hierarchical dilated network using parallel dilated convolutions to learn the contextual information of different types of objects at multiple scales and multiple field-of-views. The introduced hierarchical dilated network captures the visual information of aerial image more effectively and enhances the detection capability of the model. Further, the extensive experiments conducted on three challenging publicly available datasets, i.e., Visdrone2019, DOTA (OBB & HBB), NWPU VHR-10, demonstrate the effectiveness of the proposed mSODANet and achieve the state-of-the-art performance on all three datasets.

Enhancing object detection in aerial images.

Unmanned Aerial Vehicles have proven to be helpful in domains like defence and agriculture and will play a vital role in implementing smart cities in the upcoming years. Object detection is an essential feature in any such application. This work addresses the challenges of object detection in aerial images like improving the accuracy of small and dense object detection, handling the class-imbalance problem, and using contextual information to boost the performance. We have used a density map-based approach on the drone dataset VisDrone-2019 accompanied with increased receptive field architecture such that it can detect small objects properly. Further, to address the class imbalance problem, we have picked out the images with classes occurring fewer times and augmented them back into the dataset with rotations. Subsequently, we have used RetinaNet with adjusted anchor parameters instead of other conventional detectors to detect aerial imagery objects accurately and efficiently. The performance of the proposed three step pipeline of implementing object detection in aerial images is a significant improvement over the existing methods. Future work may include improvement in the computations of the proposed method, and minimising the effect of perspective distortions and occlusions.

Fully Convolutional Lightweight Pyramid Network for Vehicle Detection in Aerial Images

Vehicle detection in aerial images has a wide range of applications, and the majority of vehicle detection methods use the bounding-box approach for localization. But the bounding-box approach usually yields low precision and recall rates, especially under the dense vehicle density situations where the close spatial proximity of the vehicles confuses the bounding-box detectors. This letter proposes a Fully Convolutional Lightweight Pyramid Network (FCLPN) to detect vehicles in visible-spectrum aerial images. Unlike the bounding-box approach, FCLPN performs pixel-level localization and classification. FCLPN trained on the DLR-3K dataset is directly tested on DLR-3K, VEDAI, COWC, and VAID datasets to validate its generalization strength. Experimental results show that FCLPN performs better than state-of-the-art methods in aerial vehicle detection in terms of Precision, F1 score, and mAP.

Rethinking Classification of Oriented Object Detection in Aerial Images

With the help of the rapid development of technology, especially the prevalence of UAVs (unmanned aerial vehicles), object detection in aerial images gains much more attention in computer vision and deep learning. However, traditional methods use horizontal bounding boxes for object representation leading to inconsistency between objects and features. Therefore, many detectors are being built to tackle this problem, and normally they use the conventional approaches of training and testing to achieve the results. Our pipeline proposed to strengthen not only the classification but also localization via independent training processes using convex-hull transformation in data pre-processing phase. We experimented with the well-designed S2ANet, R3Det, ReDet, RoI Transformer and Oriented R-CNN on the well-established oriented object detection dataset DOTA. Then we adopt the best detectors with the well-known classification network EfficientNet to our proposed pipeline and achieve promising results on the oriented object detection DOTA dataset. Moreover, our pipeline can flexibly be adapted to various oriented object detection baselines improving the results in classification via independent extensive training cycles.

Gaussian Focal Loss: Learning Distribution Polarized Angle Prediction for Rotated Object Detection in Aerial Images

With the increasing availability of aerial data, object detection in aerial images has aroused more and more attention in remote sensing community. The difficulty lies in accurately predicting the angular information for each target when using the oriented bounding boxes to represent the arbitrary oriented objects, as the periodicity of the angle could cause inconsistency between target angle values. To resolve the problem, recent works propose to perform angular prediction from a regression problem to a classification task with circular smooth label. However, we find that current loss functions applying to binary soft labels need to approximate the soft label values at each position. When summed over all the negative angle categories, these relatively insignificant loss values can overwhelm the target angle category, thus preventing the network from predicting precise angle information. In this paper, we propose a novel loss function that acts as a more effective alternative to the classification-based rotated detectors. By constructing the classification loss with adaptive Gaussian attenuation on the negative locations, our training objective can not only avoid discontinuous angle boundaries but also enable the network to obtain more accurate angle predictions with higher response at peaks. Moreover, an aspect ratio-aware factor was proposed based on our loss function to enhance the robustness of the model for determining the orientation for square-like objects. Extensive experiments on aerial image datasets DOTA, HRSC2016, and UCAS-AOD demonstrated the effectiveness and superior performances of our approaches.

Reducing False Detections in Aerial Images by Exploiting the Context Information and Centroid Relationship

Object detection in aerial images is critical for military and civilian applications. Due to its broad imaging range, an aerial image contains more complex scenes than natural images, which may result in increased confusion between targets and the background. To address this issue, we propose a context-aware and centroid-prior-based object detector for the object detection task in aerial images. Unlike existing object detectors, the proposed model investigates the feasibility of reducing false detections from confusing backgrounds by exploiting both semantic and neighbor contexts. Additionally, a new label assignment algorithm is developed based on the centroid prior assumption to mitigate the localization errors introduced by context information. The experimental results demonstrate that the proposed model outperforms six advanced object detectors on the optical and synthetic aperture radar (SAR) image datasets, indicating the proposed model’s effectiveness.

DARDet: A Dense Anchor-Free Rotated Object Detector in Aerial Images

Rotated object detection in aerial images has received increasing attention for a wide range of applications. However, it is also a challenging task due to the huge variations of scale, rotation, aspect ratio, and densely arranged targets. Most existing methods heavily rely on a large number of pre-defined anchors with different scales, angles, and aspect ratios, and are optimized with a distance loss. Therefore, these methods are sensitive to anchor hyper-parameters and easily suffer from performance degradation caused by boundary discontinuity. To handle this problem, in this paper, we propose a dense anchor-free rotated object detector (DARDet) for rotated object detection in aerial images. Our DARDet directly predicts five parameters of rotated boxes at each foreground pixel of feature maps. We design a new alignment convolution module to extracts aligned features and introduce a PIoU loss for precise and stable regression. Our method achieves state-of-the-art performance on three commonly used aerial objects datasets (i.e., DOTA, HRSC2016, and UCAS-AOD) while keeping high efficiency. Code is available at https://github.com/zf020114/DARDet.

R2YOLOX: A Lightweight Refined Anchor-Free Rotated Detector for Object Detection in Aerial Images

Existing anchor-based rotated object detection methods have achieved some amazing results, but these methods require some manual preset anchors, which not only introduce additional hyperparameters but also introduce extra computational burdens. Due to the above drawbacks, anchor-free methods have been rapidly developed in recent years. However, the existing high-performance anchor-free rotated object detection methods are relatively complex and the inference speed is also slow. And Yolo series models not only maintain high-efficiency inference but also keep competitive performance detection performance in the general object detection tasks. Hence, we propose an anchor-free rotated detector based on the YOLOX method for object detection in aerial images. Our methods consist of two improvements: a Refined Rotated Module (RRM) and a new assigner method which is called the Gaussian distribution Sampling Optimal Transport Assignment method (GSOTA). The RRM can align features and get more useful priors for final detector heads. The GSOTA uses Gaussian Distribution to model the oriented bounding box (OBB) firstly, and a Gaussian Center Sampling method (GCS) with maximum classification center mean (MCCM) is proposed to simplify the label Assignment Optimal Transport problem, finally using an improved dynamic top-k strategy to get an approximate solution. Extensive experiments demonstrate that our models can achieve competitive performance in several challenging aerial object detection datasets while keeping the best efficiency. Our R2YOLOX-X model achieves 79.33%, 97.4%, 97.7%, and 92.5% mAP on the DOTA, HRSC2016, UCAS-AOD, and FGSD2021, respectively, while R2YOLOX-S can reach the fastest 58.2 FPS when inferencing on aerial datasets and R2YOLOX-L gets the best speed-accuracy trade-off.

Wall Cracks Detection in Aerial Images Using Improved Mask R-CNN

The present paper proposes a detection method for building exterior wall cracks since manual detection methods have high risk and low efficiency. The proposed method is based on Unmanned Aerial Vehicle (UAV) and computer vision technology. First, a crack dataset of 1920 images was established using UAV to collect the images of a residential building exterior wall under different lighting conditions. Second, the average crack detection precisions of different methods including the Single Shot MultiBox Detector, You Only Look Once v3, You Only Look Once v4, Faster Regional Convolutional Neural Network (R-CNN) and Mask R-CNN methods were compared. Then, the Mask R-CNN method with the best performance and average precision of 0.34 was selected. Finally, based on the characteristics of cracks, the utilization ratio of Mask R-CNN to the underlying features was improved so that the average precision of 0.9 was achieved. It was found that the positioning accuracy and mask coverage rate of the proposed Mask R-CNN method are greatly improved. Also, it will be shown that using UAV is safer than manual detection because manual parameter setting is not required. In addition, the proposed detection method is expected to greatly reduce the cost and risk of manual detection of building exterior wall cracks and realize the efficient identification and accurate labeling of building exterior wall cracks.

Multi-Oriented Object Detection in Aerial Images With Double Horizontal Rectangles.

Most existing methods adopt the quadrilateral or rotated rectangle representation to detect multi-oriented objects. Yet, the same oriented object may correspond to several different representations, due to different vertex ordering, or angular periodicity and edge exchangeability. To ensure the uniqueness of the representation, some engineered rules are usually added. This makes these methods suffer from discontinuity problem, resulting in degraded performance for objects around some orientation. In this article, we propose to encode the multi-oriented object with double horizontal rectangles (DHRec) to solve the discontinuity problem. Specifically, for an oriented object, we arrange the horizontal and vertical coordinates of its four vertices in left-right and top-down order, respectively. The first (resp. second) horizontal box is given by two diagonal points with smallest (resp. second) and third (resp. largest) coordinates in both horizontal and vertical dimensions. We then regress three factors given by area ratios between different regions, helping to guide the oriented object decoding from the predicted DHRec. Inherited from the uniqueness of horizontal rectangle representation, the proposed method is free of discontinuity issue, and can accurately detect objects of arbitrary orientation. Extensive experimental results show that the proposed method significantly improves the existing baseline representation, and outperforms state-of-the-art methods. The code is available at: https://github.com/lightbillow/DHRec.

Improved Yolov5 for Small Target Detection in Aerial Images

Improved Yolov5 for Small Target Detection in Aerial Images

An Anchor-Free Network With Density Map and Attention Mechanism for Multiscale Object Detection in Aerial Images

Accurate detection of the multiple classes in aerial images has become possible with the use of anchor-based object detectors. However, anchor-based object detectors place a large number of preset anchors on images and regress the target bounding box while anchor-free object detections predict the location of objects directly and avoid the carefully predefined anchor box parameters. Object detection in aerial images is faced with two main challenges: 1) the scale diversity of the geospatial objects; and 2) the cluttered background in complex scenes. In this letter, to address these challenges, we present a novel Anchor-Free Network with a Density map and attention mechanism (DA <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> FNet). Considering the extreme density variations of the detection instances among the different categories in aerial images, the proposed DA <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> FNet model conducts density map estimation with image-level supervision for the geospatial object counting, to acquire global knowledge about the scale information. A simple and effective image-level global counting loss function is also introduced. In addition, a compositional attention network is further introduced to enhance the saliency of the foreground objects. The proposed DA <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> FNet method was compared with the state-of-the-art object detection models, achieving excellent performance on the NWPU VHR-10, RSOD, and DOTA datasets.

MRDet: A Multihead Network for Accurate Rotated Object Detection in Aerial Images

Objects in aerial images usually have arbitrary orientations and are densely located over the ground, making them extremely challenge to be detected. Many of the recent developed methods attempt to solve these issues by estimating an extra orientation parameter and placing dense anchors, which will result in high model complexity and computational costs. In this article, we propose an arbitrary-oriented region proposal network (AO-RPN) to generate oriented proposals transformed from horizontal anchors. The AO-RPN is very efficient with only a few amounts of parameters increase than the original RPN. Furthermore, to obtain accurate bounding boxes, we decouple the detection task into multiple subtasks and propose a multihead network to accomplish them. Each head is specially designed to learn the features optimal for the corresponding task, which allows our network to detect objects accurately. We name it multihead rotated object detector (MRDet). We evaluate the performance of the proposed MRDet on two challenging benchmarks, i.e., DOTA and HRSC2016, and compare it with several state-of-the-art methods. Our method achieves very promising results, which clearly demonstrates its effectiveness. Code has been available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/qinr/MRDet</uri> .

Probability Differential-Based Class Label Noise Purification for Object Detection in Aerial Images

Modern object detection for aerial images requires numerous annotated data. However, the data annotation process inevitably introduces noise due to the bird’s eye view perspective of aerial images and the professional requirements of annotations. While recent noise-robust object detection methods achieved great success, the noise side-effect during the early training stage was still a problem. As demonstrated in this letter, noise during the early training stage will cumulatively affect the final performance. Based on the above observations, we propose a training strategy called correction maximization training to purify the noisy annotations and then train models. Specifically, we design a novel noise filter called the probability differential (PD) to identify and revise wrong labels. After purification, we train the detector with the revised dataset. Compared to existing works, the proposed method could be adapted in most modern object detectors (e.g., Faster RCNN and RetinaNet) and requires little hyperparameter tuning across different datasets and models. Extensive experiments on DOTA show that the proposed method achieves state-of-the-art results with both symmetric and asymmetric noise.