Small Object Detection Research Articles

Deer survey from drone thermal imagery using enhanced faster R-CNN based on ResNets and FPN

Deer surveys play an important role in the estimation of local ecological balance. In the Chitwan National Park of Nepal, the dense tree canopies and tall vegetation often obscure the presence of wild deer, which has a negative effect on the accurate population surveys of wild deer. UAVs equipped with infrared sensors have been increasingly used to monitor wild deer by capturing a lot of images. How to automatically recognize and obtain the number of deer objects from thermal images is becoming an important research topic. Due to the difference between thermal images and true-color images, as well as the variations in deer object sizes in these two types of images, current ready-to-use object detection models, designed for true-color imagery, are ill-suited for the task of detecting small deer objects within thermal imagery. In this paper, an enhanced Faster R-CNN was constructed to detect small deer objects from thermal images, in which a Feature Pyramid Network (FPN) based on a residual network is used to improve feature extraction for small deer objects and multi-scale feature map constrution for the subsequent region proposals searching, bounding box regression, and regions of interest (RoIs) classification. In addition, small-scaled anchor boxes and a multi-scale feature map selection criterion are devised to improve the detection accuracy of small objects. Finally, based on Faster R-CNN, FPN, and different residual networks including ResNet18, ResNet34, ResNet50, ResNet101, and ResNet152, we constructed five object detection models, and evaluated their detection performance by using COCO evaluation matrix. Under the condition of IoU≥0.5, the integration of Faster R-CNN, FPN, and ResNet18 demonstrated to perform better than others. Specifically, The COCO evaluation results revealed an Average Precision (AP) score of 91.6% for all deer objects. Small deer objects (area ≤ 200 pixels) achieved an AP score of 73.6%, medium deer objects (200 < area ≤ 400 pixels) demonstrated an AP score of 93.4%, and large deer objects (area > 400 pixels) achieved the highest AP score of 94.3%. Our research is helpful for effective wild deer monitoring and conservation and can be a valuable reference for the exploration of small object detection from low-resolution thermal images.

AMFF-Net: An Effective 3D Object Detector Based on Attention and Multi-Scale Feature Fusion

With the advent of autonomous vehicle applications, the importance of LiDAR point cloud 3D object detection cannot be overstated. Recent studies have demonstrated that methods for aggregating features from voxels can accurately and efficiently detect objects in large, complex 3D detection scenes. Nevertheless, most of these methods do not filter background points well and have inferior detection performance for small objects. To ameliorate this issue, this paper proposes an Attention-based and Multiscale Feature Fusion Network (AMFF-Net), which utilizes a Dual-Attention Voxel Feature Extractor (DA-VFE) and a Multi-scale Feature Fusion (MFF) Module to improve the precision and efficiency of 3D object detection. The DA-VFE considers pointwise and channelwise attention and integrates them into the Voxel Feature Extractor (VFE) to enhance key point cloud information in voxels and refine more-representative voxel features. The MFF Module consists of self-calibrated convolutions, a residual structure, and a coordinate attention mechanism, which acts as a 2D Backbone to expand the receptive domain and capture more contextual information, thus better capturing small object locations, enhancing the feature-extraction capability of the network and reducing the computational overhead. We performed evaluations of the proposed model on the nuScenes dataset with a large number of driving scenarios. The experimental results showed that the AMFF-Net achieved 62.8% in the mAP, which significantly boosted the performance of small object detection compared to the baseline network and significantly reduced the computational overhead, while the inference speed remained essentially the same. AMFF-Net also achieved advanced performance on the KITTI dataset.

Small object detection based on attention mechanism and enhanced network

Small object detection has a broad application prospect in image processing of unmanned aerial vehicles, autopilot and remote sensing. However, some difficulties exactly exist in small object detection, such as aggregation, occlusion and insufficient feature extraction, resulting in a great challenge for small object detection. In this paper, we propose an improved algorithm for small object detection to address these issues. By using the spatial pyramid to extract multi-scale spatial features and by applying the multi-scale channel attention to capture the global and local semantic features, the spatial pooling pyramid and multi-scale channel attention module (SPP-MSCAM) is constructed. More importantly, the fusion of the shallower layer with higher resolution and a deeper layer with more semantic information is introduced to the neck structure for improving the sensitivity of small object features. A large number of experiments on the VisDrone2019 dataset and the NWPU VHR-10 dataset show that the proposed method significantly improves the Precision, mAP and mAP50 compared to the YOLOv5 method. Meanwhile, it still preserves a considerable real-time performance. Undoubtedly, the improved network proposed in this paper can effectively alleviate the difficulties of aggregation, occlusion and insufficient feature extraction in small object detection, which would be helpful for its potential applications in the future.

DsP-YOLO: An anchor-free network with DsPAN for small object detection of multiscale defects

Industrial defect detection is of great significance to ensure the quality of industrial products. The surface defects of industrial products are characterized by multiple scales, multiple types, abundant small objects, and complex background interference. In particular, small object detection of multiscale defects under complex background interference poses significant challenges for defect detection tasks. How to improve the algorithm’s ability to detect industrial defects, especially in enhancing the detection capabilities of small-sized defects, while ensuring that the inference speed is not overly affected is a long-term prominent challenge. Aiming at achieving accurate and fast detection of industrial defects, this paper proposes an anchor-free network with DsPAN for small object detection. The core of this method is to propose a new idea i.e., feature enhancement in the feature fusion network for the feature information of small-size objects. Firstly, anchor-free YOLOv8 is adopted as the basic framework for detection to eliminate the affections of hyperparameters related to anchors, as well as to improve the detection capability of multi-scale and small-size defects. Secondly, considering the PAN path (top layer of neural networks for feature fusion) is more task-specific, we advocate that the underlying feature map of the PAN path is more vulnerable to small object detection. Hence, we in-depth study the PAN path and point out that the standard PAN will encounter several drawbacks caused by losing local details and position information in deep layer. As an alternative, we propose a lightweight and detail-sensitive PAN (DsPAN) for small object detection of multiscale defects by designing an attention mechanism embedded feature transformation module(LCBHAM) and optimizing the lightweight implementation. Our proposed DsPAN is very easy to be incorporated in YOLO series framework. The proposed method is evaluated on three public datasets, NEU-DET, PCB-DET, and GC10-DET. The mAP of the method is 80.4%, 95.8%, and 76.3%, which are 3.6%, 2.1%, and 3.9% higher than that of YOLOv8 and significantly higher than the state-of-the-art (SOTA) detection methods. Also, the method achieves the second-highest inference speed among the thirteen models tested. The results indicate that DsP-YOLO is expected to be used for real-time defect detection in industry.

ASSD-YOLO: a small object detection method based on improved YOLOv7 for airport surface surveillance

ASSD-YOLO: a small object detection method based on improved YOLOv7 for airport surface surveillance

ViT-DexiNet: a vision transformer-based edge detection operator for small object detection in SAR images

ABSTRACTThis paper introduces a novel edge detection operator called ‘vision transformer-based Dexinet (ViT-DexiNet)’ to address the challenges of detecting small objects in synthetic aperture radar (SAR) images. SAR images are typically impacted by strong multiplicative noise, making edge detection difficult. Existing traditional methods have limited spectral data preservation capabilities and often result in a loss of clarity and integrity of salient features in SAR images. The proposed ViT-DexiNet operator employs a series of interconnected layers to extract and refine salient edge features from SAR images. It utilizes a vision transformer self-attention layer to capture the pattern and structural details of image features crucial for determining edges. The extracted feature maps are then processed by the DexiNet architecture, which consists of a dense block, transfer block, and upsampling network. This architecture helps preserve edge information at different scales in deeper layers. The series of layered blocks generate edge maps, which are concatenated and averaged through smoothing to remove noise and enhance edge details in SAR images, resulting in a final high-quality edge map. To evaluate the proposed ViT-DexiNet method, both qualitative and quantitative analyses are conducted using standard edge detection operators such as Canny and Sobel. The empirical results demonstrate that the ViT-DexiNet surpasses baseline edge detection operators. The achieved values of proposed edge detection operator are 97.92%, 97.72%, 97.64% and 97.41%, respectively for the metrics accuracy, precision, recall and f1-score. The ViT-DexiNet offers high-quality edge maps, simplifying the interpretation of data for small object detection. Overall, the ViT-DexiNet method shows promise in overcoming the limitations of traditional approaches and improving the detection of edges in SAR images.

Deep learning-based object detection in maritime unmanned aerial vehicle imagery: Review and experimental comparisons

With the advancement of maritime unmanned aerial vehicles (UAVs) and deep learning technologies, the application of UAV-based object detection has become increasingly significant in the fields of maritime industry and ocean engineering. Endowed with intelligent sensing capabilities, the maritime UAVs enable effective and efficient maritime surveillance. To further promote the development of maritime UAV-based object detection, this paper provides a comprehensive review of challenges, relative methods, and UAV aerial datasets. Specifically, in this work, we first briefly summarize four challenges for object detection on maritime UAVs, i.e., object feature diversity, device limitation, maritime environment variability, and dataset scarcity. We then focus on computational methods to improve maritime UAV-based object detection performance in terms of scale-aware, small object detection, view-aware, rotated object detection, lightweight methods, and others. Next, we review the UAV aerial image/video datasets and propose a maritime UAV aerial dataset named MS2ship for ship detection. Furthermore, we conduct a series of experiments to present the performance evaluation and robustness analysis of object detection methods on maritime datasets. Eventually, we give the discussion and outlook on future works for maritime UAV-based object detection. The MS2ship dataset is available at https://github.com/zcj234/MS2ship.

Mitigate the scale imbalance via multi-scale information interaction in small object detection

Mitigate the scale imbalance via multi-scale information interaction in small object detection

Improved Object Detection Method Utilizing YOLOv7-Tiny for Unmanned Aerial Vehicle Photographic Imagery

In unmanned aerial vehicle photographs, object detection algorithms encounter challenges in enhancing both speed and accuracy for objects of different sizes, primarily due to complex backgrounds and small objects. This study introduces the PDWT-YOLO algorithm, based on the YOLOv7-tiny model, to improve the effectiveness of object detection across all sizes. The proposed method enhances the detection of small objects by incorporating a dedicated small-object detection layer, while reducing the conflict between classification and regression tasks through the replacement of the YOLOv7-tiny model’s detection head (IDetect) with a decoupled head. Moreover, network convergence is accelerated, and regression accuracy is improved by replacing the Complete Intersection over Union (CIoU) loss function with a Wise Intersection over Union (WIoU) focusing mechanism in the loss function. To assess the proposed model’s effectiveness, it was trained and tested on the VisDrone-2019 dataset comprising images captured by various drones across diverse scenarios, weather conditions, and lighting conditions. The experiments show that mAP@0.5:0.95 and mAP@0.5 increased by 5% and 6.7%, respectively, with acceptable running speed compared with the original YOLOv7-tiny model. Furthermore, this method shows improvement over other datasets, confirming that PDWT-YOLO is effective for multiscale object detection.

A lightweight small object detection algorithm based on improved YOLOv5 for driving scenarios

A lightweight small object detection algorithm based on improved YOLOv5 for driving scenarios

Small Foreign Object Detection in Automated Sugar Dispensing Processes Based on Lightweight Deep Learning Networks

This study addresses the challenges that conventional network models face in detecting small foreign objects on industrial production lines, exemplified by scenarios where a single piece of iron filing occupies approximately 0.002% of the image area. To tackle this, we introduce an enhanced YOLOv8-MeY model for detecting foreign objects on the surface of sugar bags. Firstly, the introduction of a 160 × 160-scale small object detection layer and integration of the Global Attention Mechanism (GAM) attention module into the feature fusion network (Neck) increased the network’s focus on small objects. This enhancement improved the network’s feature extraction and fusion capabilities, which ultimately increased the accuracy of small object detection. Secondly, the model employs the lightweight network GhostNet, replacing YOLOv8’s principal feature extraction network, DarkNet53. This adaptation not only diminishes the quantity of network parameters but also augments feature extraction capabilities. Furthermore, we substituted the Bottleneck in the C2f of the YOLOv8 model with the Spatial and Channel Reconstruction Convolution (SCConv) module, which, by mitigating the spatial and channel redundancy inherent in standard convolutions, reduced computational demands while elevating the performance of the convolutional network model. The model has been effectively applied to the automated sugar dispensing process in food factories, exhibiting exemplary performance. In detecting diverse foreign objects like 2 mm iron filings, 7 mm wires, staples, and cockroaches, the YOLOv8-MeY model surpasses the Faster R-CNN model and the contemporaneous YoloV8n model of equivalent parameter scale across six metrics: precision, recall, mAP@0.5, parameters, GFLOPs, and model size. Through 400 manual placement tests involving four types of foreign objects, our statistical results reveal that the model achieves a recognition rate of up to 92.25%. Ultimately, we have successfully deployed this model in automated sugar bag dispensing scenarios.

Detection of Small Objects in Side-Scan Sonar Images Using an Enhanced YOLOv7-Based Approach

Deep learning-based object detection methods have demonstrated remarkable effectiveness across various domains. Recently, there has been growing interest in applying these techniques to underwater environments. Conventional optical imaging methods face severe limitations when operating in underwater conditions, restricting their ability to identify objects with good visibility and at close distances. Consequently, side-scan sonar (SSS) has emerged as a common equipment choice for underwater detection due to its compatibility with the characteristics of sound waves in water. This paper introduces a novel method, termed the Enhanced YOLOv7-Based Approach, for detecting small objects in SSS images. Building upon the widely-adopted YOLOv7 method, the proposed approach incorporates several enhancements aimed at improving detection accuracy. First, a dedicated detection layer tailored for small objects is added to the original network architecture. Additionally, two attention mechanisms are integrated within the backbone and neck components of the network, respectively, to strengthen the network’s focus on object features. Finally, the network features are recombined based on the BiFPN structure. Experimental results demonstrate that the proposed method outperforms mainstream object detection algorithms. In comparison to the original YOLOv7 network, it achieves a Precision of 95.5%, indicating a significant improvement of 4.8%. Moreover, its Recall reaches 87.0%, representing an enhancement of 5.1%, while the mean Average Precision (mAP) at an IoU threshold of 0.5 (mAP@.5) reaches 86.9%, reflecting a 6.7% improvement. Furthermore, the mAP@.5:.95 reaches 55.1%, a 4.8% enhancement. Therefore, the method presented in this paper enhances the performance of YOLOv7 for object detection in SSS images, providing a fresh perspective on small object detection based on SSS images and contributing to the advancement of underwater detection techniques.

Boundary-aware small object detection with attention and interaction

Boundary-aware small object detection with attention and interaction

Architecture of the brain's visual system enhances network stability and performance through layers, delays, and feedback.

In the visual system of primates, image information propagates across successive cortical areas, and there is also local feedback within an area and long-range feedback across areas. Recent findings suggest that the resulting temporal dynamics of neural activity are crucial in several vision tasks. In contrast, artificial neural network models of vision are typically feedforward and do not capitalize on the benefits of temporal dynamics, partly due to concerns about stability and computational costs. In this study, we focus on recurrent networks with feedback connections for visual tasks with static input corresponding to a single fixation. We demonstrate mathematically that a network's dynamics can be stabilized by four key features of biological networks: layer-ordered structure, temporal delays between layers, longer distance feedback across layers, and nonlinear neuronal responses. Conversely, when feedback has a fixed distance, one can omit delays in feedforward connections to achieve more efficient artificial implementations. We also evaluated the effect of feedback connections on object detection and classification performance using standard benchmarks, specifically the COCO and CIFAR10 datasets. Our findings indicate that feedback connections improved the detection of small objects, and classification performance became more robust to noise. We found that performance increased with the temporal dynamics, not unlike what is observed in core vision of primates. These results suggest that delays and layered organization are crucial features for stability and performance in both biological and artificial recurrent neural networks.

Small object detection for autonomous driving under hazy conditions on mountain motorways

Small object detection for autonomous driving under hazy conditions on mountain motorways

Small object detection in unmanned aerial vehicle images using multi-scale hybrid attention

Small object detection in unmanned aerial vehicle images is always challenging due to the low resolution and the limited amount of information that they contain. Many feature enhancement effects have been introduced to improve the detection of small objects, but the extracted effective information is still insufficient, and redundant information interference is an issue. In this paper, we propose a new multi-scale hybrid attention based detector (MHA-YOLOv5), which integrates the similarity relationships between objects into you only look once version 5 (YOLOv5) for small object detection. Specifically, a novel multi-scale hybrid attention (MHA) structure is proposed to enhance the feature representation of small objects. This structure contains three modules: multi-scale attention (MsA), foreground enhancement module (FEM) and depthwise separable channel attention (DSCA). The MsA module is designed to build connections between large objects with abundant details and small objects with insufficient features on multiple scale features and capture the similarity relationships between objects. To reduce the interference of redundant information, the FEM is used to focus on the foreground features of multiple scale features, and the DSCA module is utilized to effectively extract multidimensional channel information. Sufficient experiments on the challenging VisDrone2019-DET, UAVDT and CARPK datasets demonstrate the effectiveness and superiority of the proposed approach. Specifically, compared with the performance of YOLOv5, MHA-YOLOv5 demonstrates a 2.82% mean average precision (mAP) improvement on the VisDrone2019-DET dataset, a 2.25% mAP improvement on the UAVDT dataset, and a 3.07% mAP improvement on the CARPK dataset.

SSDLiteX: Enhancing SSDLite for Small Object Detection

Object detection in many real applications requires the capability of detecting small objects in a system with limited resources. Convolutional neural networks (CNNs) show high performance in object detection, but they are not adequate to resource-limited environments. The combination of MobileNet V2 and SSDLite is one of the common choices in such environments, but it has a problem in detecting small objects. This paper analyzes the structure of SSDLite and proposes variations leading to small object detection improvement. The feature maps with the higher resolution are utilized more, and the base CNN is modified to have more layers in the high resolution. Experiments have been performed for the various configurations and the results show the proposed CNN, SSDLiteX, improves the detection accuracy AP of small objects by 1.5 percent points in the MS COCO data set.

Small Object Detection Algorithm for Underwater Organisms based on Improved Transformer

In view of the problem of difficult feature extraction and target missed detection, propose an improved transformer based underwater target detection algorithm. The proposed algorithm improves the linear embedding module of the Transformer to supplement the linear embedding information of small objects and builds the local relationship model of each layer feeling field, in order to achieve end-to-end accurate detection. An experiment on URPU underwater target detection data set is conducted. Compared with Swim Transformer, the proposed algorithm detection accuracy is 77.4%, 1. 5%, detection speed is 7 frame/s, 35.6%, the number is 30.4 MB, and compression is 84.1%. Experimental results have shown that the improved algorithm improves the performance of underwater small target detection

A Dense Small Object Detection Algorithm Based on a Global Normalization Attention Mechanism

To address the challenges of detecting a large number of objects and a high proportion of small objects in aerial drone imagery, we proposed an aerial dense small object detection algorithm called Global Normalization Attention Mechanism You Only Look Once (GNYL) based on the Global Normalization Attention Mechanism. In the backbone network of GNYL, we embedded a GNAM (Global Normalization Attention Mechanism) that explores channel attention features and spatial attention features from input features in a concatenated manner. It utilizes batch normalization’s scale factors to suppress irrelevant channels or pixels. Furthermore, the spatial attention sub-module introduces a three-dimensional arrangement with a multi-layer perceptron to reduce information loss and amplify global interaction representation. Finally, the computed attention weights are weighted to form the global normalized attention weights, which increases the utilization of effective information in input feature channels and spatial dimensions. We have optimized the backbone network, feature enhancement network, and detection heads to improve detection accuracy while ensuring a lightweight detection network. Specifically, we have added a small object detection layer to enhance the localization accuracy for the abundant small objects in aerial imagery. The algorithm’s performance was evaluated using the publicly available VisDrone2019 dataset. Compared to the baseline network YOLOv8l, GNYL achieved a 7.2% improvement in mAP0.5 and a 5.0% improvement in mAP0.95. Compared to CDNet, GNYL showed a 14.5% improvement in mAP0.5 and a 9.1% improvement in mAP0.95. These experimental results demonstrate the strong practicality of the GNYL object detection network for detecting dense small objects in the aerial imagery captured by unmanned aerial vehicles.

A Visual Fault Detection Algorithm of Substation Equipment Based on Improved YOLOv5

The development of artificial intelligence technology provides a new model for substation inspection in the power industry, and effective defect diagnosis can avoid the impact of substation equipment defects on the power grid and improve the reliability and stability of power grid operation. Aiming to combat the problem of poor recognition of small targets due to large differences in equipment morphology in complex substation scenarios, a visual fault detection algorithm of substation equipment based on improved YOLOv5 is proposed. Firstly, a deformable convolution module is introduced into the backbone network to achieve adaptive learning of scale and receptive field size. Secondly, in the neck of the network, a simple and effective BiFPN structure is used instead of PANet. The multi-level feature combination of the network is adjusted by a floating adaptive weighted fusion strategy. Lastly, an additional small object detection layer is added to detect shallower feature maps. Experimental results demonstrate that the improved algorithm effectively enhances the performance of power equipment and defect recognition. The overall recall rate has increased by 7.7%, precision rate has increased by nearly 6.3%, and mAP@0.5 has improved by 4.6%. The improved model exhibits superior performance.