Small Size Objects Research Articles

SPD-YOLOv8: an small-size object detection model of UAV imagery in complex scene

SPD-YOLOv8: an small-size object detection model of UAV imagery in complex scene

C3TB-YOLOv5: integrated YOLOv5 with transformer for object detection in high-resolution remote sensing images

ABSTRACT In the realm of object detection from high-resolution remote sensing images (HRRSIs), the existing YOLOv5 methods encounter several challenges, including dense object arrangements, small object sizes, and complex backgrounds. To tackle these challenges, we propose a novel approach called C3TB-YOLOv5, which combines traditional YOLOv5 with the Transformer model to detect objects in HRRSIs. Unlike conventional YOLOv5 methods that primarily focus on capturing local information from remote sensing scenes, our C3TB-YOLOv5 method incorporates global information through the introduction of a new C3TB module. This module, based on the Transformer multi-head attention mechanism (AM), consists of two branches that extract local and global information from feature maps. By integrating these branches and establishing long-range relationships, our method successfully detects densely arranged small objects in HRRSIs. Furthermore, to improve the accuracy of tiny object detection, a novel detection head has been developed to effectively utilize the unused C3 module, thereby preventing the loss of fine-grained textures and positional features. In addition, we integrate an enhanced SimAM, namely Sim-GMP, into the model to adjust the focus across varying regions, effectively distinguishing the features of interested objects from complex backgrounds. Finally, to address the problem of sample imbalance in remote sensing object detection, the most recent Wise-IoU v3 loss function is employed to improve the accuracy of anchor box predictions for objects. To maintain a high object detection speed, the most critical C3 modules are substituted with the proposed C3TB module for the purpose of striking a good balance between object detection accuracy and model lightweight. Extensive experiments conducted on two remote sensing datasets of NWPU VHR-10 and VisDrone 2019 demonstrates that our method achieves superior object detection performance than state-of-the-art methods.

REPS: Rotation equivariant Siamese network enhanced by probability segmentation for satellite video tracking

Satellite video is an emerging surface observation data that has drawn increasing interest due to its potential in spatiotemporal dynamic analysis. Single object tracking of satellite videos allows the continuous acquisition of the positions and ranges of objects and establishes the correspondences in the video sequence. However, small-sized objects are vulnerable to rotation and non-rigid deformation. Moreover, the horizontal bounding box of most trackers has difficulty in providing accurate semantic representations such as object position, orientation, and spatial distribution. In this article, we propose a unified framework, named rotation equivariant Siamese network enhanced by probability segmentation (REPS), to enhance the tracking accuracy and semantic representations simultaneously. First, to deal with the inconsistency of representations, we design a rotation equivariant (RE) Siamese network architecture to detect the rotation variations of objects right from the start frame, achieving the RE tracking. Second, a pixel-level (PL) refinement is proposed to refine the spatial distribution of objects. In addition, we proposed an adaptive Gaussian fusion that synergizes tracking and segmentation results to obtain compact outputs for satellite object representations. Extensive experiments on satellite videos demonstrate the superiority of the proposed approach. The code will be available at https://github.com/YZCU/REPS

DeLiVoTr: Deep and light-weight voxel transformer for 3D object detection

The image-based backbone (feature extraction) networks downsample the feature maps not only to increase the receptive field but also to efficiently detect objects of various scales. The existing feature extraction networks in LiDAR-based 3D object detection tasks follow the feature map downsampling similar to image-based feature extraction networks to increase the receptive field. But, such downsampling of LiDAR feature maps in large-scale autonomous driving scenarios hinder the detection of small size objects, such as pedestrians. To solve this issue we design an architecture that not only maintains the same scale of the feature maps but also the receptive field in the feature extraction network to aid for efficient detection of small size objects. We resort to attention mechanism to build sufficient receptive field and we propose a Deep and Light-weight Voxel Transformer (DeLiVoTr) network with voxel intra- and inter-region transformer modules to extract voxel local and global features respectively. We introduce DeLiVoTr block that uses transformations with expand and reduce strategy to vary the width and depth of the network efficiently. This facilitates to learn wider and deeper voxel representations and enables to use not only smaller dimension for attention mechanism but also a light-weight feed-forward network, facilitating the reduction of parameters and operations. In addition to model scaling, we employ layer-level scaling of DeLiVoTr encoder layers for efficient parameter allocation in each encoder layer instead of fixed number of parameters as in existing approaches. Leveraging layer-level depth and width scaling we formulate three variants of DeLiVoTr network. We conduct extensive experiments and analysis on large-scale Waymo and KITTI datasets. Our network surpasses state-of-the-art methods for detection of small objects (pedestrians) with an inference speed of 20.5 FPS.

FVCNet: Detection obstacle method based on feature visual clustering network in power line inspection

AbstractPower line inspection is an important means to eliminate hidden dangers of power lines. It is a difficult research problem how to solve the low accuracy of power line inspection based on deep neural network (DNN) due to the problems of multi‐view‐shape, small‐size object. In this paper, an automatic detection model based on Feature visual clustering network (FVCNet) for power line inspection is established. First, an unsupervised clustering method for power line inspection is proposed, and applied to construct a detection model which can recognize multi‐view‐shape objects and enhanced object features. Then, the bilinear interpolation method is used to Feature enhancement method, and the enhanced high‐level semantics and low‐level semantics are fused to solve the problems of small object size and single sample. In this paper, FVCNet is applied to the MS‐COCO 2017 data set and self‐made power line inspection data set, and the test accuracy is increased to 61.2% and 82.0%, respectively. Compared with other models, especially for the categories that are greatly affected by multi‐view‐shape, the test accuracy has been improved significantly.

Learning Point Processes and Convolutional Neural Networks for Object Detection in Satellite Images

Convolutional neural networks (CNN) have shown great results for object-detection tasks by learning texture and pattern-extraction filters. However, object-level interactions are harder to grasp without increasing the complexity of the architectures. On the other hand, Point Process models propose to solve the detection of the configuration of objects as a whole, allowing the factoring in of the image data and the objects’ prior interactions. In this paper, we propose combining the information extracted by a CNN with priors on objects within a Markov Marked Point Process framework. We also propose a method to learn the parameters of this Energy-Based Model. We apply this model to the detection of small vehicles in optical satellite imagery, where the image information needs to be complemented with object interaction priors because of noise and small object sizes.

M-DETR: Multi-scale DETR for Optical Music Recognition

Optical Music Recognition (OMR) is an important way to digitize score images and has broad application prospects in fields such as the storage of music documents, music education and digital creation. As a new paradigm for object detection, DETR (detection transformer) has the ability to associate contextual information, which can be exploited to resolve the OMR task. However, the original DETR does not fit OMR well due to its high computational complexity and numerous parameters. To address the DETR defects and improve the recognition accuracy of OMR, we propose a novel multi-scale DETR (M-DETR) with a multi-scale feature fusion mechanism and improved attention mechanisms. First, a new multi-scale feature fusion mechanism is designed to let the backbone network of M-DETR get rich multi-scale information. Then, a key-region attention mechanism is incorporated based on the character that the key information is concentrated on a score image. Finally, the pre-context attention mechanism is introduced to make better use of the contextual association between recognition notes in music scores. Experiment results show that M-DETR achieves recognition accuracy of 90.6% for 7 typical small-sized notes, which is better than Faster R-CNN and YOLO v5, and the improvement rate is 10.02% compared to the original DETR algorithm. The results indicate that M-DETR is an effective way for the OMR task, which also provides a new solution for the detection of small-sized objects with contextual association.

Application of a Three-Stage Procedure for Extracting a Small-Sized Anchor Object on a Noisy Image

The purpose of the article is to develop a three-stage procedure for identifying an anchor object in a noisy current image. This goal is achieved by determining the sampling threshold levels at which the greatest similarity of the compared images is ensured; development of a procedure for refining the maximum of the decision function by quantizing the current image. The solution to the first problem is based on the formation of a correlation field of radio brightness temperatures and the choice of a sampling threshold. It is proposed to use the cross-correlation coefficient as a criterion for the degree of image matching. The effectiveness of the procedure for selecting a fragment of a reference image is assessed based on the criterion of the probability of selecting a fragment of a reference image. It is shown that image noise can lead to a decrease in the probability of selecting a fragment of a reference image, down to 0.4. It is proposed to refine the maximum of the decision function based on the iteration method. The most significant results are the obtained dependences of the probability of choosing a fragment of the reference image on the threshold value and the signal-to-noise ratio, as well as the analytical relation for the asymptote of this dependence. The novelty of the work lies in the fact that the procedure for forming the decisive function has been further developed. This will significantly improve the operating efficiency of unmanned aerial vehicles, especially in conditions of interference.

Vegetable disease detection using an improved YOLOv8 algorithm in the greenhouse plant environment

This study introduces YOLOv8n-vegetable, a model designed to address challenges related to imprecise detection of vegetable diseases in greenhouse plant environment using existing network models. The model incorporates several improvements and optimizations to enhance its effectiveness. Firstly, a novel C2fGhost module replaces partial C2f. with GhostConv based on Ghost lightweight convolution, reducing the model’s parameters and improving detection performance. Second, the Occlusion Perception Attention Module (OAM) is integrated into the Neck section to better preserve feature information after fusion, enhancing vegetable disease detection in greenhouse settings. To address challenges associated with detecting small-sized objects and the depletion of semantic knowledge due to varying scales, an additional layer for detecting small-sized objects is included. This layer improves the amalgamation of extensive and basic semantic knowledge, thereby enhancing overall detection accuracy. Finally, the HIoU boundary loss function is introduced, leading to improved convergence speed and regression accuracy. These improvement strategies were validated through experiments using a self-built vegetable disease detection dataset in a greenhouse environment. Multiple experimental comparisons have demonstrated the model's effectiveness, achieving the objectives of improving detection speed while maintaining accuracy and real-time detection capability. According to experimental findings, the enhanced model exhibited a 6.46% rise in mean average precision (mAP) over the original model on the self-built vegetable disease detection dataset under greenhouse conditions. Additionally, the parameter quantity and model size decreased by 0.16G and 0.21 MB, respectively. The proposed model demonstrates significant advancements over the original algorithm and exhibits strong competitiveness when compared with other advanced object detection models. The lightweight and fast detection of vegetable diseases offered by the proposed model presents promising applications in vegetable disease detection tasks.

A novel small object detection algorithm for UAVs based on YOLOv5

Due to the advances in deep learning, artificial intelligence is widely utilized in numerous areas. Technologies frontier, including computer vision, represented by object detection, have endowed unmanned aerial vehicles (UAVs) with autonomous perception, analysis, and decision-making capabilities. UAVs extensively used in numerous fields including photography, industry and agriculture, surveillance, disaster relief, and play an important role in real life. However, current object detection algorithms encountered challenges when it came to detecting small objects in images captured by UAVs. The small size of the objects, with high density, low resolution, and few features make it difficult for the algorithms to achieve high detection accuracy and are prone to miss and false detections especially when detecting small objects. For the case of enhancing the performance of UAV detection on small objects, a novel small object detection algorithm for UAVs adaptation based on YOLOv5s (UA-YOLOv5s) was proposed. (1) To achieve effective small-sized objects detection, a more accurate small object detection (MASOD) structure was adopted. (2) To boost the detection accuracy and generalization ability of the model, a multi-scale feature fusion (MSF) approach was proposed, which fused the feature information of the shallow layers of the backbone and the neck. (3) To enhance the model stability properties and feature extraction capability, a more efficient and stable convolution residual Squeeze-and-Excitation (CRS)module was introduced. Compared with the YOLOv5s, mAP@0.5 was achieved an impressive improvement of 7.2%. Compared with the YOLOv5l, mAP@0.5 increased by 1.0%, and GFLOPs decreased by 69.1%. Compared to the YOLOv3, mAP@0.5 decreased by 0.2% and GFLOPs by 78.5%. The study’s findings demonstrated that the proposed UA-YOLOv5s significantly enhanced the object detection performance of UAVs campared to the traditional algorithms.

Intelligent detection of hidden distresses in asphalt pavement based on GPR and deep learning algorithm

Hidden distresses of asphalt pavement are hidden dangers in the service of expressways. Efficient and accurate detection of hidden distresses is of great importance for maintaining and managing the expressways suffering from them. In this study, the intelligent detection technology of asphalt pavement hidden distresses was studied using the object detection algorithm and ground penetrating radar (GPR) image data. A GPR image data set was established to train and test the object detection model. And the distresses samples in the data set include settlement, hidden cracks and loosening. According to the distribution of hidden distresses in GPR image data, a two-way object detection model for targets of different scales was proposed, and each part of the two-way model was optimized and improved based on the basic model. At the same time, a copy-and-paste data enhancement scheme was designed to expand and balance the number of distress data samples. The final model greatly improved the detection accuracy of asphalt pavement hidden distresses. The results of the model test showed that the two-way object detection model proposed in this paper can effectively improve the detection accuracy of small-sized distress objects. Compared to the basic model, the final model had an increase of 17.9% in the average precision (AP) of small objects and a 9.9% increase in the comprehensive index AP (0.5: 0.95). The results of the ablation experiment show that the optimization methods for each part of the model improved the comprehensive detection effect of the model. When compared with other object detection models, the final model was superior to other models in all AP indicators, and had obvious advantages in the detection accuracy of the three types of hidden distresses. The confusion matrix of the test results of the final model showed that the recall rate of the final model for the three types of hidden distresses was above 96%. Thus, applying the final model established in this study to actual road detection can effectively improve the efficiency of distress detection in GPR images.

One-Stage Small Object Detection Using Super-Resolved Feature Map for Edge Devices

Despite the achievements of deep neural-network-based object detection, detecting small objects in low-resolution images remains a challenging task due to limited information. A possible solution to alleviate the issue involves integrating super-resolution (SR) techniques into object detectors, particularly enhancing feature maps for small-sized objects. This paper explores the impact of high-resolution super-resolved feature maps generated by SR techniques, especially for a one-stage detector that demonstrates a good compromise between detection accuracy and computational efficiency. Firstly, this paper suggests the integration of an SR module named feature texture transfer (FTT) into the one-stage detector, YOLOv4. Feature maps from the backbone and the neck of vanilla YOLOv4 are combined to build a super-resolved feature map for small-sized object detection. Secondly, it proposes a novel SR module with more impressive performance and slightly lower computation demand than the FTT. The proposed SR module utilizes three input feature maps with different resolutions to generate a super-resolved feature map for small-sized object detection. Lastly, it introduces a simplified version of an SR module that maintains similar performance while using only half the computation of the FTT. This attentively simplified module can be effectively used for real-time embedded systems. Experimental results demonstrate that the proposed approach substantially enhances the detection performance of small-sized objects on two benchmark datasets, including a self-built surveillance dataset and the VisDrone2019 dataset. In addition, this paper employs the proposed approach on an embedded system with a Qualcomm QCS610 and demonstrates its feasibility for real-time operation on edge devices.

Detection algorithm for dense small objects in high altitude image

The high altitude images captured by drones often contain small-sized objects, which pose several challenges including significant scale variations and dense clustering. To tackle these challenges in small object detection, this paper presents EM-YOLO (Efficient Switching Network and Multi-Scale Feature Fusion Reinforced YOLO for Small Object Detection), an enhanced algorithm based on the YOLOv5s architecture. Firstly, to enhance the network's adaptability to varying object feature scales, the EM-YOLO redesigns the feature extraction backbone network by using different dilation rates in convolutions within the backbone. This flexible structure allows for improved extraction of object features, enhancing the network's feature extraction capability. Secondly, additional detection layer for small objects and combines it with the dynamic detection head. This combination addresses the issue of small object information loss due to multiple down-sampling operations on the feature maps. Finally, Focal-EIoU Loss is employed to enhance regression accuracy in EM-YOLO. Extensive experiments were conducted on the VisDrone2019-DET dataset. The results of these experiments demonstrate that the improved algorithm achieved a significant increase in performance. Specifically, the algorithm achieves 43.5% and 25.1% on the mAP0.5 and mAP0.5:0.95 metrics, which are 13.9% and 8.9% higher than YOLOv5s algorithm, respectively. Furthermore, comparative experiments with other mainstream algorithms indicate that EM-YOLO outperforms them in the task of small object detection.

Semantic and Correlation Disentangled Graph Convolutions for Multilabel Image Recognition.

Multilabel image recognition (MLR) aims to annotate an image with comprehensive labels and suffers from object occlusion or small object sizes within images. Although the existing works attempt to capture and exploit label correlations to tackle these issues, they predominantly rely on global statistical label correlations as prior knowledge for guiding label prediction, neglecting the unique label correlations present within each image. To overcome this limitation, we propose a semantic and correlation disentangled graph convolution (SCD-GC) method, which builds the image-specific graph and employs graph propagation to reason the labels effectively. Specifically, we introduce a semantic disentangling module to extract categorywise semantic features as graph nodes and develop a correlation disentangling module to extract image-specific label correlations as graph edges. Performing graph convolutions on this image-specific graph allows for better mining of difficult labels with weak visual representations. Visualization experiments reveal that our approach successfully disentangles the dominant label correlations existing within the input image. Through extensive experimentation, we demonstrate that our method achieves superior results on the challenging Microsoft COCO (MS-COCO), PASCAL visual object classes (PASCAL-VOC), NUS web image dataset (NUS-WIDE), and Visual Genome 500 (VG-500) datasets. Code is available at GitHub: https://github.com/caigitrepo/SCDGC.

Implementation and Application of a Regional Fireball Monitoring Network Based on All-sky Camera Networking

Fireball monitoring network is the main equipment for monitoring impact from small-size near-Earth objects (NEOs), and determining meteorite fall locations. In this paper, a monitoring system using a multi-station distributed all-sky video camera network is proposed, and a regional prototype system has been constructed in Jiangsu and surrounding areas. The main processes of fireball monitoring were implemented, including network control, video capture, data processing, trajectory determination, and orbit calculation. After running for 1 yr, the results show that the limiting apparent magnitude - 1.0 for meteors in the video, and the absolute magnitude exhaustive detection regime can reach around - 2.5. The flux of fireballs is found to be 2.68×10−7 km−2· h−1. The percentages of streams and sporadic meteors are 46% and 54%, respectively, and the fraction of cometary orbits equals 72.9%, while for asteroidal-type it is 27.1%. The statistical results are close to those of the international meteor monitoring networks, which verified the monitoring capability of the network system in practical operation.

System of high-precision trajectory measurements for testing high-speed small-size objects

System of high-precision trajectory measurements for testing high-speed small-size objects

An airport apron ground service surveillance algorithm based on improved YOLO network

<abstract> <p>To assure operational safety in the airport apron area and track the process of ground service, it is necessary to analyze key targets and their activities in the airport apron surveillance videos. This research shows an activity identification algorithm for ground service objects in an airport apron area and proposes an improved YOLOv5 algorithm to increase the precision of small object detection by introducing an SPD-Conv (spath-to-depth-Conv) block in YOLOv5's backbone layer. The improved algorithm can efficiently extract the information features of small-sized objects, medium-sized objects, and moving objects in large scenes, and it achieves effective detection of activities of ground service in the apron area. The experimental results show that the detection average precision of all objects is more than 90%, and the whole class mean average precision (mAP) is 98.7%. At the same time, the original model was converted to TensorRT and OpenVINO format models, which increased the inference efficiency of the GPU and CPU by 55.3 and 137.1%, respectively.</p> </abstract>

DATran: Dual Attention Transformer for Multi-Label Image Classification

Multi-label image classification is a fundamental yet challenging task, which aims to predict the labels associated with a given image. Most of previous methods directly exploit the high-level features from the last layer of convolutional neural network for classification. However, these methods cannot obtain global features due to the limited size of convolutional kernels, and they fail to extract multi-scale features to effectively recognize small-scale objects in the images. Recent studies exploit the graph convolution network to model the label correlations for boosting the classification performance. Despite substantial progress, these methods rely on manually pre-defined graph structures. Besides, they ignore the associations between semantic labels and image regions, and do not fully explore the spatial context of images. To address above issues, we propose a novel Dual Attention Transformer (DATran) model, which adopts a dual-stream architecture that simultaneously learns spatial and channel correlations from multi-label images. Firstly, in order to solve the problem that current methods are difficult to recognize small-size objects, we develop a new multi-scale feature fusion (MSFF) module to generate multi-scale feature representation by jointly integrating both high-level semantics and low-level details. Secondly, we design a prior-enhanced spatial attention (PSA) module to learn the long-range correlation between objects from different spatial positions in images to enhance the model performance. Thirdly, we devise a prior-enhanced channel attention (PCA) module to capture the inter-dependencies between different channel maps, thus effectively improving the correlation between semantic categories. It is worth noting that PSA module and PCA module complement and promote each other to further augment the feature representations. Finally, the outputs of these two attention modules are fused to obtain the final features for classification. Performance evaluation experiments are conducted on MS-COCO 2014, PASCAL VOC 2007 and VG-500 datasets, demonstrating that DATran model achieves better performance than current state-of-the-art models.

An Efficient Rep-Style Gaussian–Wasserstein Network: Improved UAV Infrared Small Object Detection for Urban Road Surveillance and Safety

Owing to the significant application potential of unmanned aerial vehicles (UAVs) and infrared imaging technologies, researchers from different fields have conducted numerous experiments on aerial infrared image processing. To continuously detect small road objects 24 h/day, this study proposes an efficient Rep-style Gaussian–Wasserstein network (ERGW-net) for small road object detection in infrared aerial images. This method aims to resolve problems of small object size, low contrast, few object features, and occlusions. The ERGW-net adopts the advantages of ResNet, Inception net, and YOLOv8 networks to improve object detection efficiency and accuracy by improving the structure of the backbone, neck, and loss function. The ERGW-net was tested on a DroneVehicle dataset with a large sample size and the HIT-UAV dataset with a relatively small sample size. The results show that the detection accuracy of different road targets (e.g., pedestrians, cars, buses, and trucks) is greater than 80%, which is higher than the existing methods.

ETSR-YOLO: An improved multi-scale traffic sign detection algorithm based on YOLOv5.

In the application of driverless technology, current traffic sign recognition methods are susceptible to the influence of ambient light interference, target size changes and complex backgrounds, resulting in reduced recognition accuracy. To address these challenges, this study introduces an optimisation algorithm called ETSR-YOLO, which is based on the YOLOv5s algorithm. First, this study improves the path aggregation network (PANet) of YOLOv5s to enhance multi-scale feature fusion by generating an additional high-resolution feature layer to improve the recognition of YOLOv5s for small-sized objects. Second, the study introduces two improved C3 modules that aim to suppress background noise interference and enhance the feature extraction capabilities of the network. Finally, the study uses the Wise-IoU (WIoU) function in the post-processing stage to improve the learning ability and robustness of the algorithm to different samples. The experimental results show that ETSR-YOLO improves mAP@0.5 by 6.6% on the Tsinghua-Tencent 100K (TT100K) dataset and by 1.9% on the CSUST Chinese Traffic Sign Detection Benchmark 2021 (CCTSDB2021) dataset. In the experiments conducted on the embedded computing platform, ETSR-YOLO demonstrates a short average inference time, thereby affirming its capability to deliver dependable traffic sign detection for intelligent vehicles operating in real-world traffic scenes. The source code and test results of the models used in this study are available at https://github.com/cbrook16/ETSR-YOLO.