Object Detection Network Research Articles

A novel edge intelligence-based solution for safer footpath navigation of visually impaired using computer vision

Navigating through a tactile paved footpath surrounded by various sizes of static and dynamic obstacles is one of the biggest impediments visually impaired people face, especially in Dhaka, Bangladesh. This problem is important to address, considering the number of accidents in such densely populated footpaths. We propose a novel deep-edge solution using Computer Vision to make people aware of the obstacles in the vicinity and reduce the necessity of a walking cane. This study introduces a diverse novel tactile footpath dataset of Dhaka covering different city areas. Additionally, existing state-of-the-art deep neural networks for object detection have been fine-tuned and investigated using this dataset. A heuristic-based breadth-first navigation algorithm (HBFN) is developed to provide navigation directions that are safe and obstacle-free, which is then deployed in a smartphone application that automatically captures images of the footpath ahead to provide real-time navigation guidance delivered by speech. The findings from this study demonstrate the effectiveness of the object detection model, YOLOv8s, which outperformed other benchmark models on this dataset, achieving a high mAP of 0.974 and an F1 score of 0.934. The model’s performance is analyzed after quantization, reducing its size by 49.53% while retaining 98.97% of the original mAP.

CTAFFNet: CNN–Transformer Adaptive Feature Fusion Object Detection Algorithm for Complex Traffic Scenarios

As the core technology of an environmental perception system, object detection has received more and more attention and has become a hot research direction for intelligent driving vehicles. The CNN–Transformer hybrid model has poor generalization ability, making it difficult to meet the detection requirements for small objects in complex scenes. We propose a novel convolutional neural network (CNN)–Transformer Adaptive Feature Fusion Network (CTAFFNet) for object detection. First, we design a Local–Global Feature Fusion unit known as the Convolutional Transformation Adaptive Fusion Kernel (CTAFFK), which is integrated into CTAFFNet. The CTAFFK kernel utilizes two branches, namely CNN and Transformer, to extract local and global features from the image, and adaptively fuses the features from both branches. Subsequently, we develop an adaptive feature fusion strategy that combines local high-frequency and global low-frequency features to obtain comprehensive feature information. Finally, CTAFFNet employs an encoder–decoder structure to facilitate the flow of fused local–global information between different stages, ensuring the model’s generalization capabilities. Results from the experiment conducted on the large and challenging KITTI dataset demonstrate the effectiveness and efficiency of the proposed network. Compared with other mainstream networks, it achieves an average precision of 91.17%, particularly excelling in the detection of small objects at longer distances with a remarkable 70.18% accuracy, while also providing real-time detection capabilities.

Leveraging Perspective Transformation for Enhanced Pothole Detection in Autonomous Vehicles.

Road conditions, often degraded by insufficient maintenance or adverse weather, significantly contribute to accidents, exacerbated by the limited human reaction time to sudden hazards like potholes. Early detection of distant potholes is crucial for timely corrective actions, such as reducing speed or avoiding obstacles, to mitigate vehicle damage and accidents. This paper introduces a novel approach that utilizes perspective transformation to enhance pothole detection at different distances, focusing particularly on distant potholes. Perspective transformation improves the visibility and clarity of potholes by virtually bringing them closer and enlarging their features, which is particularly beneficial given the fixed-size input requirement of object detection networks, typically significantly smaller than the raw image resolutions captured by cameras. Our method automatically identifies the region of interest (ROI)-the road area-and calculates the corner points to generate a perspective transformation matrix. This matrix is applied to all images and corresponding bounding box labels, enhancing the representation of potholes in the dataset. This approach significantly boosts detection performance when used with YOLOv5-small, achieving a 43% improvement in the average precision (AP) metric at intersection-over-union thresholds of 0.5 to 0.95 for single class evaluation, and notable improvements of 34%, 63%, and 194% for near, medium, and far potholes, respectively, after categorizing them based on their distance. To the best of our knowledge, this work is the first to employ perspective transformation specifically for enhancing the detection of distant potholes.

CP-RDM: a new object detection algorithm for casting and pouring robots

Abstract Automating the casting sector heavily relies on pivotal technology for object detection in pouring robots. A sophisticated algorithm designed to identify and locate target pouring holes in intricate casting workshops is crucial for advancing the intelligence of the casting process. However, the workshop environment for pouring is generally challenging, with uneven lighting, varying sizes of pouring holes, and significant occlusion in the target area, all impacting the accuracy of target detection tasks.
To overcome these challenges, this paper proposes enhancing the YOLOv8s algorithm for object detection in pouring robots. Firstly, to address the issue of different scales in pouring holes, a Multi-Scale Residual Channel and Spatial Information Fusion Module (MRCS) is designed to aggregate channel and spatial information, thereby enhancing the feature extraction capability of the model. The proposed enhancement is validated on the Pascal VOC dataset. Secondly, a SimAM attention mechanism is added at the end of the backbone network to focus the object detection network more on the positional region of the pouring hole. Importantly, this addition does not introduce extra parameters or computational burden to the model.
Finally, in the detection part of the model, the detection head from the RT-DETR model is introduced. This combination of real-time detection capability from YOLO and deep feature extraction capability from RT-DETR enhances the detection accuracy of the model while ensuring real-time performance. Experimental results on the updated pouring hole dataset reveal that, with only a slight increase in parameters, the proposed model achieves a 2.5% and 3.5% improvement in mAP@0.5 and F1-Score, respectively, compared to the baseline algorithm YOLOv8s. Precision (P) is enhanced by 1.8%, recall (R) by 3.5%, and PFS reaches 110, meeting the requirements for real-time pouring in pouring robots.

Pollen detection through integrated microfluidics and smartphone-driven deep learning systems

The presence of atmospheric particulates, including bioaerosols such as pollen, significantly impacts air quality and poses potential health risks. Therefore, accurate detection and categorization of these particulates are crucial for evaluating their health implications. This study introduces an innovative, automated system for analyzing atmospheric particulates, combining microfluidic technology with a smartphone-based photonic detection platform. The system integrates a miniaturized optical microscopy module and a consumer-grade high-frame-rate camera, creating an affordable, accessible solution priced at only 50 USD. Such cost-effectiveness promotes the widespread adoption of the technology and facilitates community-driven environmental monitoring. The study developed a compact, rapid, real-time, and continuous air micro-particle sampling and detection platform. The integration of an automatic collection device with a microfluidic detection chip enables continuous sampling and detection. A lightweight object detection network, designated as “PollenDet,” was trained, achieving a mean average precision (mAP) of 94.6%. This technology provides essential insights into the dynamics of airborne particulates, thereby enhancing our understanding of their overall impact.

RT-SNDETR: real-time supernova detection via end-to-end image transformers

ABSTRACT In large-scale astronomical surveys, traditional supernova detection pipelines rely on complex and relatively inefficient image differencing techniques. This paper proposes an end-to-end deep-learning supernova detection network, the Real-Time SuperNova DEtection TRansformer (RT-SNDETR). This network partially replaces traditional pipelines by integrating image differencing, source detection, and Real-bogus classification, achieving a speed 51.49 times that of the fastest image differencing method, SFFT. Additionally, it remains competitive with methods like YOLO v8, offering a well-balanced trade-off between speed and accuracy. Experimental results highlight RT-SNDETR’s superior performance, with an average precision(AP) of 96.30 per cent on synthetic samples and 76.60 per cent on real supernova samples. It significantly outperforms other detection networks, including RT-DETR (+5.6 per cent AP on synthetic/+5.1 per cent AP on real samples) and Cascade R-CNN (+8.9 per cent AP on synthetic/ +28.6 per cent AP on real samples). The incorporation of CycleGAN-based data generation methods plays a significant role in enhancing RT-SNDETR’s performance. These methods simulate realistic PSF variations, enabling the object detection network to learn more robust features and improving its generalization to real supernovae data. Additionally, by integrating unsupervised domain adaptation techniques, RT-SNDETR achieves an AP of 81.70 per cent on real SDSS supernova survey samples. This study demonstrates RT-SNDETR’s potential to significantly enhance both the speed and accuracy of supernova detection, making it a highly effective solution for large-scale astronomical surveys.

UAV image object detection based on self-attention guidance and global feature fusion

Unmanned aerial vehicle (UAV) image object detection has garnered considerable attentions in fields such as Intelligent transportation, urban management and agricultural monitoring. However, it suffers from key challenges of the deficiency in multi-scale feature extraction and the inaccuracy when processing complex scenes and small-sized targets in practical applications. To address this challenge, we propose a novel UAV image object detection network based on self-attention guidance and global feature fusion, named SGGF-Net. First, in order to optimizing feature extraction in global perspective and enhancing target localization precision, the global feature extraction module (GFEM) is introduced by exploiting the self-attention mechanism to capture and integrate long-range dependencies within images. Second, a normal distribution-based prior assigner (NDPA) is developed by measuring the resemblance between ground truth and the priors, which improves the precision of target position matching and thus handle the problem of inaccurate localization of small targets. Furthermore, we design an attention-guided ROI pooling module (ARPM) via a deep fusion strategy of multilevel features for optimizing the integration of multi-scale features and improving the quality of feature representation. Finally, experimental results demonstrate the effectiveness of the proposed SGGF-Net approach.

Urban traffic tiny object detection via attention and multi-scale feature driven in UAV-vision

The unmanned aerial vehicle (UAV) city patrol is of great significance in ensuring the safety of residents’ lives and properties, as well as maintaining the normal operation of the city. However, the detection of UAV images faces challenges such as numerous small-scale objects, complex backgrounds, and high requirements for detection speed. In response to these issues, we introduce a Real-time Small Object Detection network in UAV-vision (RTS-Net), tailored for UAV patrols. Initially, we introduce a multiscale feature fusion module (MFFM) designed to augment the expressiveness of features across scales, thereby enhancing the detection of smaller objects. Subsequently, leveraging attention mechanisms, we present the coordinated attention detection module (CADM), which bolsters the detection model’s ability to accurately segregate objects from the background in expansive, complex scenarios. Lastly, a lightweight real-time feature extraction module (RFEM) is crafted to diminish model computational complexity and boost inference speed. On the UAV road patrol image dataset we constructed, our proposed method attains a detection accuracy of 89.9%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\%$$\\end{document} mAP, breaking previous records. It surpasses all prevailing detection methods, particularly for small-scale objects. Simultaneously, it achieves an inference speed of 163.9 FPS. The experimental results show that RTS-Net can satisfy the accurate and efficient detection of ground objects by various different UAV platforms in different complex scenarios.

Light field salient object detection network based on feature enhancement and mutual attention

Light field salient object detection network based on feature enhancement and mutual attention

MFFNet: Multimodal feature fusion network for RGB-D transparent object detection

Transparent objects are ubiquitous in everyday life, but how to detect them is full of challenges. Transparent objects hardly reflect light, and they usually transmit the appearance of their surroundings, making it difficult to distinguish them from their surroundings. Existing methods usually use only RGB (Red Green Blue) images as input, ignoring the role of depth maps in transparent object detection. In this article, we try to improve the detection performance of transparent objects by fusing RGB and depth information. Specifically, we propose a multimodal fusion network that fuses RGB and depth modalities in a complementary way. Moreover, extensive experiments and ablation studies on the RGB-D (RGB-Depth) transparent object dataset demonstrate the excellent performance of our method.

S$^{2}$O-Det: A Semisupervised Oriented Object Detection Network for Remote Sensing Images

S$^{2}$O-Det: A Semisupervised Oriented Object Detection Network for Remote Sensing Images

A detection method for dead caged hens based on improved YOLOv7

In large-scale laying hen farms, daily inspection of dead hens is a relevant task to monitor the health of the flock and prevent disease spreading. The current manual inspection method used in caged hen farms is inefficient, costly, and particularly difficult for high-rise cages. To address this issue, a dead caged hen detection method based on improved You Only Look Once version 7 (YOLOv7) was proposed in this study, which was optimized to improve detection performance and speed in complex farming environments, such as cage wire mesh occlusion and crowded hen occlusion. First, the Convolutional Block Attention Module was used to enable the model to learn target features accurately. Second, the Distance Intersection over Union Non-maximum Suppression and repulsion loss were introduced to improve crowded hen occlusion and reduce missed detections. Additionally, to facilitate the deployment of the proposed method on mobile devices, the MobileNetv3 lightweight network was used to replace the backbone of YOLOv7. Furthermore, the lightweight model was trained using the knowledge distillation method to enhance its performance. Finally, a comparison experiment of different object detection networks and an ablation experiment were conducted to evaluate the proposed method. The experimental results reveal that the improved YOLOv7 model proposed in this study performs optimally. Its precision, recall, F1 score, and mAP@0.5 for the dead hens in the test set are 95.7 %, 86.8 %, 0.910, and 86.2 %, respectively. Compared with the original YOLOv7 model, precision, recall, and mAP@0.5 were increased by 6 %, 10 %, and 13.4 %, respectively. The model parameters and Giga Floating-point Operations were decreased by 31.95 % and 60.56 %, respectively, resulting in a detection speed increase of 43 Frames Per Second. Furthermore, with the assistance of an inspection robot, the proposed dead hen detection model was deployed in the actual farming environments. Compared with methods proposed by other researchers, the proposed model is more suitable for complex actual farming environments and achieves higher detection accuracy, which can offer a reference for automated caged hen detection.

Perceptual localization and focus refinement network for RGB-D salient object detection

RGB-D salient object detection task still encounters three challenges: (1) how to effectively integrate superior information from different modalities, (2) how to effectively mine common information of features at different levels, and (3) how to detect salient objects in complex scenes, such as complex backgrounds, low-quality depth maps, small targets, and high foreground-background similarity. To address the above challenges, we propose a novel Perceptual Localization and Focus Refinement Network, termed PLFRNet, based on the mechanism of human visual capture of salient objects in images. The network includes three key components: an encoder, a Perceptual Localization Module (PLM), and a Focus-Refinement Decoder (FRD). Specifically, we first adopt a two-stream asymmetric Pyramid Visual Transformer as the encoder to extract RGB and depth features. Then, we develop the PLM under the guidance of a Perceptual Localization Unit (PLU) delicately designed. This module can mine the common information of features at different levels and integrate the advantageous information from multiple modalities to localize salient objects. Finally, we propose the FRD focusing on detailed information guided by the attention mechanism. Furthermore, it further refines the located objects by gradually interacting with low-level features to achieve salient object detection. Extensive experimental results show that this method achieves state-of-the-art performance compared with 13 RGB-D models on 6 public datasets. The codes are released at https://github.com/hjy0518/PLFRNet/.

YOLO-SGF: Lightweight network for object detection in complex infrared images based on improved YOLOv8

The current mainstream object detection networks perform well in RGB visible images, but they require high computational resource and degrade in performance when applied to low-resolution infrared images. To address above issues, we propose a lightweight algorithm YOLO-SGF based on you-only-look-once version8 (YOLOv8). Firstly, the lightweight cross-scale feature map fusion network GCFVoV designed as neck to solve poor detection accuracy and maintain low complexity in lightweight networks. And a lightweight GCVF module in GCFVoV neck uses GSConv and Conv to process deep and shallow features respectively, which maximally preserves implicit connections between each channel and integrates multi-scale features. Secondly, we utilize ShuffleNetV2-block1 in combination with C2f for feature extraction, making the algorithm more lightweight and effectively. Finally, we propose the FIMPDIoU loss function, which focuses on overlooked objects in complex backgrounds and adjusts the prediction boxes using ratios specific to different sizes of objects. Compared with YOLOv8 in our infrared dataset, YOLO-SGF reduces the computational space complexity by 50 % and time complexity by 42 %, increases FPS32 by 36.3 % and improves mAP@0.5 ∼ 0.95 by 1.1 % in object detection. Our algorithm enhances the capability of object detection in infrared images especially in nighttime, low light, and occluded conditions. YOLO-SGF enables deployment on embedded edge devices with limited computing power, and provides a new idea for lightweight networks.

Lightweight Single-Stage Ship Object Detection Algorithm for Unmanned Surface Vessels Based on Improved YOLOv5.

Object detection is applied extensively in various domains, including industrial manufacturing, road traffic management, warehousing and logistics, and healthcare. In ship object detection tasks, detection networks are frequently deployed on devices with limited computational resources, e.g., unmanned surface vessels. This creates a need to balance accuracy with a low parameter count and low computational load. This paper proposes an improved object detection network based on YOLOv5. To reduce the model parameter count and computational load, we utilize an enhanced ShuffleNetV2 network as the backbone. In addition, a split-DLKA module is devised and implemented in the small object detection layer to improve detection accuracy. Finally, we introduce the WIOUv3 loss function to minimize the impact of low-quality samples on the model. Experiments conducted on the SeaShips dataset demonstrate that the proposed method reduces parameters by 71% and computational load by 58% compared to YOLOv5s. In addition, the proposed method increases the mAP@0.5 and mAP@0.5:0.95 values by 3.9% and 3.3%, respectively. Thus, the proposed method exhibits excellent performance in both real-time processing and accuracy.

Foreign object detection in urban rail transit based on deep differentiation segmentation neural network

With the increasing scale of urban rail transit, foreign object intrusion has become a significant operational safety hazard in urban rail transit. Although the laser-based automatic foreign object detection system has advantages such as long-distance detection and insensitivity to light changes, it has drawbacks such as large blind spots and low visualization. In response to the problems existing in laser detection systems, we proposed a novel video-based deep differentiation segmentation neural network for foreign object detection. Firstly, the foreign object detection is transformed into a binary classification problem, and the foreign object is determined as the image's foreground using image segmentation principles. Secondly, build a deep segmentation network based on deep convolution. Finally, perform morphological operations and threshold judgment on the foreground segmentation image to filter out the final detection results. To improve the detection effect, we reduced the impact of airflow disturbance by sampling and calculating the average background image. At the same time, the channel attention model and spatial attention model are added to the deep differentiation neural network. Collecting real data on subway platforms for experiments shows that the proposed method has a detection accuracy of 95.8 %, which is superior to traditional detection methods and recent image segmentation neural networks.

Cross-stage feature fusion and efficient self-attention for salient object detection

Salient Object Detection (SOD) approaches usually aggregate high-level semantics with object details layer by layer through a pyramid fusion structure. However, the progressive feature fusion mechanism may lead to gradually dilution of valuable semantics and prediction accuracy. In this work, we propose a Cross-stage Feature Fusion Network (CFFNet) for salient object detection. CFFNet consists of a Cross-stage Semantic Fusion Module (CSF), a Feature Filtering and Fusion Module (FFM), and a progressive decoder to tackle the above problems. Specifically, to alleviate the semantics dilution problem, CSF concatenates different stage backbone features and extracts multi-scale global semantics using transformer blocks. Global semantics are then distributed to corresponding backbone stages for cross-stage semantic fusion. The FFM module implements efficient self-attention-based feature fusion. Different from regular self-attention which has quadratic computational complexity. Finally, a progressive decoder is adopted to refine saliency maps. Experimental results demonstrate that CFFNet outperforms state-of-the-arts on six SOD datasets.

A Lightweight Crop Pest Detection Method Based on Improved RTMDet

To address the issues of low detection accuracy and large model parameters in crop pest detection in natural scenes, this study improves the deep learning object detection model and proposes a lightweight and accurate method RTMDet++ for crop pest detection. First, the real-time object detection network RTMDet is utilized to design the pest detection model. Then, the backbone and neck structures are pruned to reduce the number of parameters and computation. Subsequently, a shortcut connection module is added to the classification and regression branches, respectively, to enhance its feature learning capability, thereby improving its accuracy. Experimental results show that, compared to the original model RTMDet, the improved model RTMDet++ reduces the number of parameters by 15.5%, the computation by 25.0%, and improves the mean average precision by 0.3% on the crop pest dataset IP102. The improved model RTMDet++ achieves a mAP of 94.1%, a precision of 92.5%, and a recall of 92.7% with 4.117M parameters and 3.130G computations, outperforming other object detection methods. The proposed model RTMDet++ achieves higher performance with fewer parameters and computations, which can be applied to crop pest detection in practice and aids in pest control research.

Progressive cross-level fusion network for RGB-D salient object detection

Depth maps can provide supplementary information for salient object detection (SOD) and perform better in handling complex scenes. Most existing RGB-D methods only utilize deep cues at the same level, and few methods focus on the information linkage between cross-level features. In this study, we propose a Progressive Cross-level Fusion Network (PCF-Net). It ensures the cross-flow of cross-level features by gradually exploring deeper features, which promotes the interaction and fusion of information between different-level features. First, we designed a Cross-Level Guide Cross-Modal Fusion Module (CGCF) that utilizes the spatial information of upper-level features to suppress modal feature noise and to guide lower-level features for cross-modal feature fusion. Next, the proposed Semantic Enhancement Module (SEM) and Local Enhancement Module (LEM) are used to further introduce deeper features, enhance the high-level semantic information and low-level structural information of cross-modal features, and use self-modality attention refinement to improve the enhancement effect. Finally, the multi-scale aggregation decoder mines enhanced feature information in multi-scale spaces and effectively integrates cross-scale features. In this study, we conducted numerous experiments to demonstrate that the proposed PCF-Net outperforms 16 of the most advanced methods on six popular RGB-D SOD datasets.

Drone-DETR: Efficient Small Object Detection for Remote Sensing Image Using Enhanced RT-DETR Model.

Performing low-latency, high-precision object detection on unmanned aerial vehicles (UAVs) equipped with vision sensors holds significant importance. However, the current limitations of embedded UAV devices present challenges in balancing accuracy and speed, particularly in the analysis of high-precision remote sensing images. This challenge is particularly pronounced in scenarios involving numerous small objects, intricate backgrounds, and occluded overlaps. To address these issues, we introduce the Drone-DETR model, which is based on RT-DETR. To overcome the difficulties associated with detecting small objects and reducing redundant computations arising from complex backgrounds in ultra-wide-angle images, we propose the Effective Small Object Detection Network (ESDNet). This network preserves detailed information about small objects, reduces redundant computations, and adopts a lightweight architecture. Furthermore, we introduce the Enhanced Dual-Path Feature Fusion Attention Module (EDF-FAM) within the neck network. This module is specifically designed to enhance the network's ability to handle multi-scale objects. We employ a dynamic competitive learning strategy to enhance the model's capability to efficiently fuse multi-scale features. Additionally, we incorporate the P2 shallow feature layer from the ESDNet into the neck network to enhance the model's ability to fuse small-object features, thereby enhancing the accuracy of small object detection. Experimental results indicate that the Drone-DETR model achieves an mAP50 of 53.9% with only 28.7 million parameters on the VisDrone2019 dataset, representing an 8.1% enhancement over RT-DETR-R18.