Object Detection Research Articles

Enhancing Surveillance Systems with Cost-Effective Ultrasonic Radar for Object Detection and Tracking

Object detection is critical in surveillance systems because it enables timely identification of unauthorized objects, which is essential for maintaining security and mitigating risks. Existing approaches to surveillance typically suffer from limited coverage, variable detection range, moderate accuracy, high cost, and complex setup requirements. This article presents an approach to object detection using microcontroller-based ultrasonic sensors. Our approach leverages four distinct processing phases to create a radar-like system capable of real-time detection and autonomous decision-making. In this study, we conducted experiments by integrating more than six peripheral modules with the Arduino platform. By evaluating 17 diverse test cases in various scenarios and environments, the approach demonstrated enhanced object tracking and robustness compared to existing methods. The system effectively detects objects within the targeted area, providing precise distance measurements (around 50 cm) and position information (0 to 360 degrees). Moreover, our approach enabled the identification of three risk zones—medium, high-risk, and mild danger—within the critical region, contributing to improved security and risk management. The results reveal the system’s effectiveness in real-time detection, accurate distance estimation, and comprehensive risk assessment. Overall, our study significantly contributes to the advancement of object detection technology by effectively addressing the limitations of existing methods and offering a cost-effective and practical solution. Our research has the potential to monitor objects in surveillance systems across various domains, with significant implications for enhancing security, risk mitigation, and monitoring in industries reliant on effective object detection systems. Jagannath University Journal of Science, Volume 11, Number 1, June 2024, pp. 157−171

Multi-Modal Object Detection Method Based on Dual-Branch Asymmetric Attention Backbone and Feature Fusion Pyramid Network

With the simultaneous acquisition of the infrared and optical remote sensing images of the same target becoming increasingly easy, using multi-modal data for high-performance object detection has become a research focus. In remote sensing multi-modal data, infrared images lack color information, it is hard to detect difficult targets with low contrast, and optical images are easily affected by illuminance. One of the most effective ways to solve this problem is to integrate multi-modal images for high-performance object detection. The challenge of fusion object detection lies in how to fully integrate multi-modal image features with significant modal differences and avoid introducing interference information while taking advantage of complementary advantages. To solve these problems, a new multi-modal fusion object detection method is proposed. In this paper, the method is improved in terms of two aspects: firstly, a new dual-branch asymmetric attention backbone network (DAAB) is designed, which uses a semantic information supplement module (SISM) and a detail information supplement module (DISM) to supplement and enhance infrared and RGB image information, respectively. Secondly, we propose a feature fusion pyramid network (FFPN), which uses a Transformer-like strategy to carry out multi-modal feature fusion and suppress features that are not conducive to fusion during the fusion process. This method is a state-of-the-art process for both FLIR-aligned and DroneVehicle datasets. Experiments show that this method has strong competitiveness and generalization performance.

Spiking Neural Networks for Object Detection Based on Integrating Neuronal Variants and Self-Attention Mechanisms

Thanks to their event-driven asynchronous computing capabilities and low power consumption advantages, spiking neural networks (SNNs) show significant potential for computer vision tasks, especially in object detection. However, effective training methods and optimization mechanisms for SNNs remain underexplored. This study proposes two high accuracy SNNs for object detection, AMS_YOLO and AMSpiking_VGG, integrating neuronal variants and attention mechanisms. To enhance these proposed networks, we explore the impact of incorporating different neuronal variants.The results show that the optimization in the SNN’s structure with neuronal variants outperforms that in the attention mechanism for object detection. Compared to the state-of-the-art in the current SNNs, AMS_YOLO improved by 6.7% in accuracy on the static dataset COCO2017, and AMS_Spiking has improved by 11.4% on the dynamic dataset GEN1.

Research on Target Image Classification in Low-Light Night Vision.

In extremely dark conditions, low-light imaging may offer spectators a rich visual experience, which is important for both military and civic applications. However, the images taken in ultra-micro light environments usually have inherent defects such as extremely low brightness and contrast, a high noise level, and serious loss of scene details and colors, which leads to great challenges in the research of low-light image and object detection and classification. The low-light night vision image used as the study object in this work has an excessively dim overall picture and very little information about the screen's features. Three algorithms, HE, AHE, and CLAHE, were used to enhance and highlight the image. The effectiveness of these image enhancement methods is evaluated using metrics such as the peak signal-to-noise ratio and mean square error, and CLAHE was selected after comparison. The target image includes vehicles, people, license plates, and objects. The gray-level co-occurrence matrix (GLCM) was used to extract the texture features of the enhanced images, and the extracted image texture features were used as input to construct a backpropagation (BP) neural network classification model. Then, low-light image classification models were developed based on VGG16 and ResNet50 convolutional neural networks combined with low-light image enhancement algorithms. The experimental results show that the overall classification accuracy of the VGG16 convolutional neural network model is 92.1%. Compared with the BP and ResNet50 neural network models, the classification accuracy was increased by 4.5% and 2.3%, respectively, demonstrating its effectiveness in classifying low-light night vision targets.

Synthetic meets authentic: Leveraging LLM generated datasets for YOLO11 and YOLOv10-based apple detection through machine vision sensors

Training machine learning (ML) models for artificial intelligence (AI) and computer vision-based object detection process typically requires large, labeled datasets, a process often burdened by significant human effort and high costs associated with imaging systems and image acquisition. This research aimed to simplify image data collection for object detection in orchards by avoiding traditional fieldwork with different imaging sensors. Utilizing OpenAI's DALLE, a large language model (LLM) for realistic image generation, we generated and annotated a cost-effective dataset. This dataset, exclusively generated by LLM, was then utilized to train two state-of-the-art deep learning models: YOLOV10 and YOLO11. The YOLO11 model for apple detection was trained with its five configurations (YOLO11n, YOLO11 s, YOLO11 m, YOLO11l and YOLO11x), and YOLOv10 model with its six configurations (YOLOv10n, YOLOv10 s, YOLOv10 m, YOLOv10b, YOLOv10l and YOLOv10x), which was then tested with real-world (outdoor orchard) images captured by a digital (Nikon D5100) camera and a consumer RGB-D camera (Microsoft Azure Kinect). YOLO11 outperformed YOLOv10 as YOLO11x and YOLO11n exhibited superior precision of 0.917 and 0.916, respectively. Furthermore, YOLO11l demonstrated the highest recall among its counterparts, achieving a recall of 0.889. Likewise, the YOLO11n variant excelled in terms of mean average precision (mAP@50), achieving the highest value of 0.958. Validation tests against actual images collected through a digital camera (Nikon D5100) over Scilate apple variety in a commercial orchard environment showed a highest precision of 0.874 for YOLO11 s, recall of 0.877 for YOLO11l and mAP@50 of 0.91 for YOLO11x. Additionally, validation test against actual images collected through a Microsoft Azure camera over the same orchard showed a highest precision, recall and mAP@50 respectively of 0.924, 0.781 and 0.855 with YOLO11x. All variants of YOLO11 surprisingly demonstrated a pre-processing time of just 0.2 milliseconds (ms), which was faster than any variant of YOLOv10. The fastest inference time for the YOLO11n model using the training dataset generated by the language model was 3.2 ms, while YOLOv10n, fastest among YOLOv10 variants, had a longer inference time of 5.5 ms. Likewise, the fastest inference time for the sensor-based images was 7.1 ms (for Nikon D5100 camera images) and 4.7 ms (for Azure images) with YOLO11n. This study presents a pathway for generating large image datasets using LLM in challenging agricultural fields with minimal or no labor-intensive efforts in field data-collection, which could accelerate the development and deployment of computer vision and robotic technologies in orchard environments.

Low power RTL implementation of Tiny-YOLO-v2 DCNN hardware accelerator on Virtex-7 FPGA

Deep Convolution Neural Networks (DCNNs) are widely used in real-time applications, including image classification, speech recognition, and object detection. However, there are challenges for real-time applications on portable devices like mobile phones or embedded systems. Most object detection models are optimized for desktop configurations, requiring fast GPUs. Shallower networks with fewer computational complexities have been proposed for real-time detection, but ultimately compromise detection accuracy. Performance and complexity trade-offs are significant for computationally complex deep networks. This work aims to implement the CNN-based object detection model Tiny-Yolo-v2 on a Field Programmable Gate Array (FPGA) using Register Transfer Logic (RTL) as a native language. Hardware implementation is synthesized on The AMD Virtex 7 FPGA VC709 Connectivity Kit using VHDL code on Vivado 2020.1. This is the first, up to the authors' knowledge, RTL implementation of the Tiny-Yolo-v2 object identification algorithm on FPGA. The power consumed by the CNN layers equals 7.09W at a frequency of 100MHz.

Unsupervised Clustering-based 3D Static Scene Construction Using LiDAR Channel and Azimuth Angle

Abstract. Cameras are typically used at road intersections to collect data to perform object detection but struggle in low-light and harsh weather. On the other hand, Light Detection and Ranging (LiDAR) is used as a key technology in 3D vision systems. It gives the 3D point cloud, which includes accurate depth information, but its resolution is cost-dependent, with higher resolutions being more expensive. Deep learning-based method requires large, labelled dataset which increases the cost, time and accuracy depending on the model trained on labelled dataset. To perform object detection in point cloud data is a challenging task due to the incomplete representations, data sparsity and unavailability of training data. To overcome this by using an unsupervised approach, it is important to identify the static scene and then detect the moving object. This work incorporated a novel approach to construct static scene using azimuth angle and laser channel information using an unsupervised clustering approach. This work incorporates two modules I) data collection using VLP-16 LiDAR, and II) static scene construction using DBSCAN (density-based spatial clustering and noise) clustering-based approach. Data is collected at a 4-legged intersection and pre-processed to extract aggregated distances corresponding to the unique pair of azimuth angle and laser channel. DBSCAN is used to perform clustering on the aggregated distances, based on the highest silhouette score and lowest intra distance between points in cluster, static points are identified, and static scene constructed. The qualitative evaluation of method demonstrates that the algorithm effectively and accurately filters out background points.

YOLO-MMS for aerial object detection model based on hybrid feature extractor and improved multi-scale prediction

YOLO-MMS for aerial object detection model based on hybrid feature extractor and improved multi-scale prediction

YOLO-GE: An Attention Fusion Enhanced Underwater Object Detection Algorithm

Underwater object detection is a challenging task with profound implications for fields such as aquaculture, marine ecological protection, and maritime rescue operations. The presence of numerous small aquatic organisms in the underwater environment often leads to issues of missed detections and false positives. Additionally, factors such as the water quality result in weak target features, which adversely affect the extraction of target feature information. Furthermore, the lack of illumination underwater causes image blur and low contrast, thereby increasing the difficulty of the detection task. To address these issues, we propose a novel underwater object detection algorithm called YOLO-GE (GCNet-EMA). First, we introduce an image enhancement module to mitigate the impact of underwater image quality issues on the detection task. Second, a high-resolution feature layer is added into the network to improve the problems of missed detections and false positives for small targets. Third, we propose GEBlock, an attention-based fusion module that captures long-range contextual information and suppresses noise from lower-level feature layers. Finally, we combine an adaptive spatial fusion module with the detection head to filter out conflicting feature information from different feature layers. Experiments on the UTDAC2020, DUO and RUOD datasets show that the proposed method achieves an optimal detection accuracy.

Multi-feature enhancement based on sparse networks for single-stage 3D object detection

In the field of autonomous driving, the accuracy and real-time requirements for 3D object detection technology continue to improve, which is directly related to the commercialization process and market popularity of autonomous vehicles. Despite the efficiency of pillar-based coding for onboard systems, it falls short in terms of accuracy and the reduction of incorrect positives. In this paper, we will examine how to solve the problem of high incorrect rate and low accuracy of existing methods. Firstly, a MAP coding module is introduced to optimize previous point cloud feature coding modules, allowing for the efficient extraction of fine-grained features from point cloud data. Then, we introduce an innovative sparse dual attention (SDA) to efficiently filter out irrelevant details in feature extraction, thereby improving the pertinence and efficiency of information extraction. Finally, to address the potential loss of information from single local feature extraction, a local and global fusion module (CTGC) is introduced. Our method has proactively demonstrated its efficiency and accuracy through rigorous experimentation across diverse datasets. Analysis of the results leads to the conclusion that our solutions provide accurate and robust detection results. Code will be available at https://github.com/lcy199905/MyOpenPCDet.git.

Real-time detection of coke particles in blast furnace operations using machine learning: Case of a steel plant in India

The efficiency of blast furnace operations in a steel plant relies on the quality of raw materials, including coke. The particle size distribution (PSD) of coke plays a vital role in ensuring the efficiency of the blast furnace. Conventional methods of measuring average particle size are time-consuming and labour-intensive leading to delayed operations. To address the issue, various studies have used different techniques for real-time measurement of coke particle size, including laser diffraction, acoustic sounding, X-ray computed tomography, and computer vision. While these methods have shown successful applications, there have been limitations in their lower prediction accuracy and detection speed. Therefore, our study proposes a practical approach that leverages the integration of computer vision algorithms to enhance accuracy and detection speed. The proposed methodology is compared with the conventional method of measuring particle size in the lab at a blast furnace. Our results reveal that the YOLOv8 algorithm outperforms the conventional method, providing efficient detection of 5.419 frames per second and with a mean absolute error of [Formula: see text]1.77 mm within the lab results. YOLOv8 can perform object segmentation providing much more precise polygon masks of the coke particles instead of simply providing rectangular dimensions using traditional object detection, as explored in previous studies. This approach enables real-time monitoring of PSD and timely alerts to the onsite team, significantly improving the efficiency of blast furnace operations.

YOLO-ESL: An Enhanced Pedestrian Recognition Network Based on YOLO

Pedestrian detection is a critical task in computer vision; however, mainstream algorithms often struggle to achieve high detection accuracy in complex scenarios, particularly due to target occlusion and the presence of small objects. This paper introduces a novel pedestrian detection algorithm, YOLO-ESL, based on the YOLOv7 framework. YOLO-ESL integrates the ELAN-SA module, designed to enhance feature extraction, with the LGA module, which improves feature fusion. The ELAN-SA module optimizes the flexibility and efficiency of small object feature extraction, while the LGA module effectively integrates multi-scale features through local and global attention mechanisms. Additionally, the CIOUNMS algorithm addresses the issue of target loss in cases of high overlap, improving boundary box filtering. Evaluated on the VOC2012 pedestrian dataset, YOLO-ESL achieved an accuracy of 93.7%, surpassing the baseline model by 3.0%. Compared to existing methods, this model not only demonstrates strong performance in handling occluded and small object detection but also remarkable robustness and efficiency.

A deep learning based assisted analysis approach for Sjogren's syndrome pathology images.

Diagnosing Sjogren's syndrome requires considerable time and effort from physicians, primarily because it necessitates rigorously establishing the presence lymphatic infiltration in the pathological tissue of the labial gland. The aim of this study is to use deep learning techniques to overcome these limitations and improve diagnostic accuracy and efficiency in pathology. We develop an auxiliary diagnostic system for Sjogren's syndrome. The system incorporates the state-of-the-art object detection neural network, YOLOv8, and enables the precise identification and flagging of suspicious lesions. We design the multi-dimensional attention module and S-MPDIoU loss function to improve the detection performance of YOLOv8. By extracting features from multiple dimensions of the feature map, the utilization of the multi-dimensional attention mechanism enhances the feature interaction across disparate positions, enabling the network to proficiently learn and retain salient cell features. S-MPDIoU introduces an angle penalty term that efficiently minimizes the diagonal distance between predicted and ground truth boxes. Additionally, it incorporates a flexible scale factor tailored to different size feature maps, which balances the issue of sudden gradient decrease during high overlap, thereby accelerating the overall convergence rate. To verify the effectiveness of our methods, we create a dataset of lymphocytes using labial gland biopsy pathology images collected from YanTaiShan hospital and trained the model with this dataset. The proposed model is assessed using standard metrics like precision, recall, mAP. The improved model achieves an increase in recall by 9.1%, mAP.5 by 3.2%, and mAP.95 by 2%. The study demonstrated deep learning's potential to analysis pathology images, offering a reference framework for the application of deep learning technology in the medical domain.

KOALA: A Modular Dual-Arm Robot for Automated Precision Pruning Equipped with Cross-Functionality Sensor Fusion

Landscape maintenance is essential for ensuring agricultural productivity, promoting sustainable land use, and preserving soil and ecosystem health. Pruning is a labor-intensive task among landscaping applications that often involves repetitive pruning operations. To address these limitations, this paper presents the development of a dual-arm holonomic robot (called the KOALA robot) for precision plant pruning. The robot utilizes a cross-functionality sensor fusion approach, combining light detection and ranging (LiDAR) sensor and depth camera data for plant recognition and isolating the data points that require pruning. The You Only Look Once v8 (YOLOv8) object detection model powers the plant detection algorithm, achieving a 98.5% pruning plant detection rate and a 95% pruning accuracy using camera, depth sensor, and LiDAR data. The fused data allows the robot to identify the target boxwood plants, assess the density of the pruning area, and optimize the pruning path. The robot operates at a pruning speed of 10–50 cm/s and has a maximum robot travel speed of 0.5 m/s, with the ability to perform up to 4 h of pruning. The robot’s base can lift 400 kg, ensuring stability and versatility for multiple applications. The findings demonstrate the robot’s potential to significantly enhance efficiency, reduce labor requirements, and improve landscape maintenance precision compared to those of traditional manual methods. This paves the way for further advancements in automating repetitive tasks within landscaping applications.

Van-DETR: enhanced real-time object detection with vanillanet and advanced feature fusion

Van-DETR: enhanced real-time object detection with vanillanet and advanced feature fusion

AMW-YOLOv8n: Road Scene Object Detection Based on an Improved YOLOv8

This study introduces an improved YOLOv8 model tailored for detecting objects in road scenes. To overcome the limitations of standard convolution operations in adapting to varying targets, we introduce Adaptive Kernel Convolution (AKconv). AKconv dynamically adjusts the convolution kernel’s shape and size, enhancing the backbone network’s feature extraction capabilities and improving feature representation across different scales. Additionally, we employ a Multi-Scale Dilated Attention (MSDA) mechanism to focus on key target features, further enhancing feature representation. To address the challenge posed by YOLOv8’s large down sampling factor, which limits the learning of small target features in deeper feature maps, we add a small target detection layer. Finally, to improve model training efficiency, we introduce a regression loss function with a Wise-IoU dynamic non-monotonic focusing mechanism. With these enhancements, our improved YOLOv8 model excels in road scene object detection tasks, achieving a 5.6 percentage point improvement in average precision over the original YOLOv8n on real road datasets.

Fruit Distribution Density Estimation in YOLO-Detected Strawberry Images: A Kernel Density and Nearest Neighbor Analysis Approach

Precise information on strawberry fruit distribution is of significant importance for optimizing planting density and formulating harvesting strategies. This study applied a combined analysis of kernel density estimation and nearest neighbor techniques to estimate fruit distribution density from YOLOdetected strawberry images. Initially, an improved yolov8n strawberry object detection model was employed to obtain the coordinates of the fruit centers in the images. The results indicated that the improved model achieved an accuracy of 94.7% with an mAP@0.5~0.95 of 87.3%. The relative error between the predicted and annotated coordinates ranged from 0.002 to 0.02, demonstrating high consistency between the model predictions and the annotated results. Subsequently, based on the strawberry center coordinates, the kernel density estimation algorithm was used to estimate the distribution density in the strawberry images. The results showed that with a bandwidth of 200, the kernel density estimation accurately reflected the actual strawberry density distribution, ensuring that all center points in high-density regions were consistently identified and delineated. Finally, to refine the strawberry distribution information, a comprehensive method based on nearest neighbor analysis was adopted, achieving target area segmentation and regional density estimation in the strawberry images. Experimental results demonstrated that when the distance threshold ϵ was set to 600 pixels, the correct grouping rate exceeded 94%, and the regional density estimation results indicated a significant positive correlation between the number of fruits and regional density. This study provides scientific evidence for optimizing strawberry planting density and formulating harvesting sequences, contributing to improved yield, harvesting efficiency, and reduced fruit damage. In future research, this study will further explore dynamic models that link fruit distribution density, planting density, and fruit growth status.

A Computer Vision Model for Seaweed Foreign Object Detection Using Deep Learning

Seaweed foreign object detection has become crucial for food consumption and industrial use. This process not only can prevent potential health issues, but also maintain the overall marketability of seaweed production in the food industry. Traditional methods of inspecting seaweed foreign objects heavily rely on human judgment, which deals with large volumes with diverse impurities and can be inconsistent and inefficient. An automation system for real-time seaweed foreign object detection in the inspection process should be adopted. However, automated seaweed foreign object detection has several challenges due to its dependency on visual input inspection, such as an uneven surface and undistinguishable impurities. In fact, limited access to advanced technologies and high-cost equipment would also influence visual input acquisition, thereby hindering the advancement of seaweed foreign object detection in this field. Therefore, we introduce a computer vision model utilizing a deep learning-based algorithm to detect seaweed impurities and classify the samples into ‘clean’ and ‘unclean’ categories. In this study, we managed to identify six types of seaweed impurities including sand sticks, shells, discolored seaweed, grass, worm shells, and mixed impurities. We collected 1204 images and our model’s performance was thoroughly evaluated based on comparisons with three pre-trained models, i.e., Yolov8, ResNet, and MobileNet. Our experiment shows that Yolov8 outperforms the other two models with an accuracy of 98.86%. This study also included the development of an Android application to validate the deep learning engine to ensure its optimal performance. Based on our experiments, the mobile application managed to classify 50 pieces of seaweed samples within 0.2 s each, showcasing its potential use in large-scale production lines and factories. This research demonstrates the impact of Artificial Intelligence on food safety by offering a scalable and efficient solution that can be deployed in other food production processes facing similar challenges. Our approach paves the way for broader industry adoption and advancements in automated foreign object detection systems by optimizing detection accuracy and speed.

Comparison of Immersion and Portable Ultrasonic Housing to Quantify the Adhesive Bond Thickness and Sizing of Foreign Objects.

High-performance materials, such as carbon fiber laminates, are costly to manufacture and are often used in demanding environments requiring the use of high-resolution non-destructive testing (NDT) methods to confirm the integrity of the parts. One NDT method that has shown promise for qualifying carbon fiber laminates is the use of immersion ultrasound with spherically focused probes. However, many parts may not be submersible in an immersion tank due to size or material constraints. These parts must be scanned with contact transducers with inferior resolutions or with expensive and messy systems such as bubblers. This research presents the use of a novel housing system that allows for the use of focused immersion transducers in an out-of-tank portable ultrasonic scanning application. This work presents a comparison between scans taken using a custom high-resolution immersion system and scans taken using the presented housing. There are a wide variety of potential inspection applications for this novel system, and the present work focused on two specific applications: the quantification of the spatially varying adhesive thickness in bonded carbon fiber laminates and the quantification of foreign object inclusions in carbon fiber laminates. The results presented show that scans using the portable housing are comparable in quality to scans performed using an immersion system. Specifically, both inspection approaches had an average error of 0.04 mm when quantifying the adhesive thickness of a bonded composite, and for the foreign object detection, the error in quantifying the dimensions of the embedded foreign object was 0.1 mm and 0.2 mm for the immersion system and the portable inspection system, respectively. The demonstration was performed in a laboratory setting, but a discussion is provided for the necessary improvements needed to extend the system for use in field applications.

Railway foreign object tracking and detection with spatial positioning and feature generalization enhancement

The existing deep learning foreign object tracking and detection algorithm is easily affected by complex environments and target occlusion, resulting in problems such as missed detection and low detection accuracy. A railway foreign object tracking and detection algorithm with spatial positioning and feature generalization enhancement is proposed. First, a multi-scale cascade improved GhostNet feature network is proposed to improve the feature extraction capability of infrared targets. Secondly, the spatial positioning and feature generalization enhancement module is designed by using the spatial position positioning and generalized morphological information of foreign objects to enhance the detection accuracy of targets with position movement and tracking trajectory changesin complex scenes. Then, a pyramid prediction network is constructed to obtain the detection anchor frame, type and confidence information of infrared railway foreign objects. Finally, by improving the DeepSORT tracking algorithm with improved category and confidence display, combined with Kalman filtering and Hungarian algorithm, railway foreign object tracking and detection in infrared weak light environment is realized. Experimental results show that the proposed method has a tracking and detection accuracy of railway foreign objects 83.3%, with an average detection rate of 11.3 frames per second. Compared with the comparison method, the proposed method has higher detection accuracy and has better performance in railway foreign object tracking and detection in infrared weak light scenes.