Object Detection System Research Articles

AI-Based Object Detection, Distance Measurement and Speaking System for Blind Stick

The sense of sight is fundamental to how humans perceive and navigate their surroundings. For visually impaired individuals, this lack of vision creates significant challenges in detecting obstacles and measuring distances, making daily navigation difficult and potentially dangerous. In response to this problem, we propose an AI-based object detection, distance measurement, and speaking system integrated with a blind stick to enhance mobility and independence for the visually impaired. This system leverages cutting-edge artificial intelligence and image processing techniques to provide real-time object detection and distance measurement. By combining ultrasonic sensors with an AI-powered object recognition module, the system accurately identifies obstacles and calculates their distance, offering directional audio feedback to the user. Designed to be compact and user-friendly, the blind stick system provides seamless assistance in both indoor and outdoor environments. The object detection process utilizes advanced algorithms like YOLO (You Only Look Once) to identify obstacles. Distance measurement is performed using ultrasonic sensors to ensure accurate proximity readings. When an obstacle is detected, the system provides immediate auditory feedback, alerting users to the direction and distance of the object. The Google Maps API enhances usability by offering location awareness and route planning, empowering users to navigate with confidence. This AI-based blind stick system aims to improve the mobility and safety of visually impaired individuals, offering a more autonomous experience in their daily lives.

Application of Reinforcement Learning in Controlling Quadrotor UAV Flight Actions

Most literature has extensively discussed reinforcement learning (RL) for controlling rotorcraft drones during flight for traversal tasks. However, most studies lack adequate details regarding the design of reward and punishment mechanisms, and there is a limited exploration of the feasibility of applying reinforcement learning in actual flight control following simulation experiments. Consequently, this study focuses on the exploration of reward and punishment design and state input for RL. The simulation environment is constructed using AirSim and Unreal Engine, with onboard camera footage serving as the state input for reinforcement learning. The research investigates three RL algorithms suitable for discrete action training. The Deep Q Network (DQN), Advantage Actor–Critic (A2C), and Proximal Policy Optimization (PPO) were combined with three different reward and punishment design mechanisms for training and testing. The results indicate that employing the PPO algorithm along with a continuous return method as the reward mechanism allows for effective convergence during the training process, achieving a target traversal rate of 71% in the testing environment. Furthermore, this study proposes integrating the YOLOv7-tiny object detection (OD) system to assess the applicability of reinforcement learning in real-world settings. Unifying the state inputs of simulated and OD environments and replacing the original simulated image inputs with a maximum dual-target approach, the experimental simulation achieved a target traversal rate of 52% ultimately. In summary, this research formulates a set of logical frameworks for an RL reward and punishment design deployed with real-time Yolo’s OD implementation synergized as a useful aid for related RL studies.

Enhancing Surveillance Systems with Cost-Effective Ultrasonic Radar for Object Detection and Tracking

Object detection is critical in surveillance systems because it enables timely identification of unauthorized objects, which is essential for maintaining security and mitigating risks. Existing approaches to surveillance typically suffer from limited coverage, variable detection range, moderate accuracy, high cost, and complex setup requirements. This article presents an approach to object detection using microcontroller-based ultrasonic sensors. Our approach leverages four distinct processing phases to create a radar-like system capable of real-time detection and autonomous decision-making. In this study, we conducted experiments by integrating more than six peripheral modules with the Arduino platform. By evaluating 17 diverse test cases in various scenarios and environments, the approach demonstrated enhanced object tracking and robustness compared to existing methods. The system effectively detects objects within the targeted area, providing precise distance measurements (around 50 cm) and position information (0 to 360 degrees). Moreover, our approach enabled the identification of three risk zones—medium, high-risk, and mild danger—within the critical region, contributing to improved security and risk management. The results reveal the system’s effectiveness in real-time detection, accurate distance estimation, and comprehensive risk assessment. Overall, our study significantly contributes to the advancement of object detection technology by effectively addressing the limitations of existing methods and offering a cost-effective and practical solution. Our research has the potential to monitor objects in surveillance systems across various domains, with significant implications for enhancing security, risk mitigation, and monitoring in industries reliant on effective object detection systems. Jagannath University Journal of Science, Volume 11, Number 1, June 2024, pp. 157−171

A Computer Vision Model for Seaweed Foreign Object Detection Using Deep Learning

Seaweed foreign object detection has become crucial for food consumption and industrial use. This process not only can prevent potential health issues, but also maintain the overall marketability of seaweed production in the food industry. Traditional methods of inspecting seaweed foreign objects heavily rely on human judgment, which deals with large volumes with diverse impurities and can be inconsistent and inefficient. An automation system for real-time seaweed foreign object detection in the inspection process should be adopted. However, automated seaweed foreign object detection has several challenges due to its dependency on visual input inspection, such as an uneven surface and undistinguishable impurities. In fact, limited access to advanced technologies and high-cost equipment would also influence visual input acquisition, thereby hindering the advancement of seaweed foreign object detection in this field. Therefore, we introduce a computer vision model utilizing a deep learning-based algorithm to detect seaweed impurities and classify the samples into ‘clean’ and ‘unclean’ categories. In this study, we managed to identify six types of seaweed impurities including sand sticks, shells, discolored seaweed, grass, worm shells, and mixed impurities. We collected 1204 images and our model’s performance was thoroughly evaluated based on comparisons with three pre-trained models, i.e., Yolov8, ResNet, and MobileNet. Our experiment shows that Yolov8 outperforms the other two models with an accuracy of 98.86%. This study also included the development of an Android application to validate the deep learning engine to ensure its optimal performance. Based on our experiments, the mobile application managed to classify 50 pieces of seaweed samples within 0.2 s each, showcasing its potential use in large-scale production lines and factories. This research demonstrates the impact of Artificial Intelligence on food safety by offering a scalable and efficient solution that can be deployed in other food production processes facing similar challenges. Our approach paves the way for broader industry adoption and advancements in automated foreign object detection systems by optimizing detection accuracy and speed.

SSIM-Based Autoencoder Modeling to Defeat Adversarial Patch Attacks

Object detection systems are used in various fields such as autonomous vehicles and facial recognition. In particular, object detection using deep learning networks enables real-time processing in low-performance edge devices and can maintain high detection rates. However, edge devices that operate far from administrators are vulnerable to various physical attacks by malicious adversaries. In this paper, we implement a function for detecting traffic signs by using You Only Look Once (YOLO) as well as Faster-RCNN, which can be adopted by edge devices of autonomous vehicles. Then, assuming the role of a malicious attacker, we executed adversarial patch attacks with Adv-Patch and Dpatch. Trying to cause misdetection of traffic stop signs by using Adv-Patch and Dpatch, we confirmed the attacks can succeed with a high probability. To defeat these attacks, we propose an image reconstruction method using an autoencoder and the Structural Similarity Index Measure (SSIM). We confirm that the proposed method can sufficiently defend against an attack, attaining a mean Average Precision (mAP) of 91.46% even when two adversarial attacks are launched.

WGS-YOLO: A real-time object detector based on YOLO framework for autonomous driving

The safety and reliability of autonomous driving depends on the precision and efficiency of object detection systems. In this paper, a refined adaptation of the YOLO architecture (WGS-YOLO) is developed to improve the detection of pedestrians and vehicles. Specifically, its information fusion is enhanced by incorporating the Weighted Efficient Layer Aggregation Network (W-ELAN) module, an innovative dynamic weighted feature fusion module using channel shuffling. Meanwhile, the computational demands and parameters of the proposed WGS-YOLO are significantly reduced by employing the Space-to-Depth Convolution (SPD-Conv) and the Grouped Spatial Pyramid Pooling (GSPP) modules that have been strategically designed. The performance of our model is evaluated with the BDD100k and DAIR-V2X-V datasets. In terms of mean Average Precision (mAP0.5), the proposed model outperforms the baseline Yolov7 by 12%. Furthermore, extensive experiments are conducted to verify our analysis and the model’s robustness across diverse scenarios.

Browser-Based Object Detection System for Isolating Plastic Bottles using the COCO-SSD Model

Plastic waste, especially in urban environments and water bodies, poses a significant environmental threat. This paper presents a browser-based object detection system for identifying and isolating plastic bottles using state-of-the-art machine learning models. The system leverages TensorFlow.js, ML5.js, and P5.js libraries along with the COCO-SSD model to detect plastic bottles in real time using a mobile camera interface. By employing a browser-based architecture, the system offers cross-platform functionality, eliminating the need for server-based computations or specialised hardware. Experimental evaluation showed high detection accuracy across various environments, underscoring the potential for real-world applications in waste management and recycling efforts.

Lidar-Based Obstacle Detection and Path Prediction for Unmanned Surface Vehicles

Abstract Unmanned Surface Vehicles (USVs) represent an advanced technology with a wide range of applications in maritime contexts, and they offer a promising alternative to manned systems, particularly for the provision of services such as riverbed mapping. Furthermore, the use of USVs in combination with other robotic systems, such as flying drones, remotely operated underwater vehicles (ROVs) and autonomous underwater vehicles (AUVs), opens up innovative ways to inspect and monitor maritime infrastructure. However, the use of these systems in dynamic environments, such as ports or shipping lanes, requires a high degree of adaptability and precision in navigation to manoeuvre safely between other traffic participants and obstacles to prevent collisions. To address this problem, USVs require sophisticated perception systems for the precise detection and classification of nearby objects. Leveraging Light Detection and Ranging (LiDAR) technology, which is renowned for its efficacy in obstacle avoidance in robotics, this paper outlines the adaptation of a LiDAR sensor for USVs. The proposed pipeline enables the detection of other moving vessels and the accurate determination of their positions and trajectories up to 150 m from the USV. This paper introduces an obstacle detection system and validates it within a port setting. A methodical approach to LiDAR data preprocessing, clustering, and point cloud extraction will be outlined. Furthermore, it explores the application of an Unscented Kalman Filter (UKF) for the projection of motion paths of identified entities. This advanced form of data analysis provides the ability to predict the future position and path of potential obstacles, optimize decision-making for autonomous navigation algorithms and significantly improve the safety of USV operations. The pipeline is tested experimentally in sea trials by enacting five different scenarios that USV may encounter in a port setting. The results of the pipeline’s tracking capabilities are compared to the ground truth data collected by the tracked vessel and plotted to visualize the error.

A deep learning framework for automated anomaly detection and localization in fused filament fabrication

Under-extrusion poses a common challenge in filament 3D printing, necessitating precise adjustments to printing parameters for optimal resolution. Swift identification of this flaw is crucial for implementing timely corrective measures. In response, we present a novel machine learning approach designed to detect anomalies in 3D printing, specifically in the fused filament fabrication process (FFF). Our framework utilizes “You Only Look Once” (YOLO), a real-time object detection system, and “Visual Geometry Group-16” (VGG-16), a Convolutional Neural Network (CNN) model for image recognition, to accurately identify and localize under-extrusion events. Initially, models such as VGG-16, VGG-19, and ResNet-50 were trained without the inclusion of YOLO to establish baseline accuracies. Subsequently, an automated image pre-processing phase employs YOLO to discern the nozzle head, facilitating subsequent cropping around the region of interest for retraining the models. This inclusion of the nozzle head detection approach significantly improved the accuracy of all models, with the combined application of YOLO and VGG-16 demonstrating the most substantial enhancement, boosting detection accuracy to 97%. Moreover, Gradient-weighted Class Activation Mapping (Grad-CAM) technique, another CNN-based method, is employed to effectively highlight predicted areas in under-extrusion scenarios. While our method significantly advances the automatic detection of printing anomalies, it primarily serves as a diagnostic tool, signaling the need for intervention. Our rigorous testing on images of varying complexities confirms the model’s robustness, evaluated using comprehensive metrics such as precision, recall, and F1 score.

Lightweight Deep Learning Algorithm for Sorting Medical Waste Embedded System

The recent COVID-19 pandemic has led to an increased number of hospitalized patients, and in some countries, the overwhelming problem has even caused the collapse of the healthcare system. Thus, the associated waste generated and inadequate waste disposal may have deleterious impacts on public health and the surrounding environment. This paper aims to design and implement an automatic medical waste identification and sorting mechanism that can distinguish the items disposed of via a series of image processing procedures. Specifically, the system can classify, sort, and calculate the disposed items based on their visual appearance. To establish this, both software and hardware systems have been devised. Particularly, the vision system for object detection and classification is constructed based on several popular pre-trained convolutional neural networks. Upon verifying the type of waste (i.e., general infection, dangerous infection, and general garbage), an automatic sorting mechanism will be triggered to dispose of the object in the corresponding garbage bin. The experimentation has been validated on a total of self-collected 2025 images from 3 categories, and the best accuracy attained is 99.34% when adopting GoogLeNet as the backbone architecture.

YOLOv5_mamba: unmanned aerial vehicle object detection based on bidirectional dense feedback network and adaptive gate feature fusion

Addressing the problem that the object size in Unmanned Aerial Vehicles (UAVs) aerial images is too small and contains limited feature information, leading to existing detection algorithms having less than ideal performance in small object detection, we propose a UAV aerial object detection system named YOLv5_mamba based on bidirectional dense feedback network and adaptive gate feature fusion. This paper improves the You Only Look Once Version 5 (YOLOv5) algorithm by firstly introducing the Faster Implementation of CSP Bottleneck with 2 convolutions (C2f) module from YOLOv8 into the backbone network to enhance the feature extraction capability of the backbone network. Furthermore, the mamba module and C2f module are introduced to construct a bidirectional dense feedback network to enhance the transfer of contextual information in the neck part. Thirdly, an adaptive gate feature fusion network is proposed to improve the head part of YOLOv5 and enhance its final detection capability. Experimental results on the public UAV aerial dataset VisDrone2019 demonstrate that the proposed algorithm improves the detection accuracy by 9.3% compared to the original YOLOv5 baseline network, showing better detection performance for small objects. For the UCAS_AOD dataset, the proposed algorithm outperforms YOLOv5-s by 9%. In the case of the DIOR dataset, the proposed algorithm exceeds YOLOv5-s by 12%.

An in-vehicle real-time infrared object detection system based on deep learning with resource-constrained hardware

Advanced driver assistance systems primarily rely on visible images for information. However, in low-visibility weather conditions, such as heavy rain or fog, visible images struggle to capture road conditions accurately. In contrast, infrared (IR) images can overcome this limitation, providing reliable information regardless of external lighting. Addressing this problem, we propose an in-vehicle IR object detection system. We optimize the you only look once (YOLO) v4 object detection algorithm by replacing its original backbone with MobileNetV3, a lightweight feature extraction network, resulting in the MobileNetV3-YOLOv4 model. Furthermore, we replace traditional pre-processing methods with an Image Enhancement Conditional Generative Adversarial Network inversion algorithm to enhance the pre-processing of the input IR images. Finally, we deploy the model on the Jetson Nano, an edge device with constrained hardware resources. Our proposed method achieves an 82.7% mean Average Precision and a frame rate of 55.9 frames per second on the FLIR dataset, surpassing state-of-the-art methods. The experimental results confirm that our approach provides outstanding real-time detection performance while maintaining high precision.

Seeing the invisible: test prioritization for object detection system

Seeing the invisible: test prioritization for object detection system

An FPGA-Based YOLOv5 Accelerator for Real-Time Industrial Vision Applications.

The You Only Look Once (YOLO) object detection network has garnered widespread adoption in various industries, owing to its superior inference speed and robust detection capabilities. This model has proven invaluable in automating production processes such as material processing, machining, and quality inspection. However, as market competition intensifies, there is a constant demand for higher detection speed and accuracy. Current FPGA accelerators based on 8-bit quantization have struggled to meet these increasingly stringent performance requirements. In response, we present a novel 4-bit quantization-based neural network accelerator for the YOLOv5 model, designed to enhance real-time processing capabilities while maintaining high detection accuracy. To achieve effective model compression, we introduce an optimized quantization scheme that reduces the bit-width of the entire YOLO network-including the first layer-to 4 bits, with only a 1.5% degradation in mean Average Precision (mAP). For the hardware implementation, we propose a unified Digital Signal Processor (DSP) packing scheme, coupled with a novel parity adder tree architecture that accommodates the proposed quantization strategies. This approach efficiently reduces on-chip DSP utilization by 50%, offering a significant improvement in performance and resource efficiency. Experimental results show that the industrial object detection system based on the proposed FPGA accelerator achieves a throughput of 808.6 GOPS and an efficiency of 0.49 GOPS/DSP for YOLOv5s on the ZCU102 board, which is 29% higher than a commercial FPGA accelerator design (Xilinx's Vitis AI).

Weld seam object detection system based on the fusion of 2D images and 3D point clouds using interpretable neural networks

This study introduces a novel approach that addresses the limitations of existing methods by integrating 2D image processing with 3D point cloud analysis, enhanced by interpretable neural networks. Unlike traditional methods that rely on either 2D or 3D data alone, our approach leverages the complementary strengths of both data types to improve detection accuracy in environments adversely affected by welding spatter and smoke. Our system employs an improved Faster R-CNN model with a ResNet50 backbone for 2D image analysis, coupled with an innovative orthogonal plane intersection line extraction algorithm for 3D point cloud processing. By incorporating explainable components such as visualizable feature maps and a transparent region proposal network, we address the “black box” issue common in deep learning models.This architecture enables a more transparent decision-making process, providing technicians with necessary insights to understand and trust the system’s outputs. The Faster-RCNN structure is designed to break down the object detection process into distinct, understandable steps, from initial feature extraction to final bounding box refinement. This fusion of 2D-3D data analysis and interpretability not only improves detection performance but also sets a new standard for transparency and reliability in automated welding systems, facilitating wider adoption in industrial applications.

Active Security Surveillance and Object Detection

Active security surveillance and object detection have become crucial components in modern security systems, driven by advancements in computer vision and machine learning. These systems are designed to enhance real-time monitoring and threat detection across various environments, such as public spaces, transportation hubs, and critical infrastructure. Active security surveillance systems continuously scan and analyze video feeds or sensor data to identify potential security threats, unauthorized activities, or anomalies. They employ advanced algorithms for motion detection, facial recognition, and behavioural analysis, allowing for the rapid identification of suspicious activities or individuals. Object detection, a key element of these systems, involves the identification and classification of objects within a given scene. This is typically achieved through deep learning techniques, such as convolutional neural networks (CNNs), which can accurately detect and categorize objects like vehicles, weapons, or unattended bags. The integration of object detection with active surveillance enables more precise and automated responses to potential threats, reducing reliance on human operators and minimizing the risk of oversight. The adoption of active surveillance and object detection technologies has led to significant improvements in security management, offering enhanced situational awareness and faster response times. However, these technologies also raise concerns regarding privacy, data security, and ethical implications, particularly in their deployment in public and private spaces. Addressing these concerns requires careful consideration of data governance, transparency, and regulatory compliance to balance security needs with individual rights. As technology continues to evolve, ongoing research and development are essential to improving the accuracy, efficiency, and ethical deployment of active security surveillance and object detection systems.

Real-time object detection and augmentation

This study presents the development of a real-time object detection and augmentation system using TensorFlow and Unity it leverages TensorFlow model to identify and classify objects in real-time from a camera feed, and Unity to integrate and display corresponding 3D virtual objects in an augmented reality environment. By creating a mapping system to pair detected objects with their virtual counterparts, the system aims to enhance user interaction through dynamic and immersive AR experiences. The study addresses the challenges of real-time processing and seamless integration, demonstrating the potential of combining machine learning and augmented reality to enrich interactive applications across various domains.

Foreign object detection in urban rail transit based on deep differentiation segmentation neural network

With the increasing scale of urban rail transit, foreign object intrusion has become a significant operational safety hazard in urban rail transit. Although the laser-based automatic foreign object detection system has advantages such as long-distance detection and insensitivity to light changes, it has drawbacks such as large blind spots and low visualization. In response to the problems existing in laser detection systems, we proposed a novel video-based deep differentiation segmentation neural network for foreign object detection. Firstly, the foreign object detection is transformed into a binary classification problem, and the foreign object is determined as the image's foreground using image segmentation principles. Secondly, build a deep segmentation network based on deep convolution. Finally, perform morphological operations and threshold judgment on the foreground segmentation image to filter out the final detection results. To improve the detection effect, we reduced the impact of airflow disturbance by sampling and calculating the average background image. At the same time, the channel attention model and spatial attention model are added to the deep differentiation neural network. Collecting real data on subway platforms for experiments shows that the proposed method has a detection accuracy of 95.8 %, which is superior to traditional detection methods and recent image segmentation neural networks.

Model-agnostic explainable artificial intelligence for object detection in image data

In recent years, deep neural networks have been widely used for building high-performance Artificial Intelligence (AI) systems for computer vision applications. Object detection is a fundamental task in computer vision, which has been greatly progressed through developing large and intricate AI models. However, the lack of transparency is a big challenge that may not allow the widespread adoption of these models. Explainable artificial intelligence is a field of research where methods are developed to help users understand the behavior, decision logics, and vulnerabilities of AI systems. Previously, few explanation methods were developed for object detection based on random masking. However, random masks may raise some issues regarding the actual importance of pixels within an image. In this paper, we design and implement a black-box explanation method named Black-box Object Detection Explanation by Masking (BODEM) through adopting a hierarchical random masking approach for object detection systems. We propose a hierarchical random masking framework in which coarse-grained masks are used in lower levels to find salient regions within an image, and fine-grained mask are used to refine the salient regions in higher levels. Experimentations on various object detection datasets and models showed that BODEM can effectively explain the behavior of object detectors. Moreover, our method outperformed Detector Randomized Input Sampling for Explanation (D-RISE) and Local Interpretable Model-agnostic Explanations (LIME) with respect to different quantitative measures of explanation effectiveness. The experimental results demonstrate that BODEM can be an effective method for explaining and validating object detection systems in black-box testing scenarios.

ESFD-YOLOv8n: Early Smoke and Fire Detection Method Based on an Improved YOLOv8n Model

Ensuring fire safety is essential to protect life and property, but modern infrastructure and complex settings require advanced fire detection methods. Traditional object detection systems, often reliant on manual feature extraction, may fall short, and while deep learning approaches are powerful, they can be computationally intensive, especially for real-time applications. This paper proposes a novel smoke and fire detection method based on the YOLOv8n model with several key architectural modifications. The standard Complete-IoU (CIoU) box loss function is replaced with the more robust Wise-IoU version 3 (WIoUv3), enhancing predictions through its attention mechanism and dynamic focusing. The model is streamlined by replacing the C2f module with a residual block, enabling targeted feature extraction, accelerating training and inference, and reducing overfitting. Integrating generalized efficient layer aggregation network (GELAN) blocks with C2f modules in the neck of the YOLOv8n model further enhances smoke and fire detection, optimizing gradient paths for efficient learning and high performance. Transfer learning is also applied to enhance robustness. Experiments confirmed the excellent performance of ESFD-YOLOv8n, outperforming the original YOLOv8n by 2%, 2.3%, and 2.7%, with a mean average precision (mAP@0.5) of 79.4%, precision of 80.1%, and recall of 72.7%. Despite its increased complexity, the model outperforms several state-of-the-art algorithms and meets the requirements for real-time fire and smoke detection.