Object Detection System Research Articles

Language guided 3D object detection in point clouds for MEP scenes

AbstractIn recent years, contrastive language‐image pre‐training (CLIP) has gained popularity for processing 2D data. However, the application of cross‐modal transferable learning to 3D data remains a relatively unexplored area. In addition, high‐quality, labelled point cloud data for Mechanical, Electrical, and Plumbing (MEP) scenarios are in short supply. To address this issue, the authors introduce a novel object detection system that employs 3D point clouds and 2D camera images, as well as text descriptions as input, using image‐text matching knowledge to guide dense detection models for 3D point clouds in MEP environments. Specifically, the authors put forth the proposition of a language‐guided point cloud modelling (PCM) module, which leverages the shared image weights inherent in the CLIP backbone. This is done with the aim of generating pertinent category information for the target, thereby augmenting the efficacy of 3D point cloud target detection. After sufficient experiments, the proposed point cloud detection system with the PCM module is proven to have a comparable performance with current state‐of‐the‐art networks. The approach has 5.64% and 2.9% improvement in KITTI and SUN‐RGBD, respectively. In addition, the same good detection results are obtained in their proposed MEP scene dataset.

Deep CNN Framework for Object Detection and Classification System from Real Time Videos

Deep CNN Framework for Object Detection and Classification System from Real Time Videos

AI-Powered Object Detection to The Seamless Integration of Renewable Energy Into Electric Vehicles

This article explores the symbiotic relationship between cutting-edge technologies, focusing on the evolution from AI-driven object detection systems to the seamless incorporation of renewable energy sources into electric vehicles (EVs). Initially, advancements in artificial intelligence, particularly in the realm of object detection, have revolutionized real-time identification processes. The integration of TensorFlow models within edge computing architectures has significantly enhanced accuracy and efficiency, serving as a cornerstone across various industries. Concurrently, research efforts have been directed towards the integration of renewable energy sources into EV systems. This multifaceted approach aims to minimize carbon footprints and augment the sustainability quotient of transportation. Understanding the pivotal role of meticulous electrical design, harnessing mechanisms, and structural optimizations in EVs, this article emphasizes their interconnectedness with the broader scope of renewable energy integration. Through the amalgamation of AI-powered object detection systems and renewable energy synergies within electric vehicles, this article encapsulates the technological trajectory towards a more efficient, sustainable, and interconnected future in transportation.

Visual object detection using audio data

Nowadays Internet of Things (IoT) and Machine Learning (ML) are growing fields. One application of these two fields is object detection, which detects semantic objects using digital images and videos of classes like humans, vehicles, buildings, etc. Visual object detection systems are very effective and accurate due to the appearance information obtained from the cameras. But they face the problem of a limited Field of View. This paper aims to tackle this issue by using audio data to localize the object. A microphone is used to estimate the angular position of the object emitting the sound. Objects currently in the Field of View of a camera are detected and tracked using optical flow, but when they go out of the Field of View, the sound emitted by the object is used by the microphone to calculate the object‘s angular position. Once the angle is calculated, the camera is rotated in that direction. This thus ensures that the object can be located even if it goes out of frame. Once the object is located through IoT devices, we use ML to identify the person‘s face.

Impact of ISP Tuning on Object Detection.

In advanced driver assistance systems (ADAS) or autonomous vehicle research, acquiring semantic information about the surrounding environment generally relies heavily on camera-based object detection. Image signal processors (ISPs) in cameras are generally tuned for human perception. In most cases, ISP parameters are selected subjectively and the resulting image differs depending on the individual who tuned it. While the installation of cameras on cars started as a means of providing a view of the vehicle's environment to the driver, cameras are increasingly becoming part of safety-critical object detection systems for ADAS. Deep learning-based object detection has become prominent, but the effect of varying the ISP parameters has an unknown performance impact. In this study, we analyze the performance of 14 popular object detection models in the context of changes in the ISP parameters. We consider eight ISP blocks: demosaicing, gamma, denoising, edge enhancement, local tone mapping, saturation, contrast, and hue angle. We investigate two raw datasets, PASCALRAW and a custom raw dataset collected from an advanced driver assistance system (ADAS) perspective. We found that varying from a default ISP degrades the object detection performance and that the models differ in sensitivity to varying ISP parameters. Finally, we propose a novel methodology that increases object detection model robustness via ISP variation data augmentation.

IoT-Based Object-Detection System to Safeguard Endangered Animals and Bolster Agricultural Farm Security

Significant threats to ecological equilibrium and sustainable agriculture are posed by the extinction of animal species and the subsequent effects on farms. Farmers face difficult decisions, such as installing electric fences to protect their farms, although these measures can harm animals essential for maintaining ecological equilibrium. To tackle these essential issues, our research introduces an innovative solution in the form of an object-detection system. In this research, we designed and implemented a system that leverages the ESP32-CAM platform in conjunction with the YOLOv8 object-detection model. Our proposed system aims to identify endangered species and harmful animals within farming environments, providing real-time alerts to farmers and endangered wildlife by integrating a cloud-based alert system. To train the YOLOv8 model effectively, we meticulously compiled diverse image datasets featuring these animals in agricultural settings, subsequently annotating them. After that, we tuned the hyperparameter of the YOLOv8 model to enhance the performance of the model. The results from our optimized YOLOv8 model are auspicious. It achieves a remarkable mean average precision (mAP) of 92.44% and an impressive sensitivity rate of 96.65% on an unseen test dataset, firmly establishing its efficacy. After achieving an optimal result, we employed the model in our IoT system and when the system detects the presence of these animals, it immediately activates an audible buzzer. Additionally, a cloud-based system was utilized to notify neighboring farmers effectively and alert animals to potential danger. This research’s significance lies in its potential to drive the conservation of endangered species while simultaneously mitigating the agricultural damage inflicted by these animals.

A Comprehensive Review of YOLO Architectures in Computer Vision: From YOLOv1 to YOLOv8 and YOLO-NAS

YOLO has become a central real-time object detection system for robotics, driverless cars, and video monitoring applications. We present a comprehensive analysis of YOLO’s evolution, examining the innovations and contributions in each iteration from the original YOLO up to YOLOv8, YOLO-NAS, and YOLO with transformers. We start by describing the standard metrics and postprocessing; then, we discuss the major changes in network architecture and training tricks for each model. Finally, we summarize the essential lessons from YOLO’s development and provide a perspective on its future, highlighting potential research directions to enhance real-time object detection systems.

High-performance Reconfigurable DNN Accelerator on a Bandwidth-limited Embedded System

Deep convolutional neural networks (DNNs) have been widely used in many applications, particularly in machine vision. It is challenging to accelerate DNNs on embedded systems because real-world machine vision applications should reserve a lot of external memory bandwidth for other tasks, such as video capture and display, while leaving little bandwidth for accelerating DNNs. In order to solve this issue, in this study, we propose a high-throughput accelerator, called reconfigurable tiny neural network accelerator (ReTiNNA), for the bandwidth-limited system and present a real-time object detection system for the high-resolution video image. We first present a dedicated computation engine that takes different data mapping methods for various filter types to improve data reuse and reduce hardware resources. We then propose an adaptive layer-wise tiling strategy that tiles the feature maps into strips to reduce the control complexity of data transmission dramatically and to improve the efficiency of data transmission. Finally, a design space exploration (DSE) approach is presented to explore design space more accurately in the case of insufficient bandwidth to improve the performance of the low-bandwidth accelerator. With a low bandwidth of 2.23 GB/s and a low hardware consumption of 90.261K LUTs and 448 DSPs, ReTiNNA can still achieve a high performance of 155.86 GOPS on VGG16 and 68.20 GOPS on ResNet50, which is better than other state-of-the-art designs implemented on FPGA devices. Furthermore, the real-time object detection system can achieve a high object detection speed of 19 fps for high-resolution video.

A Motion Deblurring Network for Enhancing UAV Image Quality in Bridge Inspection

Unmanned aerial vehicles (UAVs) have been increasingly utilized for facility safety inspections due to their superior safety, cost effectiveness, and inspection accuracy compared to traditional manpower-based methods. High-resolution images captured by UAVs directly contribute to identifying and quantifying structural defects on facility exteriors, making image quality a critical factor in achieving accurate results. However, motion blur induced by external factors such as vibration, low light conditions, and wind during UAV operation significantly degrades image quality, leading to inaccurate defect detection and quantification. To address this issue, this research proposes a deblurring network using a Generative Adversarial Network (GAN) to eliminate the motion blur effect in UAV images. The GAN-based motion deblur network represents an image inpainting method that leverages generative models to correct blurry artifacts, thereby generating clear images. Unlike previous studies, this proposed approach incorporates deblur and blur learning modules to realistically generate blur images required for training the generative models. The UAV images processed using the motion deblur network are evaluated using a quality assessment method based on local blur map and other well-known image quality assessment (IQA) metrics. Moreover, in the experiment of crack detection utilizing the object detection system, improved detection results are observed when using enhanced images. Overall, this research contributes to improving the quality and accuracy of facility safety inspections conducted with UAV-based inspections by effectively addressing the challenges associated with motion blur effects in UAV-captured images.

An airborne object detection and location system based on deep inference

In recent years, with the development of sensors, communication networks, and deep learning, drones have been widely used in the field of object detection, tracking, and positioning. However, there are inefficient task execution and some complex algorithms still need to rely on large servers, which is intolerable in rescue and traffic scheduling tasks. Designing fast algorithms that can run on the airborne computer can effectively solve the problem. In this paper, an object detection and location system for drones is proposed. We combine the improved object detection algorithm ST-YOLO based on YOLOX and Swin Transformer with the visual positioning algorithm and deploy it on the airborne end by using TensorRT to realize the detection and location of objects during the flight of the drone. Field experiments show that the established system and algorithm are effective.

A Smart IoT Enabled End-to-End 3D Object Detection System for Autonomous Vehicles

Integration of advanced signal processing, image processing, deep learning, edge computing, and the Internet of Things (IoT) into vehicles allows intelligent automated vehicles to navigate autonomously in different environments. It is crucial for reliable and safe driving that an autonomous vehicle can accurately, effectively, and efficiently recognize, perceive, and observe the surrounding environments. Autonomous vehicles comprise advanced sensor technologies such as RGB cameras and LiDaR that produce an extensive data set in the form of RGB images and 3D measurement points, also recognized as a point cloud. It is necessary to understand and interpret collected data information efficiently and to identify other road users, such as pedestrians and vehicles. Thus, we introduced a smart IoT-enabled deep learning based end-to-end 3D object detection system that works in real-time, emphasizing autonomous driving situations. The detection model is based on YOLOv3; firstly, the model is utilized for 2D object detection and then modified for 3D object detection purposes. The presented model uses point cloud, and RGB image data as input and outputs detected bounding boxes with confidence scores and class labels. Experiments are carried out on the Lyft data set; results reveal that the YOLOv3 model achieves high accuracy and outperforms from other state-of-the-art detection models in terms of effectiveness and accuracy. The overall accuracy of the model is 96% and 97% for 2D and 3D object detection, respectively.

Rancang Bangun Deteksi Objek dengan Metode Filter Warna HSV pada Sistem Klasifikasi Kualitas Biji Kopi Berbasis NVIDIA Jetson Nano

The Post-harvest coffee bean selection plays a crucial role in ensuring optimal bean quality during production processes. Currently, this process is manually conducted in Indonesia. Implementing computer vision can enhance the objectivity of the sorting process through machine vision. An effective object detection system is essential to support a prototype coffee bean quality classification system based on NVIDIA Jetson Nano. The Hue Saturation Value (HSV) color filter method proves effective in detecting objects within a given image frame. Performance evaluation is conducted by assessing the alignment between workflow design and system operation. While the webcam-based object detection system successfully deployed, its effectively identifies coffee bean objects, it faces limitations in detecting smaller, dark-colored beans beyond the specified HSV color threshold. These limitations are attributed to the webcam's specifications, including its rolling shutter, which results in a 'jello effect' when dealing with moving objects.

Enhancement of Real-Time Object Detection and Tracking in Collaborative Environment using AI and Mixed Reality

The area of mixed reality has had rapid growth in recent years, with a notable rise in funding. This may be attributed to the rising recognition of the potential advantages associated with the integration of virtual information into the physical environment. The majority of contemporary mixed reality apps that rely on markers use algorithms for local feature identification and tracking. This study aims to enhance the accuracy of object recognition in complicated environment and enable real-time classification operations via the introduction of a unique detection approach known as the lightweight and efficient YOLOv4 model. In the present setting, Computational vision emerges as a very valuable and engaging manifestation of artificial intelligence (AI) that finds widespread application in many aspects of daily existence. The field of computer vision is dedicated to the development of advanced artificial intelligence and computer systems that aim to replace complex elements of the human environment. In recent times, deep neural networks have emerged as a crucial component in several sectors owing to their well-established capacity to process visual input. This study presents a methodology for classifying and identifying objects using the YOLOv4 object detection algorithm. Convolutional neural networks (CNNs) have shown exceptional efficacy in the tasks of object tracking and feature extraction from pictures. Therefore, the enhanced network architecture optimizes both the precision of identification and the speed at which it operates. This research will contribute to developing mixed-reality simulations system for object detection and tracking in collaborative environment that are accessible to everyone, including users in the architectural filed. The model was evaluated in comparison to other object detection approaches. Based on the empirical results, it was observed that the YOLOv4 model exhibited a mean average precision (mAP) of 0.988, surpassing the performance of both YOLOv3 and other object identification models.

Review of Adversarial Attacks in Object Detection

Object detection, a fundamental element of computer vision and artificial intelligence, has experienced considerable advancements through the incorporation of deep learning-based techniques. Yet, despite the impressive strides in both accuracy and efficiency, object detection algorithms harbor inherent vulnerabilities to adversarial attacks. These well-crafted disruptions pose significant risks, especially considering the broad application of object detection across an array of safety-critical sectors such as autonomous transportation, medical imaging, and security systems. This comprehensive paper offers a thorough review of adversarial attacks against object detection systems, dissecting the methods employed, and scrutinizing the implications of their exploits. It dives deep into the mechanics and consequences of both white-box and black-box attacks on prevalent object detection networks, including but not limited to Faster R-CNN, YOLO, and SSD. Furthermore, this paper underscores an assortment of defense strategies developed to mitigate the effects of adversarial attacks. These include adversarial training, gradient masking, input transformations, and randomized defenses. While these strategies hold promise, it is acknowledged that they have their limitations and do not offer a universal solution against all adversarial attacks. As such, this paper underscores the urgent necessity for robust defense mechanisms and stimulates further discourse and investigation into developing truly resilient object detection systems, capable of withstanding the growing threat of adversarial attacks.

Electric vehicle wireless charging system for the foreign object detection with the inducted coil with magnetic field variation

Abstract Foreign object detection is one of the most critical issues in electric vehicle wireless charging systems. This article proposes a foreign object detection scheme with an induced coil in the charging system. The proposed method uses a multifunctional tunneling resistance sensor matrix to detect the presence of a foreign metal object between the coils. An asymmetrical induction coil design scheme is proposed to eliminate the blind area. The suggested method utilizes size-modulated c-shaped coil units to remove invisible zones that result from the magnetic field’s axial uniformity. The induced voltage in the transmission coil is measured using ANSYS/MAXWELL software. The experimental results show that the suggested method has a number of benefits over regular even-sensing coils, including higher uniformity in the induced current, position-dependent detection sensitivity, and detection accuracy. It provides a feasible and affordable way to get around the drawbacks of the traditional detecting coil.

FenceTalk: Exploring False Negatives in Moving Object Detection

Deep learning models are often trained with a large amount of labeled data to improve the accuracy for moving object detection in new fields. However, the model may not be robust enough due to insufficient training data in the new field, resulting in some moving objects not being successfully detected. Training with data that is not successfully detected by the pre-trained deep learning model can effectively improve the accuracy for the new field, but it is costly to retrieve the image data containing the moving objects from millions of images per day to train the model. Therefore, we propose FenceTalk, a moving object detection system, which compares the difference between the current frame and the background image based on the structural similarity index measure (SSIM). FenceTalk automatically selects suspicious images with moving objects that are not successfully detected by the Yolo model, so that the training data can be selected at a lower labor cost. FenceTalk can effectively define and update the background image in the field, reducing the misjudgment caused by changes in light and shadow, and selecting images containing moving objects with an optimal threshold. Our study has demonstrated its performance and generality using real data from different fields. For example, compared with the pre-trained Yolo model using the MS COCO dataset, the overall recall of FenceTalk increased from 72.36% to 98.39% for the model trained with the data picked out by SSIM. The recall of FenceTalk, combined with Yolo and SSIM, can reach more than 99%.

Comparison of the Effectiveness of Yolov5 and yolov8 Algorithms for Detecting Human Personal Protective Equipment

The article deals with the comparative analysis of different architectures of convolutional neural networks of YOLO class: YOLOv5 and YOLOv8. Neural network was trained on a dataset consisting of more than 2,200 marked digital images. Detection and recognition of 8 classes of personal protective equipment such as medical masks, gloves, helmets, goggles, uniforms and others was performed. To evaluate the efficiency of the object detection algorithms such characteristics as accuracy metrics, learning time and memory size of the neural networks were used. The research confirms the high efficiency and accuracy of the YOLO algorithms in the detection and recognition of objects in digital still and video images. The research shows that the average accuracy of mAP object detection in YOLOv8 is 3% higher than in YOLOv5, while the performance in the new algorithm has increased by more than 50% compared with the earlier version of the neural network. The results obtained can be used to improve the object detection system in various fields, such as the automotive industry, medical and scientific research, the security field, and etc. Based on the results of the experiment, conclusions were made regarding the selection of an algorithm for the detection of human personal protective equipment. Further research in this area may be based on expanding the volume of training datasets to improve the accuracy of object recognition and evaluate the performance of algorithms on large amounts of data. Further research is planned in the area of optimizing the architecture of convolutional neural networks to improve the efficiency, speed and accuracy of object detection in digital images.

Underwater Detection of Ancient Pottery Sherds Using Deep Learning

This paper outlines the creation of a machine learning model designed to identify ancient pottery fragments near a submerged shipwreck off Modi Island, Greece. We trained multiple iterations of the YOLOv8 model using a custom dataset comprised of underwater videos taken during diving expeditions at the wreck site. The primary goal of this research is to integrate the resulting object detection system into a remotely operated vehicle (ROV) for automated pottery shard recognition, thereby aiding archaeological excavations. The paper elaborates on the model's development methodology and presents comprehensive experimental and evaluative results. These findings underscore the model's potential to significantly enhance the efficiency and accuracy of underwater archaeological exploration and analysis.

Small object detection (SOD) system for comprehensive construction site safety monitoring

Although object detection is essential for recognizing hazardous situations in construction sites where various objects coexist, existing systems fail to ensure real-time accuracy and flexibility in detecting small objects in various scene scales. Therefore, a small object detection (SOD) system was developed based on the YOLOv5 algorithm for comprehensive site monitoring. The proposed SOD simultaneously crops images into multiple segments for small object detection set by the user's desired flexibility while gaining real-time inference in edge computing environments. The SOD outperforms existing systems, especially regarding small object detection accuracy and flexibility for detecting objects of different sizes. The SOD can detect multi-scale objects not initially detected by existing methods (i.e., workers) to large construction equipment without much inference time lost in the edge device. The proposed system facilitates real-time site monitoring by correcting existing system limitations, thereby improving site monitoring and safety management.

3D lidar SLAM-based systems in object detection and navigation applications

ABSTRACT This paper considered an object detection system based on 3D LiDAR Sensor and Simultaneous Localization and Mapping (SLAM) to complete the navigation applications of mobile robots. A 3D-based SLAM with lightweight and ground-optimized Lidar odometry and mapping (LeGO-LOAM) appropriately generated the environmental maps. SLAM is a tool used to obtain information from the environment, allowing mobile robots to know their location. Indoor environment data is immedicably created while SLAM is processing the information. The dynamic object detection algorithm depends on the available information to realize the external morphology and circle the bounding box of moving objects. Therefore, a wheeled mobile robot (WMR) was employed to dynamically trace the object’s movement direction. Finally, This study found that the quantum genetic algorithm (QGA) is more efficient in generating a shorter path than the particle swarm optimization, and a dynamic window approach (DWA) is immediately detected as a dynamic obstacle. Therefore, WMR obtains enough object, obstacle, and routing information to effectively and safely reach the destination through the Move_base software package in Robot Operating System.