Deep Learning-based Object Detection Research Articles

The evolution of object detection methods

Object detection is one of the most important domains in computer vision tasks, which is an important branch of artificial intelligence. It aims at finding and locating the accurate position of objects in given pictures or videos. With the development of deep learning techniques, more powerful and robust algorithms have emerged to deal with multi-scale, high-level features to overcome the limitations of traditional pipeline of object detectors. The popularity of transformer framework enables larger capacity datasets by processing self-attention mechanism, and the object detection methods have evolved into a new era. This paper first reviews traditional object detection pipeline and brief history of deep learning, afterwards it focuses on the classification of deep learning-based object detection methods covering Convolution Neural Network based and transformer-based methods. Commonly used datasets and metrics are also covered in the next part. The Convolution Neural Network based methods mainly contain two-stage and one-stage detectors, Convolution Neural Network is the underlying structure of these methods convolutional stages are fundamental parts. Transformer-based models convert traditional object detection issues into end-to-end detection, which is widely used in dealing with images. Finally, the promising future of object detection areas are listed to show guidance on future work.

Deep Learning-Based Fish Detection Using Above-Water Infrared Camera for Deep-Sea Aquaculture: A Comparison Study.

Long-term, automated fish detection provides invaluable data for deep-sea aquaculture, which is crucial for safe and efficient seawater aquafarming. In this paper, we used an infrared camera installed on a deep-sea truss-structure net cage to collect fish images, which were subsequently labeled to establish a fish dataset. Comparison experiments with our dataset based on Faster R-CNN as the basic objection detection framework were conducted to explore how different backbone networks and network improvement modules influenced fish detection performances. Furthermore, we also experimented with the effects of different learning rates, feature extraction layers, and data augmentation strategies. Our results showed that Faster R-CNN with the EfficientNetB0 backbone and FPN module was the most competitive fish detection network for our dataset, since it took a significantly shorter detection time while maintaining a high AP50 value of 0.85, compared to the best AP50 value of 0.86 being achieved by the combination of VGG16 with all improvement modules plus data augmentation. Overall, this work has verified the effectiveness of deep learning-based object detection methods and provided insights into subsequent network improvements.

Multi-nanoparticle recognition and tracking based on SPTGAN-YOLOv3 modeling

Nanoparticle tracking techniques based on point spread function (PSF) engineering can monitor the movement of nanoparticles in 3D space in real time. However, increasing the axial tracking depth decreases the signal-to-noise ratio (SNR), which makes the reconstruction of nanoparticle trajectories more difficult. This paper proposes a deep learning-based object detection model SPTGAN-YOLOv3 for nanoparticle tracking technique. The SPTGAN-YOLOv3 model utilizes a noisy image with a low SNR as its input. It learns the mapping from noisy to noise-free images, generating a high SNR image. This rendered image is then used as the input for the object detection network, enabling the recognition and tracking of nanoparticles. Comparative analysis with conventional particle tracking methods demonstrates that the SPTGAN-YOLOv3 model significantly improves the precision and efficiency in recognizing and tracking multiple nanoparticles. Moreover, experimental results indicate that this technique notably enhances the SNR of nanoparticle tracking images and enables precise identification of nanoparticles.

Development of an AI-based image/ultrasonic convergence camera system for accurate gas leak detection in petrochemical plants

Outdoor pipeline leaks are difficult to accurately measure using existing concentration measurement systems installed in petrochemical plants owing to external air currents. Besides, leak detection is only possible for a specific gas. The purpose of this study was to develop an image/ultrasonic convergence camera system that incorporates artificial intelligence (AI) to improve pipe leak detection and establish a real-time monitoring system. Our system includes an advanced ultrasonic camera coupled with a deep learning-based object-detection algorithm trained on pipe image data from petrochemical plants. The collected data improved the accuracy of detected gas leak localization through deep learning. Our detection model achieves an mAP50 (Mean average precision calculated at an intersection over union (IoU) threshold of 0.50)score of 0.45 on our data and is able to detect the majority of leak points within a system. The petrochemical plant environment was simulated by visiting petrochemical plants and reviewing drawings, and an outdoor experimental demonstration site was established. Scenarios such as flange connection failure were set under medium-/low-pressure conditions, and the developed product was experimented under gas leak conditions that simulated leakage accidents. These experiments enabled the removal of potentially confounding surrounding noise sources, which led to the false detection of actual gas leaks using the AI piping detection technique.

Automated License Plate Recognition for Non-Helmeted Motor Riders Using YOLO and OCR

One of the leading causes of serious injuries in motorcycle accidents is the failure to wear a helmet, pressing the need for effective measures to encourage riders to use helmets. To stop these frequent violations of business regulations, regular observation is necessary. The proposed system is for detecting traffic violations in India related to riding motorcycles without helmets. The system utilizes deep learning-based object detection using YOLO and OCR techniques to automatically detect non-helmet riders and extract license plate numbers. The proposed system aims to improve efficiency and accuracy in detecting violations by automating the process and reducing the need for manpower. The system involves three levels of object detection: person and motorcycle/moped, helmet, and license plate. OCR is used to extract the license plate number, and all techniques are subject to predefined conditions and constraints. The system is designed to operate on video input to ensure high-speed execution, and it intends to offer a complete solution for both detection of helmet and extracting the license plate number.

Loose to compact feature alignment for domain adaptive object detection

Recently, great achievements have been made for deep learning based object detection methods. But their performance drops significantly when domain shifts occur. To address this problem, in this work we propose a loose to compact feature alignment method under an unsupervised domain adaptation framework. The entire feature alignment is performed in a divide and conquer manner, so as to distribute the alignment difficulties at two steps. At the first step, we loosen the goal at both image and instance levels. At the image level, a new Mask Guided Foreground Alignment (MGFA) module is proposed to make the alignment focus more on easier foreground regions, leaving the more diverse and more difficult background regions to the second step; at the instance level, we propose a Class-Wise Instance Alignment (CWIA) module with separated domain classifiers for different categories so as to ease the alignment. At the second step, the alignment is performed per pixel and per instance, achieving a more compact and better aligned feature space. We conduct experiments on three different adaptation scenarios, where we achieve comparable results, demonstrating the effectiveness of our proposed method.

Testing a Vision-Based Autonomous Drone Navigation Model in a Forest Environment

Drones play a pivotal role in various industries of Industry 4.0. For achieving the application of drones in a dynamic environment, finding a clear path for their autonomous flight requires more research. This paper addresses the problem of finding a navigation path for an autonomous drone based on visual scene information. A deep learning-based object detection approach can localize obstacles detected in a scene. Considering this approach, we propose a solution framework that includes masking with a color-based segmentation method to identify an empty area where the drone can fly. The scene is described using segmented regions and localization points. The proposed approach can be used to remotely guide drones in dynamic environments that have poor coverage from global positioning systems. The simulation results show that the proposed framework with object detection and the proposed masking technique support drone navigation in a dynamic environment based only on the visual input from the front field of view.

Performance analysis of deep learning-based object detection algorithms on COCO benchmark: a comparative study

This paper thoroughly explores the role of object detection in smart cities, specifically focusing on advancements in deep learning-based methods. Deep learning models gain popularity for their autonomous feature learning, surpassing traditional approaches. Despite progress, challenges remain, such as achieving high accuracy in urban scenes and meeting real-time requirements. The study aims to contribute by analyzing state-of-the-art deep learning algorithms, identifying accurate models for smart cities, and evaluating real-time performance using the Average Precision at Medium Intersection over Union (IoU) metric. The reported results showcase various algorithms’ performance, with Dynamic Head (DyHead) emerging as the top scorer, excelling in accurately localizing and classifying objects. Its high precision and recall at medium IoU thresholds signify robustness. The paper suggests considering the mean Average Precision (mAP) metric for a comprehensive evaluation across IoU thresholds, if available. Despite this, DyHead stands out as the superior algorithm, particularly at medium IoU thresholds, making it suitable for precise object detection in smart city applications. The performance analysis using Average Precision at Medium IoU is reinforced by the Average Precision at Low IoU (APL), consistently depicting DyHead’s superiority. These findings provide valuable insights for researchers and practitioners, guiding them toward employing DyHead for tasks prioritizing accurate object localization and classification in smart cities. Overall, the paper navigates through the complexities of object detection in urban environments, presenting DyHead as a leading solution with robust performance metrics.

A comparison of deep learning-based object detection for unmanned aerial vehicle

Unmanned aerial vehicle (UAV) plays a critical role in the field of object detection, and machine learning techniques have significantly advanced the field in recent years. This paper provides a comprehensive overview of machine learning developments for UAV object detection. The process involves multiple steps. Firstly, deep learning and Convolutional Neural Network (CNN) are widely utilized to extract precise features and train object detection models for accurate classification and efficient recognition of target objects in images and videos. Secondly, classical object detection algorithms such as Faster R-CNN, You Only Look Once (YOLO), and Single Shot MultiBox Detector (SSD), have been enhanced to improve accuracy and real-time performance. In this review, we primarily focus on comparing the principles of CNN and YOLOv5 themselves, as well as their applications in object detection and image recognition. Ultimately, it becomes apparent that CNN is better suited for processing image data, automatically extracting features, and achieving more accurate classification and detection. On the other hand, YOLOv5 directly performs detection on large images, significantly reducing computation time compared to CNN.

Semi-supervised learning approach for construction object detection by integrating super-resolution and mean teacher network

Deep learning-based object detection methods are utilized for safety management at construction sites, which require large-scale, high-quality, and well-labeled datasets for training. The existing construction datasets are relatively small due to the high expense of labor-intensive annotation, and the varying quality of the construction images also affects the detection performance of the model. To address the limitations of datasets, this study proposes a new method for construction object detection by integrating super-resolution and semi-supervised learning. The proposed method improves the quality of construction images and achieves excellent detection performance with limited labeled data. First, the Real-ESRGAN model is introduced to improve the quality of construction images and make the construction objects visible. The proposed super-resolution method can enhance the texture details of low-resolution images, hence improving the performance of object detection models. Second, the mean-teacher network is adopted to expand the training set, thus avoiding the labor-intensive annotation work. To verify the effectiveness of the proposed method, the method is applied to the state-of-the-art Yolov5 object detection model, and construction images from the Site Object Detection Dataset (SODA) with different labeled data proportions (from 10% to 50% in 10% intervals with an extreme case of 5%) are used as the training set. By comparing with the existing supervised learning method, it is shown that the proposed method can achieve better detection performance. In particular, the method is more effective in enhancing detection performance when the proportion of the labeled data is smaller, which is of great practical value in real-world engineering. The experimental results show the potential of the proposed method in improving image quality and reducing the expense of developing construction datasets.

Comparative Analysis of Deep Learning Models for Olive Detection on the Branch

The future of deep learning integration in agriculture holds great potential for advancing sustainable agricultural practices, precision agriculture and improved decision-making. With the rapid development of image processing and artificial intelligence technologies in recent years, deep learning has begun to play a major role in identifying agricultural pests and optimizing agricultural product marketing. However, there are challenges related to data quality, model scalability, and geographical limitations for widespread adoption of deep learning in agriculture. This study on Olive was conducted to improve the quality of the data set and to ensure more reliable training of object detection models. According to the result of the training process of YOLOv7 used in the study, it was concluded that it was characterized by decreasing loss values and showed an increase in the model's ability to detect objects correctly. It was observed that the other model, YOLOv8l, had a more effective learning capacity and a tendency to learn faster. The performance of both models was evaluated with various metrics, and it was determined that YOLOv8l had higher Precision, Recall, and mAP values. It was emphasized that YOLOv8l showed high performance even in low epoch numbers and can be preferred especially in cases where time and computational resources were limited. It was determined that YOLOv7 made detections in a wide confidence range, but had difficulty in detections with low confidence scores. It was observed that YOLOv8l made more stable and reliable detections with higher confidence scores. The metric data of the "YOLOv8l" model was found to be higher compared to other models. The F1 score of the YOLOv5l model was 92.337%, precision 96.568%, recall %88,462,mAP@0.5:0.65 value gave the highest score with 94.608%. This research on deep learning-based object detection models indicated that YOLOv8l showed superior performance compared to YOLOv7 and was a more reliable option for agricultural applications.

Deep Learning Based Object Detection with Unmanned Aerial Vehicle Equipped with Embedded System

This study aims to introduce an Unmanned Aerial Vehicle (UAV) platform capable of performing real-time object detection and classification tasks using computer vision techniques in the field of artificial intelligence. Previous scientific research reveals the utilization of two different methods for object detection and classification via UAVs. One of these methods involves transmitting the acquired UAV images to a ground control center for processing, whereafter the processed data is relayed back to the UAV. The other approach entails transferring images over the internet to a cloud system, where image processing is conducted, and the resultant data is subsequently sent back to the UAV. This allows the UAV to autonomously perform predefined tasks. Enabling the UAV with autonomous decision-making capabilities and the ability to perform object detection and classification from recorded images requires an embedded artificial intelligence module. The ability of the UAV to utilize image processing technologies through embedded systems significantly enhances its object detection and classification capabilities, providing it with a significant advantage. This enables the UAV to be used more effectively and reliably in various tasks. In the proposed approach, image processing was achieved by mounting a Raspberry Pi 4 and camera on the UAV. Additionally, a Raspberry Pi-compatible 4G/LTE modem kit was used to provide remote intervention capability, and the Coral Edge TPU auxiliary processor was used to increase object detection speed. The TensorFlow Library and the SSD MobilNetV2 convolutional neural network model were used for image processing. During test flights, accuracy values of approximately 96.3% for car detection and 96.2% for human detection were achieved.

Video frame feeding approach for validating the performance of an object detection model in real-world conditions

The challenge of evaluating deep learning-based object detection models in complex traffic scenarios, characterized by changing weather and lighting conditions, is addressed in this study. Real-world testing proves time and cost-intensive, leading to the proposal of a Video Frame Feeding (VFF) approach as a solution. The proposed Video Frame Feeding approach acts as a bridge between object detection models and simulated environments, enabling the generation of realistic scenarios. Leveraging the CarMaker (CM) tool to simulate realistic scenarios, the framework utilizes a virtual camera to capture the simulated environment and feed video frames to an object identification model. The VFF algorithm, with automated validation using simulated ground truth data, enhances detection accuracy to over 95% at 30 frames per second within 130 meters. Employing the You Only Look Once (YOLO) version 4 and the German Traffic Sign Recognition Benchmark dataset, the study assesses a traffic signboard identification model across various climatic conditions. Notably, the VFF algorithm improves accuracy by 2% to 5% in challenging scenarios like foggy days and nights. This innovative approach not only identifies object detection issues efficiently but also offers a versatile solution applicable to any object detection model, promising improved dataset quality and robustness for enhanced model performance.

Automatic Detection of Rice Blast Fungus Spores by Deep Learning-Based Object Detection: Models, Benchmarks and Quantitative Analysis

The severity of rice blast and its impacts on rice yield are closely related to the inoculum quantity of Magnaporthe oryzae, and automatic detection of the pathogen spores in microscopic images can provide a rapid and effective way to quantify pathogen inoculum. Traditional spore detection methods mostly rely on manual feature extraction and shallow machine learning models, and are mostly designed for the indoor counting of a single spore class, which cannot handle the interference of impurity particles in the field. This study achieved automatic detection of rice blast fungus spores in the mixture with other fungal spores and rice pollens commonly encountered under field conditions by using deep learning based object detection techniques. First, 8959 microscopic images of a single spore class and 1450 microscopic images of mixed spore classes, including the rice blast fungus spores and four common impurity particles, were collected and labelled to form the benchmark dataset. Then, Faster R-CNN, Cascade R-CNN and YOLOv3 were used as the main detection frameworks, and multiple convolutional neural networks were used as the backbone networks in training of nine object detection algorithms. The results showed that the detection performance of YOLOv3_DarkNet53 is superior to the other eight algorithms, and achieved 98.0% mean average precision (intersection over union > 0.5) and an average speed of 36.4 frames per second. This study demonstrated the enormous application potential of deep object detection algorithms in automatic detection and quantification of rice blast fungus spores.

YOLOv7-TS: A Traffic Sign Detection Model Based on Sub-Pixel Convolution and Feature Fusion

In recent years, significant progress has been witnessed in the field of deep learning-based object detection. As a subtask in the field of object detection, traffic sign detection has great potential for development. However, the existing object detection methods for traffic sign detection in real-world scenes are plagued by issues such as the omission of small objects and low detection accuracies. To address these issues, a traffic sign detection model named YOLOv7-Traffic Sign (YOLOv7-TS) is proposed based on sub-pixel convolution and feature fusion. Firstly, the up-sampling capability of the sub-pixel convolution integrating channel dimension is harnessed and a Feature Map Extraction Module (FMEM) is devised to mitigate the channel information loss. Furthermore, a Multi-feature Interactive Fusion Network (MIFNet) is constructed to facilitate enhanced information interaction among all feature layers, improving the feature fusion effectiveness and strengthening the perception ability of small objects. Moreover, a Deep Feature Enhancement Module (DFEM) is established to accelerate the pooling process while enriching the highest-layer feature. YOLOv7-TS is evaluated on two traffic sign datasets, namely CCTSDB2021 and TT100K. Compared with YOLOv7, YOLOv7-TS, with a smaller number of parameters, achieves a significant enhancement of 3.63% and 2.68% in the mean Average Precision (mAP) for each respective dataset, proving the effectiveness of the proposed model.

Deep Learning for Midfacial Fracture Detection in CT Images.

This study deploys the deep learning-based object detection algorithms to detect midfacial fractures in computed tomography (CT) images. The object detection models were created using faster R-CNN and RetinaNet from 2,000 CT images. The best detection model, faster R-CNN, yielded an average precision of 0.79 and an area under the curve (AUC) of 0.80. In conclusion, faster R-CNN model has good potential for detecting midfacial fractures in CT images.

Performance Analysis of YOLO Architectures for Surgical Waste Detection in Post-COVID-19 Medical Waste Management

In the wake of the COVID-19 outbreak, there has been a dramatic uptick in the need for efficient medical waste management, making it imperative that more surgical waste management systems are developed. Used surgical masks and gloves are examples of potentially infectious materials that are the subject of this research. By utilizing its real-time object detection capabilities, the You Only Look Once (YOLO) deep learning-based object detection algorithm is used to identify surgical waste. Using the MSG dataset, a deep dive into the performance of three different YOLO architectures (YOLOv5, YOLOv7, and YOLOv8) was undertaken. According to the findings, YOLOv5-s, YOLOv7-x, and YOLOv8-m all perform exceptionally well when it comes to identifying surgical waste. YOLOv8-m was the best model, with a mAP of 82.4%, among these three. To mitigate post-COVID-19 infection risks and improve waste management efficiency, these results can be used to the creation of automated systems for medical waste sorting.

Deep Learning-Based Object Detection and Classification for Autonomous Vehicles in Different Weather Scenarios of Quebec, Canada

Deep Learning-Based Object Detection and Classification for Autonomous Vehicles in Different Weather Scenarios of Quebec, Canada

Deep Learning-Based Localization and Orientation Estimation of Pedicle Screws in Spinal Fusion Surgery.

This study investigated the application of a deep learning-based object detection model for accurate localization and orientation estimation of spinal fixation surgical instruments during surgery. We employed the You Only Look Once (YOLO) object detection framework with oriented bounding boxes (OBBs) to address the challenge of non-axis-aligned instruments in surgical scenes. The initial dataset of 100 images was created using brochure and website images from 11 manufacturers of commercially available pedicle screws used in spinal fusion surgeries, and data augmentation was used to expand 300 images. The model was trained, validated, and tested using 70%, 20%, and 10% of the images of lumbar pedicle screws, with the training process running for 100 epochs. The model testing results showed that it could detect the locations of the pedicle screws in the surgical scene as well as their direction angles through the OBBs. The F1 score of the model was 0.86 (precision: 1.00, recall: 0.80) at each confidence level and mAP50. The high precision suggests that the model effectively identifies true positive instrument detections, although the recall indicates a slight limitation in capturing all instruments present. This approach offers advantages over traditional object detection in bounding boxes for tasks where object orientation is crucial, and our findings suggest the potential of YOLOv8 OBB models in real-world surgical applications such as instrument tracking and surgical navigation. Future work will explore incorporating additional data and the potential of hyperparameter optimization to improve overall model performance.

Learning to Holistically Detect Bridges From Large-Size VHR Remote Sensing Imagery.

Bridge detection in remote sensing images (RSIs) plays a crucial role in various applications, but it poses unique challenges compared to the detection of other objects. In RSIs, bridges exhibit considerable variations in terms of their spatial scales and aspect ratios. Therefore, to ensure the visibility and integrity of bridges, it is essential to perform holistic bridge detection in large-size very-high-resolution (VHR) RSIs. However, the lack of datasets with large-size VHR RSIs limits the deep learning algorithms' performance on bridge detection. Due to the limitation of GPU memory in tackling large-size images, deep learning-based object detection methods commonly adopt the cropping strategy, which inevitably results in label fragmentation and discontinuous prediction. To ameliorate the scarcity of datasets, this paper proposes a large-scale dataset named GLH-Bridge comprising 6,000 VHR RSIs sampled from diverse geographic locations across the globe. These images encompass a wide range of sizes, varying from 2,048 × 2,048 to 16,384 × 16,384pixels, and collectively feature 59,737 bridges. These bridges span diverse backgrounds, and each of them has been manually annotated, using both an oriented bounding box (OBB) and a horizontal bounding box (HBB). Furthermore, we present an efficient network for holistic bridge detection (HBD-Net) in large-size RSIs. The HBD-Net presents a separate detector-based feature fusion (SDFF) architecture and is optimized via a shape-sensitive sample re-weighting (SSRW) strategy. The SDFF architecture performs inter-layer feature fusion (IFF) to incorporate multi-scale context in the dynamic image pyramid (DIP) of the large-size image, and the SSRW strategy is employed to ensure an equitable balance in the regression weight of bridges with various aspect ratios. Based on the proposed GLH-Bridge dataset, we establish a bridge detection benchmark including the OBB and HBB tasks, and validate the effectiveness of the proposed HBD-Net. Additionally, cross-dataset generalization experiments on two publicly available datasets illustrate the strong generalization capability of the GLH-Bridge dataset. The dataset and source code will be released at https://luo-z13.github.io/GLH-Bridge-page/.