Deep Learning Object Detection Research Articles

Multidimensional Evaluation Methods for Deep Learning Models in Target Detection for SAR Images

As artificial intelligence technology advances, the application of object detection technology in the field of SAR (synthetic aperture radar) imagery is becoming increasingly widespread. However, it also faces challenges such as resource limitations in spaceborne environments and significant uncertainty in the intensity of interference in application scenarios. These factors make the performance evaluation of object detection key to ensuring the smooth execution of tasks. In the face of such complex and harsh application scenarios, methods that rely on single-dimensional evaluation to assess models have had their limitations highlighted. Therefore, this paper proposes a multi-dimensional evaluation method for deep learning models used in SAR image object detection. This method evaluates models in a multi-dimensional manner, covering the training, testing, and application stages of the model, and constructs a multi-dimensional evaluation index system. The training stage includes assessing training efficiency and the impact of training samples; the testing stage includes model performance evaluation, application-based evaluation, and task-based evaluation; and the application stage includes model operation evaluation and model deployment evaluation. The evaluations of these three stages constitute the key links in the performance evaluation of deep learning models. Furthermore, this paper proposes a multi-indicator comprehensive evaluation method based on entropy weight correlation scaling, which calculates the weights of each evaluation indicator through test data, thereby providing a balanced and comprehensive evaluation mechanism for model performance. In the experiments, we designed specific interferences for SAR images in the testing stage and tested three models from the YOLO series. Finally, we constructed a multi-dimensional performance profile diagram for deep learning object detection models, providing a new visualization method to comprehensively characterize model performance in complex application scenarios. This can provide more accurate and comprehensive model performance evaluation for remote sensing data processing, thereby guiding model selection and optimization. The evaluation method proposed in this study adopts a multi-dimensional perspective, comprehensively assessing the three core stages of a model’s lifecycle: training, testing, and application. This framework demonstrates significant versatility and adaptability, enabling it to transcend the boundaries of remote sensing technology and provide support for a wide range of model evaluation and optimization tasks.

Tongue feature dataset construction and real-time detection.

Tongue diagnosis in traditional Chinese medicine (TCM) provides clinically important, objective evidence from direct observation of specific features that assist with diagnosis. However, the current interpretation of tongue features requires a significant amount of manpower and time. TCM physicians may have different interpretations of features displayed by the same tongue. An automated interpretation system that interprets tongue features would expedite the interpretation process and yield more consistent results. This study applied deep learning visualization to tongue diagnosis. After collecting tongue images and corresponding interpretation reports by TCM physicians in a single teaching hospital, various tongue features such as fissures, tooth marks, and different types of coatings were annotated manually with rectangles. These annotated data and images were used to train a deep learning object detection model. Upon completion of training, the position of each tongue feature was dynamically marked. A large high-quality manually annotated tongue feature dataset was constructed and analyzed. A detection model was trained with average precision (AP) 47.67%, 58.94%, 71.25% and 59.78% for fissures, tooth marks, thick and yellow coatings, respectively. At over 40 frames per second on a NVIDIA GeForce GTX 1060, the model was capable of detecting tongue features from any viewpoint in real time. This study constructed a tongue feature dataset and trained a deep learning object detection model to locate tongue features in real time. The model provided interpretability and intuitiveness that are often lacking in general neural network models and implies good feasibility for clinical application.

Intra-class deep learning object detection on embedded computer system

<span lang="EN-US">Implementation of artificial intelligence tends to be portable, mobile and embeds in embedded computer system (EBD). EBD is a special-purpose computer with limited capacity in a small-form size. Deep learning (DL) had known as cutting edges for object recognition. With DL, object feature extraction analysis is omitted. DL requires large computing resources and capacity. Implement DL algorithm on EBD goal to achieves high detection accuracy and high-efficiency resources. Hence, be able to cope with intra-class variations, and image disturbances. By those challenges and limitations, this study reports the performance of EBD to recognize an object which has high variations in their class, through an optimal raw-input dataset. The raw-input dataset performed optimization process with a supervisor. Yield is the proper optimal input dataset in size. The performance results observed begin from training dataset until evaluation stage of DL. The comparison performs in efficiency resources, loss, validation-loss, timesteps, and detection accuracy by multiclass confusion matrix analysis. This study shows through this purpose method efficient resources are highly archived. Shorter timesteps ensure training stage is successful, and detection accuracy is perfectly archived. In addition, this study proves DL method archived great performances in classifying object that has identical structure.</span>

You only look once model-based object identification in computer vision

<span lang="EN-US">You only look once version 4 (YOLOv4) is a deep-learning object detection algorithm. It is used to decrease parameters and simplify network structures, making it suited for mobile and embedded device development. The YOLO detector can foresee an object's Class, bounding box, and probability of that Object's Class being found inside that bounding box. A probability value for each bounding box represents the likelihood of a given item class in that bounding box. Global features, channel attention, and special attention are also applied to extract more compelling information. Finally, the model combines the auxiliary and backbone networks to create the YOLOv4's entire network topology. Using custom functions developed upon YOLOv4, we get the count of the objects and a crop around the objects detected with a confidence score that specifies the probability of the thing seen being the same Class as predicted by YOLOv4. A confidence threshold is implemented to eliminate the detections with low confidence. </span>

Precast concrete project image dataset for deep learning object detection

Computer vision technology in precast concrete (PC) projects has the potential to enhance site management but faces challenges due to the lack of specialized datasets for this field. This study developed a PC components dataset for object detection (PCCODA) in four steps: component range selection, image collection, image preprocessing, and labeling. The performance of PCCODA was assessed using multiple algorithms, including You Only Look Once (YOLO), YOLO X, faster region-based convolutional neural networks (Faster R-CNN), and Double Head R-CNN. The dataset resulted in average accuracies of 0.88–0.97, average robustness of 0.84–0.96, and average detection speed of 3.64–23.3 frames per second. This performance fulfills the level required by most studies on construction automation. The applicability of the dataset was tested at a real site. This study contributes to the off-site construction project management theory and enhances productivity in PC projects by automating object detection-related tasks.

A Deep-Learning-Based Quality Control Evaluation Method for CT Phantom Images

Computed tomography (CT) is a rapid and precise medical imaging modality, but it poses the challenge of high radiation exposure to patients. To control this issue, stringent quality control (QC) evaluations are imperative for CT. One crucial aspect of CT QC involves the evaluation of phantom images, utilizing specifically designed phantoms for accuracy management and subsequent objective evaluation. However, CT QC has qualitative evaluation methods, particularly for evaluating spatial and contrast resolutions. To solve this problem, we propose a quality control method based on deep-learning object detection for quantitatively evaluating spatial and contrast resolutions, CT Attention You Only Look Once v8 (CTA-YOLOv8). First, we utilized the YOLOv8 network as the foundational model, optimizing it for enhanced accuracy. Second, we enhanced the network’s capabilities by integrating the Convolutional Block Attention Module (CBAM) and Swin Transformers, tailored for phantom image evaluations. The CBAM module was employed internally to pinpoint the optimal position for achieving peak performance in CT QC data. Similarly, we fine-tuned the code and patch size of the Swin Transformer module to align it with YOLOv8, culminating in the identification of the optimal configuration. Our proposed CTA-YOLOv8 network showed superior agreement with qualitative evaluation methods, achieving accuracies of 92.03% and 97.56% for spatial and contrast resolution evaluations, respectively. Thus, we suggest that our method offers nearly equivalent performance to qualitative methods. The utilization of the CTA-YOLOv8 network in evaluating CT phantom images holds potential for setting a new standard in quantitative assessment methodologies.

Sensor-based characterization of construction and demolition waste at high occupancy densities using synthetic training data and deep learning.

Sensor-based monitoring of construction and demolition waste (CDW) streams plays an important role in recycling (RC). Extracted knowledge about the composition of a material stream helps identifying RC paths, optimizing processing plants and form the basis for sorting. To enable economical use, it is necessary to ensure robust detection of individual objects even with high material throughput. Conventional algorithms struggle with resulting high occupancy densities and object overlap, making deep learning object detection methods more promising. In this study, different deep learning architectures for object detection (Region-based CNN/Region-based Convolutional Neural Network (Faster R-CNN), You only look once (YOLOv3), Single Shot MultiBox Detector (SSD)) are investigated with respect to their suitability for CDW characterization. A mixture of brick and sand-lime brick is considered as an exemplary waste stream. Particular attention is paid to detection performance with increasing occupancy density and particle overlap. A method for the generation of synthetic training images is presented, which avoids time-consuming manual labelling. By testing the models trained on synthetic data on real images, the success of the method is demonstrated. Requirements for synthetic training data composition, potential improvements and simplifications of different architecture approaches are discussed based on the characteristic of the detection task. In addition, the required inference time of the presented models is investigated to ensure their suitability for use under real-time conditions.

A deep learning object detection method to improve cluster analysis of two-dimensional data

A deep learning object detection method to improve cluster analysis of two-dimensional data

Automatic caries detection in bitewing radiographs—Part II: experimental comparison

ObjectiveThe objective of this study was to compare the detection of caries in bitewing radiographs by multiple dentists with an automatic method and to evaluate the detection performance in the absence of a reliable ground truth.Materials and methodsFour experts and three novices marked caries using bounding boxes in 100 bitewing radiographs. The same dataset was processed by an automatic object detection deep learning method. All annotators were compared in terms of the number of errors and intersection over union (IoU) using pairwise comparisons, with respect to the consensus standard, and with respect to the annotator of the training dataset of the automatic method.ResultsThe number of lesions marked by experts in 100 images varied between 241 and 425. Pairwise comparisons showed that the automatic method outperformed all dentists except the original annotator in the mean number of errors, while being among the best in terms of IoU. With respect to a consensus standard, the performance of the automatic method was best in terms of the number of errors and slightly below average in terms of IoU. Compared with the original annotator, the automatic method had the highest IoU and only one expert made fewer errors.ConclusionsThe automatic method consistently outperformed novices and performed as well as highly experienced dentists.Clinical significanceThe consensus in caries detection between experts is low. An automatic method based on deep learning can improve both the accuracy and repeatability of caries detection, providing a useful second opinion even for very experienced dentists.

Fusion of Time-of-Flight Based Sensors with Monocular Cameras for a Robotic Person Follower

Human-robot collaboration (HRC) is becoming increasingly important in advanced production systems, such as those used in industries and agriculture. This type of collaboration can contribute to productivity increase by reducing physical strain on humans, which can lead to reduced injuries and improved morale. One crucial aspect of HRC is the ability of the robot to follow a specific human operator safely. To address this challenge, a novel methodology is proposed that employs monocular vision and ultra-wideband (UWB) transceivers to determine the relative position of a human target with respect to the robot. UWB transceivers are capable of tracking humans with UWB transceivers but exhibit a significant angular error. To reduce this error, monocular cameras with Deep Learning object detection are used to detect humans. The reduction in angular error is achieved through sensor fusion, combining the outputs of both sensors using a histogram-based filter. This filter projects and intersects the measurements from both sources onto a 2D grid. By combining UWB and monocular vision, a remarkable 66.67% reduction in angular error compared to UWB localization alone is achieved. This approach demonstrates an average processing time of 0.0183s and an average localization error of 0.14 meters when tracking a person walking at an average speed of 0.21 m/s. This novel algorithm holds promise for enabling efficient and safe human-robot collaboration, providing a valuable contribution to the field of robotics.

Phytoplankton detection and recognition in freshwater digital microscopy images using deep learning object detectors

Water quality can be negatively affected by the presence of some toxic phytoplankton species, whose toxins are difficult to remove by conventional purification systems. This creates the need for periodic analyses, which are nowadays manually performed by experts. These labor-intensive processes are affected by subjectivity and expertise, causing unreliability. Some automatic systems have been proposed to address these limitations. However, most of them are based on classical image processing pipelines with not easily scalable designs. In this context, deep learning techniques are more adequate for the detection and recognition of phytoplankton specimens in multi-specimen microscopy images, as they integrate both tasks in a single end-to-end trainable module that is able to automatize the adaption to such a complex domain. In this work, we explore the use of two different object detectors: Faster R-CNN and RetinaNet, from the one-stage and two-stage paradigms respectively. We use a dataset composed of multi-specimen microscopy images captured using a systematic protocol. This allows the use of widely available optical microscopes, also avoiding manual adjustments on a per-specimen basis, which would require expert knowledge. We have made our dataset publicly available to improve the reproducibility and to foment the development of new alternatives in the field. The selected Faster R-CNN methodology reaches maximum recall levels of 95.35%, 84.69%, and 79.81%, and precisions of 94.68%, 89.30% and 82.61%, for W. naegeliana, A. spiroides, and D. sociale, respectively. The system is able to adapt to the dataset problems and improves the results overall with respect to the reference state-of-the-art work. In addition, the proposed system improves the automation and abstraction from the domain and simplifies the workflow and adjustment.

A Review of Machine Learning and Deep Learning for Object Detection, Semantic Segmentation, and Human Action Recognition in Machine and Robotic Vision

Machine vision, an interdisciplinary field that aims to replicate human visual perception in computers, has experienced rapid progress and significant contributions. This paper traces the origins of machine vision, from early image processing algorithms to its convergence with computer science, mathematics, and robotics, resulting in a distinct branch of artificial intelligence. The integration of machine learning techniques, particularly deep learning, has driven its growth and adoption in everyday devices. This study focuses on the objectives of computer vision systems: replicating human visual capabilities including recognition, comprehension, and interpretation. Notably, image classification, object detection, and image segmentation are crucial tasks requiring robust mathematical foundations. Despite the advancements, challenges persist, such as clarifying terminology related to artificial intelligence, machine learning, and deep learning. Precise definitions and interpretations are vital for establishing a solid research foundation. The evolution of machine vision reflects an ambitious journey to emulate human visual perception. Interdisciplinary collaboration and the integration of deep learning techniques have propelled remarkable advancements in emulating human behavior and perception. Through this research, the field of machine vision continues to shape the future of computer systems and artificial intelligence applications.

Pneumonia detection based on RSNA dataset and anchor-free deep learning detector.

Pneumonia is a highly lethal disease, and research on its treatment and early screening tools has received extensive attention from researchers. Due to the maturity and cost reduction of chest X-ray technology, and with the development of artificial intelligence technology, pneumonia identification based on deep learning and chest X-ray has attracted attention from all over the world. Although the feature extraction capability of deep learning is strong, existing deep learning object detection frameworks are based on pre-defined anchors, which require a lot of tuning and experience to guarantee their excellent results in the face of new applications or data. To avoid the influence of anchor settings in pneumonia detection, this paper proposes an anchor-free object detection framework and RSNA dataset based on pneumonia detection. First, a data enhancement scheme is used to preprocess the chest X-ray images; second, an anchor-free object detection framework is used for pneumonia detection, which contains a feature pyramid, two-branch detection head, and focal loss. The average precision of 51.5 obtained by Intersection over Union (IoU) calculation shows that the pneumonia detection results obtained in this paper can surpass the existing classical object detection framework, providing an idea for future research and exploration.

Automated Identification and Localization of Rail Internal Defects Based on Object Detection Networks

The timely identification of rail internal defects and the application of corresponding preventive measures would greatly reduce catastrophic failures, such as rail breakage. Ultrasonic rail defect detection is the current mainstream rail defect detection method thanks to its advantages of strong penetration, high accuracy, and ease to deploy. The 2D B-scan image output by ultrasonic detectors contains rich features of defects; however, rail engineers manually identify and localize the defect image, which can be time-consuming, and the image may be subject to missing identification or mistakes. This paper adopted state-of-the-art deep learning algorithms for novel B-scan images for the automatic identification and localization of rail internal tracks. First, through image pre-processing of classification, denoising, and augmentation, four categories of defect image datasets were established, namely crescent-shaped fatigue cracks, fishbolt hole cracks, rail web cracks, and rail base transverse cracks; then, four representatives of deep learning object detection networks, YOLOv8, YOLOv5, DETR, and Faster R-CNN, were trained with the defects dataset and further applied to the testing dataset for defect identification; finally, the performances of the three detection networks were compared and evaluated at the data level, the network structure level, and the interference adaptability level, respectively. The results show that the YOLOv8 network can effectively classify and localize four categories of internal rail defects in B-scan images with a 93.3% mean average precision at three images per second, and the detection time is 58.9%, 376.8%, and 123.2% faster than YOLO v5, DETR, and Faster R-CNN, respectively. The proposed approach could ensure the real-time, accurate, and efficient detection and analysis of internal defects to a rail.

Pothole detection for autonomous vehicles using deep learning: a robust and efficient solution

Autonomous vehicles can transform the transportation sector by offering a safer and more effective means of travel. However, the success of self-driving cars depends on their ability to navigate complex road conditions, including the detection of potholes. Potholes pose a substantial risk to vehicles and passengers, leading to potential damage and safety hazards, making their detection a critical task for autonomous driving. In this work, we propose a robust and efficient solution for pothole detection using the “you look only once (YOLO) algorithm of version 8, the newest deep learning object detection algorithm.” Our proposed system employs a deep learning methodology to identify real-time potholes, enabling autonomous vehicles to avoid potential hazards and minimise accident risk. We assess the effectiveness of our system using publicly available datasets and show that it outperforms existing state-of-the-art approaches in terms of accuracy and efficiency. Additionally, we investigate different data augmentation methods to enhance the detection capabilities of our proposed system. Our results demonstrate that YOLO V8-based pothole detection is a promising solution for autonomous driving and can significantly improve the safety and reliability of self-driving vehicles on the road. The results of our study are also compared with the results of YOLO V5.

YOLOv5n++: An edge-based improved YOLOv5n model to detect river floating debris

Rivers serve as vital water sources, maintain ecological equilibrium, and enhance landscapes. However, the looming issue of floating debris stemming from improper waste disposal and illegal discharge, poses an imminent threat to river ecosystems and their aesthetic appeal. Conventional human-led inspections prove labor-intensive, inefficient, and prone to errors. This study introduces an innovative approach for river debris detection, employing Unmanned Aerial Vehicles (UAVs) imagery in conjunction with a refined YOLOv5n model. This approach offers three key contributions. Primarily, the YOLOv5n model is bolstered by integrating the Efficient Channel Attention (ECA) module and reshaping the MobileNetV3 backbone to align with MobileNetV3S, thereby significantly streamlining computational demands and model intricacy. Additionally, precision and speed are augmented by eliminating the detection head for larger targets, while decreasing computational requirements. Subsequently, to counter dataset scarcity, we curate a UAV-derived river debris dataset, encompassing five prevalent debris types, serving as an indispensable resource for method refinement and assessment. Lastly, the upgraded model’s evaluation on Jetson Nano yields an mAP of 87.2%, merely 0.7% lower than the original YOLOv5n model. Remarkably, the refined model achieves substantial reductions of 57.1% in parameters, 52.6% in volume, and 54.8% in GFLOPs. Additionally, inference time is abbreviated to 57.3ms per Jetson Nano image, 13.4ms faster than the original. These findings underscore edge computing’s potential in river restoration. In conclusion, the fusion of deep learning object detection and UAV imagery empowers adept river debris detection.

Ensemble Deep Learning Object Detection Fusion for Cell Tracking, Mitosis, and Lineage

Ensemble Deep Learning Object Detection Fusion for Cell Tracking, Mitosis, and Lineage

Telecommunication Fiber Box Detection Using YOLO in Urban Environment

The Fiber Distribution Panel (FDP) box is an essential piece of internet access hardware because it provides users with high-speed data networking and functions as a cable organizer to reduce wire clutter. After installing the FDP, an inspection must be performed to ensure that all necessary components are present. However, This examination is still done manually; the technician snaps a picture of the panel and sends it to its supervisor for verification, which is time-consuming and often prone to errors. In addition to images captured in low-light and complex environments, it makes it more difficult for humans to identify the components with just a naked eye. On this matter, a much more efficient method to assess the FDP installation work is very much needed. Therefore, using computer vision approaches, we utilize a deep learning algorithm to perform object detection and automate the assessment of FDP installation components based on visual data. One of the deep learning models established in the literature is the You Only Look Once (YOLO) model, a one-stage deep learning object detection algorithm that employs a fully conventional approach to generate highly accurate real-time predictions. This paper uses YOLOv5s to identify the fiber box and its relevant components, even in urban environments. Experimentations show that YOLO successfully identified the installation parts with a mean average precision score of 86% at a 0.5 confidence level, even with limited data.

Telecommunication Fiber Box Detection Using YOLO in Urban Environment

The Fiber Distribution Panel (FDP) box is an essential piece of internet access hardware because it provides users with high-speed data networking and functions as a cable organizer to reduce wire clutter. After installing the FDP, an inspection must be performed to ensure that all necessary components are present. However, This examination is still done manually; the technician snaps a picture of the panel and sends it to its supervisor for verification, which is time-consuming and often prone to errors. In addition to images captured in low-light and complex environments, it makes it more difficult for humans to identify the components with just a naked eye. On this matter, a much more efficient method to assess the FDP installation work is very much needed. Therefore, using computer vision approaches, we utilize a deep learning algorithm to perform object detection and automate the assessment of FDP installation components based on visual data. One of the deep learning models established in the literature is the You Only Look Once (YOLO) model, a one-stage deep learning object detection algorithm that employs a fully conventional approach to generate highly accurate real-time predictions. This paper uses YOLOv5s to identify the fiber box and its relevant components, even in urban environments. Experimentations show that YOLO successfully identified the installation parts with a mean average precision score of 86% at a 0.5 confidence level, even with limited data.

Vegetation and Non-Vegetation Classification Using Object Detection Techniques and Deep Learning from Low/Mixed Resolution Satellite Images

Vegetation cover classification using mixed or low-resolution scalar images is challenging. Fortunately, recently deep learning object detection methods have emerged as a replacement to the conventional machine learning methods for the detection and classification of land use and land cover. This paper presents a deep learning object detection approach for land use and land cover detection using low/mixed resolution satellite images acquired from Google Earth satellite images. Google Earth images are accessible freely using the Google Earth Pro desktop application. Our dataset consists of two (02) classes (vegetation and non-vegetation) with a total of 450 labeled images captured from different parts of Pakistan. We present a comparison of the recent anchor-free object detection model YOLOX with the anchor-based object detection model YOLOR for solving real-time problems. The end-to-end differentiability, efficient GPU utilization, and absence of hand-crafted parameters make anchor-free models a compelling choice in object detection, and yet not been explored on Land cover classification using satellite images. Our experimental study shows that YOLOX delivers an overall accuracy of 83.50% on Vegetation and 86% on Non-Vegetation classes, which outperformed YOLOR by 30% on Vegetation classes and 34% on non-Vegetation classes for our dataset. We also show how an object detection system can be used for Vegetation and Non-Vegetation classification tasks, which can then be used for change monitoring and assisting in developing geographical maps using low/mixed resolution freely available satellite images.