Deep Learning Object Detection Research Articles

Detecting Moving Wildlife Using the Time Difference between Two Thermal Airborne Images

Wildlife damage to agriculture is serious in Japan; therefore, it is important to understand changes in wildlife population sizes. Although several studies have been conducted to detect wildlife from drone images, behavioral changes (such as wildlife escaping when a drone approaches) have been confirmed. To date, the use of visible and near-infrared images has been limited to the daytime because many large mammals, such as sika deer (Cervus nippon), are crepuscular. However, it is difficult to detect wildlife in the thermal images of urban areas that are not open and contain various heat spots. To address this issue, a method was developed in a previous study to detect moving wildlife using pairs of time-difference thermal images. However, the user’s accuracy was low. In the current study, two methods are proposed for extracting moving wildlife using pairs of airborne thermal images and deep learning models. The first method was to judge grid areas with wildlife using a deep learning classification model. The second method detected each wildlife species using a deep learning object detection model. The proposed methods were then applied to pairs of airborne thermal images. The classification test accuracies of “with deer” and “without deer” were >85% and >95%, respectively. The average precision of detection, precision, and recall were >85%. This indicates that the proposed methods are practically accurate for monitoring changes in wildlife populations and can reduce the person-hours required to monitor a large number of thermal remote-sensing images. Therefore, efforts should be made to put these materials to practical use.

An optimised YOLOv4 deep learning model for efficient malarial cell detection in thin blood smear images.

Malaria is a serious public health concern worldwide. Early and accurate diagnosis is essential for controlling the disease's spread and avoiding severe health complications. Manual examination of blood smear samples by skilled technicians is a time-consuming aspect of the conventional malaria diagnosis toolbox. Malaria persists in many parts of the world, emphasising the urgent need for sophisticated and automated diagnostic instruments to expedite the identification of infected cells, thereby facilitating timely treatment and reducing the risk of disease transmission. This study aims to introduce a more lightweight and quicker model-but with improved accuracy-for diagnosing malaria using a YOLOv4 (You Only Look Once v. 4) deep learning object detector. The YOLOv4 model is modified using direct layer pruning and backbone replacement. The primary objective of layer pruning is the removal and individual analysis of residual blocks within the C3, C4 and C5 (C3-C5) Res-block bodies of the backbone architecture's C3-C5 Res-block bodies. The CSP-DarkNet53 backbone is simultaneously replaced for enhanced feature extraction with a shallower ResNet50 network. The performance metrics of the models are compared and analysed. The modified models outperform the original YOLOv4 model. The YOLOv4-RC3_4 model with residual blocks pruned from the C3 and C4 Res-block body achieves the highest mean accuracy precision (mAP) of 90.70%. This mAP is > 9% higher than that of the original model, saving approximately 22% of the billion floating point operations (B-FLOPS) and 23MB in size. The findings indicate that the YOLOv4-RC3_4 model also performs better, with an increase of 9.27% in detecting the infected cells upon pruning the redundant layers from the C3 Res-block bodies of the CSP-DarkeNet53 backbone. The results of this study highlight the use of the YOLOv4 model for detecting infected red blood cells. Pruning the residual blocks from the Res-block bodies helps to determine which Res-block bodies contribute the most and least, respectively, to the model's performance. Our method has the potential to revolutionise malaria diagnosis and pave the way for novel deep learning-based bioinformatics solutions. Developing an effective and automated process for diagnosing malaria will considerably contribute to global efforts to combat this debilitating disease. We have shown that removing undesirable residual blocks can reduce the size of the model and its computational complexity without compromising its precision.

Fire and Smoke Detection Using Fine-Tuned YOLOv8 and YOLOv7 Deep Models

Viewed as a significant natural disaster, wildfires present a serious threat to human communities, wildlife, and forest ecosystems. The frequency of wildfire occurrences has increased recently, with the impacts of global warming and human interaction with the environment playing pivotal roles. Addressing this challenge necessitates the ability of firefighters to promptly identify fires based on early signs of smoke, allowing them to intervene and prevent further spread. In this work, we adapted and optimized recent deep learning object detection, namely YOLOv8 and YOLOv7 models, for the detection of smoke and fire. Our approach involved utilizing a dataset comprising over 11,000 images for smoke and fires. The YOLOv8 models successfully identified fire and smoke, achieving a mAP:50 of 92.6%, a precision score of 83.7%, and a recall of 95.2%. The results were compared with a YOLOv6 with large model, Faster-RCNN, and DEtection TRansformer. The obtained scores confirm the potential of the proposed models for wide application and promotion in the fire safety industry.

The Use of Lidar and Artificial Intelligence Algorithms for Detection and Size Estimation of Potholes

Road potholes have a well-known impact on driving quality and safety. Therefore, timely mitigation of potholes is critical for the safety of road users. However, efficient and timely maintenance relies on the presence of an effective process for pothole detection. Currently, transportation agencies primarily rely on manual inspection and road user reporting. These methods are subjective, prone to inaccuracy, and some are also laborious and time-consuming. An ideal pothole detection system would be accurate, objective, automated, and relatively inexpensive. In this context, accuracy encompasses three distinct performance areas: detection, localization, and size estimation. This study explores the potential of utilizing a mobile light detection and ranging (LiDAR) for accurate detection and size estimation, along with a global navigation satellite system (GNSS) receiver for localization, to develop an effective pothole surveillance system. To achieve this objective, the study proposes a four-step framework. Firstly, the LiDAR data are processed to generate ring-wise cross-sectional images. Secondly, a deep learning object detection network is trained to predict the presence and size of potholes. Thirdly, the ring-wise inferences are aggregated to produce a final decision. Lastly, the aggregated inferences are synchronized with GNSS locations to generate inspection maps. The system’s performance was validated using multiple road strips, never seen by the model, containing potholes of different sizes and shapes. The results demonstrated the effectiveness and accuracy of the proposed system. Overall, this research contributes to the research on LiDAR-based pothole inspection by proposing a novel four-step framework and incorporating it into an end-to-end pothole detection system, which can greatly improve the efficiency of pothole maintenance and enhance the safety of road users.

Smart Traffic Surveillance: Identifying Helmet-Less Riders and License Plate Recognition with YOLO and OCR Integration using Deep Learning

Currently, there are a number of issues with Indian traffic laws that can be resolved in a variety of ways. Riding a motorbike without a helmet is against the law and has increased the number of accidents in India. The current method mostly uses CCTV records to monitor traffic offences. When a rider is not wearing a helmet, traffic officers must zoom in on the licence plate and look into the frame where the infraction is occurring. However, because there are more and more traffic infractions and motorcycle ridership every day, this will take a lot of labour and time. What if there was a technology that would automatically search for traffic violations such as riding a motorbike without a helmet? hence, would automatically retrieve the number plate number of the bike. This study has been effectively completed by recent research using KNN, DNN-Classifier, etc. However, there are limitations to these works in terms of speed, accuracy, or efficiency in object identification and classification. The goal of this research project is to develop a non- helmet rider detection system that can automatically identify and retrieve a vehicle's number plate number when a driver fails to wear a helmet. Three-level Deep Learning for Object Detection is the key idea at play here. The items found are a human, a motorcycle at level one using YOLO, a helmet at level two using YOLO, Licence plate at the last level using YOLO. Next, OCR is used to obtain the number plate registration number. All of these methods—especially the section where the number plate number is extracted—are subject to predetermined limitations and conditions. Because video is the work's input, speed of execution is essential. We have developed a comprehensive algorithm for both number plate number extraction and helmet detection using the above-mentioned approaches.

Development of a flexible endoscopic robot with autonomous tracking control ability using machine vision and deep learning

Abstract. A flexible endoscopic robot is designed to solve the problem that it is difficult for auxiliary doctors to maintain a stable visual field in traditional endoscopic surgery. Based on geometric derivation, a motion control method under the constraint of the remote center motion (RCM) of the robot system is established, and a set of circular trajectories are planned for it. The RCM error of the robot during operation and the actual trajectory of the robot end in three-dimensional space are obtained through the motion capture system. The end of the robot is controlled by the heterogeneous primary–secondary teleoperation control algorithm based on position increments. Finally, the RTMDet deep learning object detection algorithm was selected to identify and locate surgical instruments through comparative experiments, and the autonomous tracking control was completed based on visual guidance. In the process of autonomous tracking, the RCM error was less than 1 mm, which met the actual surgical requirements.

Smart and Secure Vehicle Parking System to Avoid Theft Using Deep Image Recognition

Secure intelligent parking systems are complicated issues around the world. A secure parking system that is larceny-free is considered a luxury now. Parking is expensive and limited in almost all large cities. This research study presents a unique, safe, intelligent parking system based on deep image recognition techniques. To solve this tedious problem, an object detection deep learning model such as You Only Look Once (YOLO) is used. The proposed model is divided into three steps. The first step captures the car's image while entering the parking area. The second step is License plate recognition, in which the license plate is detected and saved the individual characters, and the recognized dataset is used when the driver leaves the parking area. The third step is facing detection and recognition of the taken plate. The fourth step is that the printer generates an invoice code used in case of a mismatch in the face of the driver. Evaluation of the proposed model indicates its efficiency. Accuracy of the following modules is achieved for vehicle detection (99%), number plate detection and recognition 98%, and 95%, respectively, and face detection and recognition 99%, and 97%, respectively. The proposed model's efficiency is shown through its evaluation. To get more security, a two-way screening procedure is used to keep the vehicles secure. So, the identification system's unique idea is to achieve better results by using two cameras and two-way authentication. It also forbids the entrance of those vehicles and drivers who are unauthorized or not allowed by the firm. Moreover, these modules were successfully implemented and evaluated using various performance measures, including precision, recall, and mean average precision.

Multidimensional Evaluation Methods for Deep Learning Models in Target Detection for SAR Images

As artificial intelligence technology advances, the application of object detection technology in the field of SAR (synthetic aperture radar) imagery is becoming increasingly widespread. However, it also faces challenges such as resource limitations in spaceborne environments and significant uncertainty in the intensity of interference in application scenarios. These factors make the performance evaluation of object detection key to ensuring the smooth execution of tasks. In the face of such complex and harsh application scenarios, methods that rely on single-dimensional evaluation to assess models have had their limitations highlighted. Therefore, this paper proposes a multi-dimensional evaluation method for deep learning models used in SAR image object detection. This method evaluates models in a multi-dimensional manner, covering the training, testing, and application stages of the model, and constructs a multi-dimensional evaluation index system. The training stage includes assessing training efficiency and the impact of training samples; the testing stage includes model performance evaluation, application-based evaluation, and task-based evaluation; and the application stage includes model operation evaluation and model deployment evaluation. The evaluations of these three stages constitute the key links in the performance evaluation of deep learning models. Furthermore, this paper proposes a multi-indicator comprehensive evaluation method based on entropy weight correlation scaling, which calculates the weights of each evaluation indicator through test data, thereby providing a balanced and comprehensive evaluation mechanism for model performance. In the experiments, we designed specific interferences for SAR images in the testing stage and tested three models from the YOLO series. Finally, we constructed a multi-dimensional performance profile diagram for deep learning object detection models, providing a new visualization method to comprehensively characterize model performance in complex application scenarios. This can provide more accurate and comprehensive model performance evaluation for remote sensing data processing, thereby guiding model selection and optimization. The evaluation method proposed in this study adopts a multi-dimensional perspective, comprehensively assessing the three core stages of a model’s lifecycle: training, testing, and application. This framework demonstrates significant versatility and adaptability, enabling it to transcend the boundaries of remote sensing technology and provide support for a wide range of model evaluation and optimization tasks.

Tongue feature dataset construction and real-time detection.

Tongue diagnosis in traditional Chinese medicine (TCM) provides clinically important, objective evidence from direct observation of specific features that assist with diagnosis. However, the current interpretation of tongue features requires a significant amount of manpower and time. TCM physicians may have different interpretations of features displayed by the same tongue. An automated interpretation system that interprets tongue features would expedite the interpretation process and yield more consistent results. This study applied deep learning visualization to tongue diagnosis. After collecting tongue images and corresponding interpretation reports by TCM physicians in a single teaching hospital, various tongue features such as fissures, tooth marks, and different types of coatings were annotated manually with rectangles. These annotated data and images were used to train a deep learning object detection model. Upon completion of training, the position of each tongue feature was dynamically marked. A large high-quality manually annotated tongue feature dataset was constructed and analyzed. A detection model was trained with average precision (AP) 47.67%, 58.94%, 71.25% and 59.78% for fissures, tooth marks, thick and yellow coatings, respectively. At over 40 frames per second on a NVIDIA GeForce GTX 1060, the model was capable of detecting tongue features from any viewpoint in real time. This study constructed a tongue feature dataset and trained a deep learning object detection model to locate tongue features in real time. The model provided interpretability and intuitiveness that are often lacking in general neural network models and implies good feasibility for clinical application.

Intra-class deep learning object detection on embedded computer system

<span lang="EN-US">Implementation of artificial intelligence tends to be portable, mobile and embeds in embedded computer system (EBD). EBD is a special-purpose computer with limited capacity in a small-form size. Deep learning (DL) had known as cutting edges for object recognition. With DL, object feature extraction analysis is omitted. DL requires large computing resources and capacity. Implement DL algorithm on EBD goal to achieves high detection accuracy and high-efficiency resources. Hence, be able to cope with intra-class variations, and image disturbances. By those challenges and limitations, this study reports the performance of EBD to recognize an object which has high variations in their class, through an optimal raw-input dataset. The raw-input dataset performed optimization process with a supervisor. Yield is the proper optimal input dataset in size. The performance results observed begin from training dataset until evaluation stage of DL. The comparison performs in efficiency resources, loss, validation-loss, timesteps, and detection accuracy by multiclass confusion matrix analysis. This study shows through this purpose method efficient resources are highly archived. Shorter timesteps ensure training stage is successful, and detection accuracy is perfectly archived. In addition, this study proves DL method archived great performances in classifying object that has identical structure.</span>

You only look once model-based object identification in computer vision

<span lang="EN-US">You only look once version 4 (YOLOv4) is a deep-learning object detection algorithm. It is used to decrease parameters and simplify network structures, making it suited for mobile and embedded device development. The YOLO detector can foresee an object's Class, bounding box, and probability of that Object's Class being found inside that bounding box. A probability value for each bounding box represents the likelihood of a given item class in that bounding box. Global features, channel attention, and special attention are also applied to extract more compelling information. Finally, the model combines the auxiliary and backbone networks to create the YOLOv4's entire network topology. Using custom functions developed upon YOLOv4, we get the count of the objects and a crop around the objects detected with a confidence score that specifies the probability of the thing seen being the same Class as predicted by YOLOv4. A confidence threshold is implemented to eliminate the detections with low confidence. </span>

Precast concrete project image dataset for deep learning object detection

Computer vision technology in precast concrete (PC) projects has the potential to enhance site management but faces challenges due to the lack of specialized datasets for this field. This study developed a PC components dataset for object detection (PCCODA) in four steps: component range selection, image collection, image preprocessing, and labeling. The performance of PCCODA was assessed using multiple algorithms, including You Only Look Once (YOLO), YOLO X, faster region-based convolutional neural networks (Faster R-CNN), and Double Head R-CNN. The dataset resulted in average accuracies of 0.88–0.97, average robustness of 0.84–0.96, and average detection speed of 3.64–23.3 frames per second. This performance fulfills the level required by most studies on construction automation. The applicability of the dataset was tested at a real site. This study contributes to the off-site construction project management theory and enhances productivity in PC projects by automating object detection-related tasks.

A Deep-Learning-Based Quality Control Evaluation Method for CT Phantom Images

Computed tomography (CT) is a rapid and precise medical imaging modality, but it poses the challenge of high radiation exposure to patients. To control this issue, stringent quality control (QC) evaluations are imperative for CT. One crucial aspect of CT QC involves the evaluation of phantom images, utilizing specifically designed phantoms for accuracy management and subsequent objective evaluation. However, CT QC has qualitative evaluation methods, particularly for evaluating spatial and contrast resolutions. To solve this problem, we propose a quality control method based on deep-learning object detection for quantitatively evaluating spatial and contrast resolutions, CT Attention You Only Look Once v8 (CTA-YOLOv8). First, we utilized the YOLOv8 network as the foundational model, optimizing it for enhanced accuracy. Second, we enhanced the network’s capabilities by integrating the Convolutional Block Attention Module (CBAM) and Swin Transformers, tailored for phantom image evaluations. The CBAM module was employed internally to pinpoint the optimal position for achieving peak performance in CT QC data. Similarly, we fine-tuned the code and patch size of the Swin Transformer module to align it with YOLOv8, culminating in the identification of the optimal configuration. Our proposed CTA-YOLOv8 network showed superior agreement with qualitative evaluation methods, achieving accuracies of 92.03% and 97.56% for spatial and contrast resolution evaluations, respectively. Thus, we suggest that our method offers nearly equivalent performance to qualitative methods. The utilization of the CTA-YOLOv8 network in evaluating CT phantom images holds potential for setting a new standard in quantitative assessment methodologies.

Sensor-based characterization of construction and demolition waste at high occupancy densities using synthetic training data and deep learning.

Sensor-based monitoring of construction and demolition waste (CDW) streams plays an important role in recycling (RC). Extracted knowledge about the composition of a material stream helps identifying RC paths, optimizing processing plants and form the basis for sorting. To enable economical use, it is necessary to ensure robust detection of individual objects even with high material throughput. Conventional algorithms struggle with resulting high occupancy densities and object overlap, making deep learning object detection methods more promising. In this study, different deep learning architectures for object detection (Region-based CNN/Region-based Convolutional Neural Network (Faster R-CNN), You only look once (YOLOv3), Single Shot MultiBox Detector (SSD)) are investigated with respect to their suitability for CDW characterization. A mixture of brick and sand-lime brick is considered as an exemplary waste stream. Particular attention is paid to detection performance with increasing occupancy density and particle overlap. A method for the generation of synthetic training images is presented, which avoids time-consuming manual labelling. By testing the models trained on synthetic data on real images, the success of the method is demonstrated. Requirements for synthetic training data composition, potential improvements and simplifications of different architecture approaches are discussed based on the characteristic of the detection task. In addition, the required inference time of the presented models is investigated to ensure their suitability for use under real-time conditions.

A deep learning object detection method to improve cluster analysis of two-dimensional data

A deep learning object detection method to improve cluster analysis of two-dimensional data

Automatic caries detection in bitewing radiographs—Part II: experimental comparison

ObjectiveThe objective of this study was to compare the detection of caries in bitewing radiographs by multiple dentists with an automatic method and to evaluate the detection performance in the absence of a reliable ground truth.Materials and methodsFour experts and three novices marked caries using bounding boxes in 100 bitewing radiographs. The same dataset was processed by an automatic object detection deep learning method. All annotators were compared in terms of the number of errors and intersection over union (IoU) using pairwise comparisons, with respect to the consensus standard, and with respect to the annotator of the training dataset of the automatic method.ResultsThe number of lesions marked by experts in 100 images varied between 241 and 425. Pairwise comparisons showed that the automatic method outperformed all dentists except the original annotator in the mean number of errors, while being among the best in terms of IoU. With respect to a consensus standard, the performance of the automatic method was best in terms of the number of errors and slightly below average in terms of IoU. Compared with the original annotator, the automatic method had the highest IoU and only one expert made fewer errors.ConclusionsThe automatic method consistently outperformed novices and performed as well as highly experienced dentists.Clinical significanceThe consensus in caries detection between experts is low. An automatic method based on deep learning can improve both the accuracy and repeatability of caries detection, providing a useful second opinion even for very experienced dentists.

Fusion of Time-of-Flight Based Sensors with Monocular Cameras for a Robotic Person Follower

Human-robot collaboration (HRC) is becoming increasingly important in advanced production systems, such as those used in industries and agriculture. This type of collaboration can contribute to productivity increase by reducing physical strain on humans, which can lead to reduced injuries and improved morale. One crucial aspect of HRC is the ability of the robot to follow a specific human operator safely. To address this challenge, a novel methodology is proposed that employs monocular vision and ultra-wideband (UWB) transceivers to determine the relative position of a human target with respect to the robot. UWB transceivers are capable of tracking humans with UWB transceivers but exhibit a significant angular error. To reduce this error, monocular cameras with Deep Learning object detection are used to detect humans. The reduction in angular error is achieved through sensor fusion, combining the outputs of both sensors using a histogram-based filter. This filter projects and intersects the measurements from both sources onto a 2D grid. By combining UWB and monocular vision, a remarkable 66.67% reduction in angular error compared to UWB localization alone is achieved. This approach demonstrates an average processing time of 0.0183s and an average localization error of 0.14 meters when tracking a person walking at an average speed of 0.21 m/s. This novel algorithm holds promise for enabling efficient and safe human-robot collaboration, providing a valuable contribution to the field of robotics.

Phytoplankton detection and recognition in freshwater digital microscopy images using deep learning object detectors

Water quality can be negatively affected by the presence of some toxic phytoplankton species, whose toxins are difficult to remove by conventional purification systems. This creates the need for periodic analyses, which are nowadays manually performed by experts. These labor-intensive processes are affected by subjectivity and expertise, causing unreliability. Some automatic systems have been proposed to address these limitations. However, most of them are based on classical image processing pipelines with not easily scalable designs. In this context, deep learning techniques are more adequate for the detection and recognition of phytoplankton specimens in multi-specimen microscopy images, as they integrate both tasks in a single end-to-end trainable module that is able to automatize the adaption to such a complex domain. In this work, we explore the use of two different object detectors: Faster R-CNN and RetinaNet, from the one-stage and two-stage paradigms respectively. We use a dataset composed of multi-specimen microscopy images captured using a systematic protocol. This allows the use of widely available optical microscopes, also avoiding manual adjustments on a per-specimen basis, which would require expert knowledge. We have made our dataset publicly available to improve the reproducibility and to foment the development of new alternatives in the field. The selected Faster R-CNN methodology reaches maximum recall levels of 95.35%, 84.69%, and 79.81%, and precisions of 94.68%, 89.30% and 82.61%, for W. naegeliana, A. spiroides, and D. sociale, respectively. The system is able to adapt to the dataset problems and improves the results overall with respect to the reference state-of-the-art work. In addition, the proposed system improves the automation and abstraction from the domain and simplifies the workflow and adjustment.

A Review of Machine Learning and Deep Learning for Object Detection, Semantic Segmentation, and Human Action Recognition in Machine and Robotic Vision

Machine vision, an interdisciplinary field that aims to replicate human visual perception in computers, has experienced rapid progress and significant contributions. This paper traces the origins of machine vision, from early image processing algorithms to its convergence with computer science, mathematics, and robotics, resulting in a distinct branch of artificial intelligence. The integration of machine learning techniques, particularly deep learning, has driven its growth and adoption in everyday devices. This study focuses on the objectives of computer vision systems: replicating human visual capabilities including recognition, comprehension, and interpretation. Notably, image classification, object detection, and image segmentation are crucial tasks requiring robust mathematical foundations. Despite the advancements, challenges persist, such as clarifying terminology related to artificial intelligence, machine learning, and deep learning. Precise definitions and interpretations are vital for establishing a solid research foundation. The evolution of machine vision reflects an ambitious journey to emulate human visual perception. Interdisciplinary collaboration and the integration of deep learning techniques have propelled remarkable advancements in emulating human behavior and perception. Through this research, the field of machine vision continues to shape the future of computer systems and artificial intelligence applications.

Pneumonia detection based on RSNA dataset and anchor-free deep learning detector.

Pneumonia is a highly lethal disease, and research on its treatment and early screening tools has received extensive attention from researchers. Due to the maturity and cost reduction of chest X-ray technology, and with the development of artificial intelligence technology, pneumonia identification based on deep learning and chest X-ray has attracted attention from all over the world. Although the feature extraction capability of deep learning is strong, existing deep learning object detection frameworks are based on pre-defined anchors, which require a lot of tuning and experience to guarantee their excellent results in the face of new applications or data. To avoid the influence of anchor settings in pneumonia detection, this paper proposes an anchor-free object detection framework and RSNA dataset based on pneumonia detection. First, a data enhancement scheme is used to preprocess the chest X-ray images; second, an anchor-free object detection framework is used for pneumonia detection, which contains a feature pyramid, two-branch detection head, and focal loss. The average precision of 51.5 obtained by Intersection over Union (IoU) calculation shows that the pneumonia detection results obtained in this paper can surpass the existing classical object detection framework, providing an idea for future research and exploration.