YOLO V3 Research Articles

Using computer vision, image analysis and UAVs for the automatic recognition and counting of common cranes (Grus grus)

Long-term monitoring of wildlife numbers traditionally uses observers, which are frequently inefficient and inaccurate due to their variable experience/training, are costly and difficult to sustain over time. Furthermore, there are other inhibiting factors for wildlife counting, such as: inhabiting inaccessible areas, fear of humans, and nocturnal behavior. There is a need to develop new technologies that will automatically identify and count wild animals in order to determine the appropriate management protocol. In this study, an advanced and accurate method for automatically calculating the number of cranes (Grus grus), using thermal cameras at night and visible light (RGB) cameras during the day onboard unmanned aerial vehicles (UAVs), based on image analysis and computer vision, was developed. The cranes congregate at night in a large communal roost, making it possible to count the birds while they are relatively static and all together. Each bird was counted individually by creating a standardized tool to determine population numbers for management, using image analysis and automatic processing. A dedicated algorithm was developed that aimed to identify the cranes based on their spectral characteristics (typical temperature, shape, size) and to effectively separate the cranes from the typical background. The automatic segmentation and counting of roosting common cranes using UAV nighttime thermal images had an Overall Accuracy (OA) of 91.47%, User's Accuracy (UA) of 99.68%, and Producer's Accuracy (PA) of 91.74%. The computer vision and machine learning algorithm based on the YOLO v3 platform of daytime RGB UAV images of common cranes at the feeding station yielded an overall loss accuracy level of 2.25%, with a mean square error of 1.87, OA of 94.51%, UA of 99.91%, PA of 94.59%. These results are highly encouraging, and although the algorithms were developed for the purpose of counting cranes, they could be adapted for other counting purposes for wildlife management.

Detection of Bottle Marine Debris Using Unmanned Aerial Vehicles and Machine Learning Techniques

Bottle marine debris (BMD) remains one of the most pressing global issues. This study proposes a detection method for BMD using unmanned aerial vehicles (UAV) and machine learning techniques to enhance the efficiency of marine debris studies. The UAVs were operated at three designed sites and at one testing site at twelve fly heights corresponding to 0.12 to 1.54 cm/pixel resolutions. The You Only Look Once version 2 (YOLO v2) object detection algorithm was trained to identify BMD. We added data augmentation and image processing of background removal to optimize BMD detection. The augmentation helped the mean intersection over the union in the training process reach 0.81. Background removal reduced processing time and noise, resulting in greater precision at the testing site. According to the results at all study sites, we found that approximately 0.5 cm/pixel resolution should be a considerable selection for aerial surveys on BMD. At 0.5 cm/pixel, the mean precision, recall rate, and F1-score are 0.94, 0.97, and 0.95, respectively, at the designed sites, and 0.61, 0.86, and 0.72, respectively, at the testing site. Our work contributes to beach debris surveys and optimizes detection, especially with the augmentation step in training data and background removal procedures.

Clipping-Based Post Training 8-Bit Quantization of Convolution Neural Networks for Object Detection

Fueled by the development of deep neural networks, breakthroughs have been achieved in plenty of computer vision problems, such as image classification, segmentation, and object detection. These models usually have handers and millions of parameters, which makes them both computational and memory expensive. Motivated by this, this paper proposes a post-training quantization method based on the clipping operation for neural network compression. By quantizing parameters of a model to 8-bit using our proposed methods, its memory consumption is reduced, its computational speed is increased, and its performance is maintained. This method exploits the clipping operation during training so that it saves a large computational cost during quantization. After training, this method quantizes the parameters to 8-bit based on the clipping value. In addition, a fully connected layer compression is conducted using singular value decomposition (SVD), and a novel loss function term is leveraged to further diminish the performance drop caused by quantization. The proposed method is validated on two widely used models, Yolo V3 and Faster R-CNN, for object detection on the PASCAL VOC, COCO, and ImageNet datasets. Performances show it effectively reduces the storage consumption at 18.84% and accelerates the model at 381%, meanwhile avoiding the performance drop (drop < 0.02% in VOC).

An End-to-End Automated License Plate Recognition System Using YOLO Based Vehicle and License Plate Detection with Vehicle Classification

An accurate and robust Automatic License Plate Recognition (ALPR) method proves surprising versatility in an Intelligent Transportation and Surveillance (ITS) system. However, most of the existing approaches often use prior knowledge or fixed pre-and-post processing rules and are thus limited by poor generalization in complex real-life conditions. In this paper, we leverage a YOLO-based end-to-end generic ALPR pipeline for vehicle detection (VD), license plate (LP) detection and recognition without exploiting prior knowledge or additional steps in inference. We assess the whole ALPR pipeline, starting from vehicle detection to the LP recognition stage, including a vehicle classifier for emergency vehicles and heavy trucks. We used YOLO v2 in the initial stage of the pipeline and remaining stages are based on the state-of-the-art YOLO v4 detector with various data augmentation and generation techniques to obtain LP recognition accuracy on par with current proposed methods. To evaluate our approach, we used five public datasets from different regions, and we achieved an average recognition accuracy of 90.3% while maintaining an acceptable frames per second (FPS) on a low-end GPU.

A deep learning method for X-ray image safety detection: YOLO-T

There are many problems in X-ray image dangerous goods target recognition with existing technology, such as low degree of automation, slow detection time, easy to misjudge under occlusion interference, etc. Based on the above problems, this paper proposes a multi-objective intelligent security inspection method for X-ray images based on the YOLO-T deep learning network. By adding the optimized Transformer structure to the YOLO architecture, this method can better solve the above problems. In order to better carry out the experiment, we proposed a set of X-ray safety detection data set GDXray-Expanded containing multiple categories of dangerous goods, and tested several versions of the deep learning network model of the YOLO series on this basis. Experiments show that the existing YOLO series algorithms still cannot solve the problem that dangerous goods in X-ray images are easy to be misjudged under occlusion interference. The YOLO-T method proposed in this paper solves this problem well, and in the big data set test, the maximum mAP can reach 97.73%, which is 7.66%, 16.47%, and 7.11% higher than the three methods of YOLO v2, YOLO v3, and YOLO v4 respectively, and has achieved the most competitive performance in the detection of seven categories of dangerous goods. To sum up, the YOLO-T network proposed in this paper mainly solves a series of problems in the field of dangerous goods target recognition and detection in X-ray security inspection images and has a high engineering application prospect in the field of X-ray security inspection.

Vehicle detection and masking in UAV images using YOLO to improve photogrammetric products

Abstract Photogrammetric products obtained by processing data acquired with Unmanned Aerial Vehicles (UAVs) are used in many fields. Various structures are analysed, including roads. Many roads located in cities are characterised by heavy traffic. This makes it impossible to avoid the presence of cars in aerial photographs. However, they are not an integral part of the landscape, so their presence in the generated photogrammetric products is unnecessary. The occurrence of cars in the images may also lead to errors such as irregularities in digital elevation models (DEMs) in roadway areas and the blurring effect on orthophotomaps. The research aimed to improve the quality of photogrammetric products obtained with the Structure from Motion algorithm. To fulfil this objective, the Yolo v3 algorithm was used to automatically detect cars in the images. Neural network learning was performed using data from a different flight to ensure that the obtained detector could also be used in independent projects. The photogrammetric process was then carried out in two scenarios: with and without masks. The obtained results show that the automatic masking of cars in images is fast and allows for a significant increase in the quality of photogrammetric products such as DEMs and orthophotomaps.

YOLO-T: Multitarget Intelligent Recognition Method for X-ray Images Based on the YOLO and Transformer Models

X-ray security inspection processes have a low degree of automation, long detection times, and are subject to misjudgment due to occlusion. To address these problems, this paper proposes a multi-objective intelligent recognition method for X-ray images based on the YOLO deep learning network and an optimized transformer structure (YOLO-T). We also construct the GDXray-Expanded X-ray detection dataset, which contains multiple types of dangerous goods. Using this dataset, we evaluated several versions of the YOLO deep learning network model and compared the results to those of the proposed YOLO-T model. The proposed YOLO-T method demonstrated higher accuracy for multitarget and hidden-target detection tasks. On the GDXray-Expanded dataset, the maximum mAP of the proposed YOLO-T model was 97.73%, which is 7.66%, 16.47%, and 7.11% higher than that obtained by the YOLO v2, YOLO v3, and YOLO v4 models, respectively. Thus, we believe that the proposed YOLO-T network has good application prospects in X-ray security inspection technologies. In all kinds of security detection scenarios using X-ray security detectors, the model proposed in this paper can quickly and accurately identify dangerous goods, which has broad application value.

Trichromatic ratiometric fluorescent sensor based on machine learning and smartphone for visual and portable monitoring of tetracycline antibiotics

In this work, a tri-color ratiometric fluorescent optical device integrated with machine learning-assisted smartphones is designed for the visual monitoring of tetracycline antibiotics. The trichromatic sensing probe consists of blue emitting carbon dots (BCDs) at 420 nm, and red-emitting bovine serum albumin-protected copper nanoclusters (BSA-Cu NCs) at 635 nm, as well as an emission peak at 520 nm, rises in the presence of tetracycline antibiotics (TCs), therefore, a three emission system is formed. BCDs/BSA-Cu NCs present three emission responses to TCs, which are based on the internal filtering effect (IFE) and the sensitization mechanisms with BSA. The reaction mechanism was verified by density functional theory (DFT) and a series of characterizations, such as infrared spectrum, ultraviolet absorption spectrum, and fluorescence lifetime. Interestingly, the fluorescence intensity of BCDs was quenched by TCs, and due to the sensitization between BSA and TCs, a green fluorescence rose at 520 nm, while the peak of BSA-Cu NCs remained stable, resulting in a significant color shift from red to cyan. Therefore, the ratiometric fluorescence intensity of I(635+520)/I420 can be used to determine TCs, and the colors change can be used to identify the quantitative analysis. In addition, the deep learning of the YOLO v3 algorithm assists the integrated tri-color portable optical device of smartphones, which can realize the visual detection of TCs. In general, the sensing system has many advantages, such as good stability, specificity, and rich color changes, which provide help for the visual detection of tetracycline residues.

Computer Vision Based Pothole Detection under Challenging Conditions.

Road discrepancies such as potholes and road cracks are often present in our day-to-day commuting and travel. The cost of damage repairs caused by potholes has always been a concern for owners of any type of vehicle. Thus, an early detection processes can contribute to the swift response of road maintenance services and the prevention of pothole related accidents. In this paper, automatic detection of potholes is performed using the computer vision model library, You Look Only Once version 3, also known as Yolo v3. Light and weather during driving naturally affect our ability to observe road damage. Such adverse conditions also negatively influence the performance of visual object detectors. The aim of this work was to examine the effect adverse conditions have on pothole detection. The basic design of this study is therefore composed of two main parts: (1) dataset creation and data processing, and (2) dataset experiments using Yolo v3. Additionally, Sparse R-CNN was incorporated into our experiments. For this purpose, a dataset consisting of subsets of images recorded under different light and weather was developed. To the best of our knowledge, there exists no detailed analysis of pothole detection performance under adverse conditions. Despite the existence of newer libraries, Yolo v3 is still a competitive architecture that provides good results with lower hardware requirements.

Real-Time Detection of Seedling Maize Weeds in Sustainable Agriculture

In recent years, automatic weed control has emerged as a promising alternative for reducing the amount of herbicide applied to the field, instead of conventional spraying. This method is beneficial to reduce environmental pollution and to achieve sustainable agricultural development. Achieving a rapid and accurate detection of weeds in maize seedling stage in natural environments is the key to ensuring maize yield and the development of automatic weeding machines. Based on the lightweight YOLO v4-tiny model, a maize weed detection model which combined an attention mechanism and a spatial pyramid pooling structure was proposed. To verify the effectiveness of the proposed method, five different deep-learning algorithms, including the Faster R-CNN, the SSD 300, the YOLO v3, the YOLO v3-tiny, and the YOLO v4-tiny, were compared to the proposed method. The comparative results showed that the mAP (Mean Average Precision) of maize seedlings and its associated weed detection using the proposed method was 86.69%; the detection speed was 57.33 f/s; and the model size was 34.08 MB. Furthermore, the detection performance of weeds under different weather conditions was discussed. The results indicated that the proposed method had strong robustness to the changes in weather, and it was feasible to apply the proposed method for the real-time and accurate detection of weeds.

A Framework to Create a Deep Learning Detector from a Small Dataset: A Case of Parawood Pith Estimation

A deep learning-based object detector has been successfully applied to all application areas. It has high immunity to variations in illumination and deviations among objects. One weakness of the detector is that it requires a huge, undefinable dataset for training the detector to avoid overtraining and make it deployable. This research proposes a framework to create a deep learning-based object detector with a limited-sized dataset. The framework is based on training the detector with the regions surrounding an object that typically contain various features over a more extensive area than the object itself. Our proposed algorithm further post-processes the detection results to identify the object. The framework is applied to the problem of wood pith approximation. The YOLO v3 framework was employed to create the detector with all default hyperparameters based on the transfer learning approach. A wood pith dataset with only 150 images is used to create the detector with a ratio between training to testing of 90:10. Several experiments were performed to compare the detection results from different approaches to preparing the regions surrounding a pith, i.e., all regions, only close neighbors, and only diagonal neighbors around a pith. The best experiment result shows that the framework outperforms the typical approach to create the detector with approximately twice the detection precision at a relative average error. Doi: 10.28991/ESJ-2023-07-01-017 Full Text: PDF

Automatic Detection and Counting System for Pavement Cracks Based on PCGAN and YOLO-MF

The regular detection of pavement cracks is critical for life and property security. However, existing deep learning-based methods of crack detection face difficulties in terms of data acquisition and defect counting. An automatic intelligent detection and tracking system for pavement cracks is proposed. Our system is formed of a pavement crack generative adversarial network (PCGAN) and a crack detection and tracking network called YOLO-MF. First, PCGAN is used to generate realistic crack images, to address the problem of the small number of available images. Next, YOLO-MF is developed based on an improved YOLO v3 modified by an acceleration algorithm and median flow (MF) algorithm to count the number of cracks. In a counting loop, our improved YOLO v3 detects cracks and the MF algorithm tracks the cracks detected in a video. This improved algorithm achieves the best accuracy of 98.47% and F1 score of 0.958 among other algorithms, and the precision-recall curve was close to the top right. A tiny model was developed and an acceleration algorithm was applied, which improved the detection speed by factors of five and six, respectively. In on-site measurement, three cracks were detected and tracked, and the total count was correct. Finally, the system was embedded in an intelligent device consisting of a calculating module, an automated unmanned aerial vehicle, and other components.

Road damage detection using UAV images based on multi-level attention mechanism

Pavement damage detection is essential for subsequent road maintenance decisions. However, recent detection networks have low accuracy and fail to detect most diseases on the road, which means that testing is very inefficient. Therefore, this study uses the unmanned aerial vehicle (UAV) road damage database and describes a multi-level attention mechanism called Multi-level Attention Block (MLAB) to strengthen the utilization of essential features by the You Only Look Once version 3 (YOLO v3). Adding MLAB between the backbone and feature fusion parts effectively increases the mAP value of the proposed network to 68.75%, while the accuracy of the original network is only 61.09%. The network is able to detect longitudinal cracks, transverse cracks, repairs, and potholes with high accuracy, and significantly improves the accuracy of alligator cracks and oblique cracks. The findings of this study will accelerate the application of non-destructive automatic road damage detection.

Masked Facial Recognition in Security Systems Using Transfer Learning.

The COVID-19 is a crisis of unprecedented magnitude, which has resulted in countless casualties and security troubles. In view of recent events of corona virus people are required to wear face masks to protect themselves from getting infected. As a result, a good portion of face (nose and mouth) is hidden by the mask and hence the facial recognition becomes difficult. Many organizations use facial recognition as a means of authentication. Researchers focus on developing rapid and efficient solutions to deal with the ongoing coronavirus pandemic by coming up with suggestions for handling the facial recognition problem. This research paper aims to identify the person, while the face is covered with a facial mask with only eyes and forehead being exposed. The first step involves marking the facial region. Next, using the data set, we will implement an object detection model YOLOv3 to identify unmasked and masked faces. The YOLO v3 object detection model is the best performing model with a detection time of 0.012 s, F1 score of 0.90 and mAP score of 0.92. Experimental results on Real-World Masked-Face-Data set show high recognition performance.

A Deep-Learning Extraction Method for Orchard Visual Navigation Lines

Orchard machinery autonomous navigation is helpful for improving the efficiency of fruit production and reducing labor costs. Path planning is one of the core technologies of autonomous navigation for orchard machinery. As normally planted in straight and parallel rows, fruit trees are natural landmarks that can provide suitable cues for orchard intelligent machinery. This paper presents a novel method to realize path planning based on computer vision technologies. We combine deep learning and the least-square (DL-LS) algorithm to carry out a new navigation line extraction algorithm for orchard scenarios. First, a large number of actual orchard images are collected and processed for training the YOLO V3 model. After the training, the mean average precision (MAP) of the model for trunk and tree detection can reach 92.11%. Secondly, the reference point coordinates of the fruit trees are calculated with the coordinates of the bounding box of trunks. Thirdly, the reference lines of fruit trees growing on both sides are fitted by the least-square method and the navigation line for the orchard machinery is determined by the two reference lines. Experimental results show that the trained YOLO V3 network can identify the tree trunk and the fruit tree accurately and that the new navigation line of fruit tree rows can be extracted effectively. The accuracy of orchard centerline extraction is 90.00%.

Automatic detection for the world's rarest primates based on a tropical rainforest environment

Hainan gibbons are the most endangered primates in the world and are at risk of extinction at any time. Infrared trigger camera technology is one of the most effective means at present, however, the data obtained are characterized by massive fragmentation, which is not conducive to post-processing and analysis. Therefore, a computer vision detection model for Hainan gibbons is proposed for the first time in this paper. Because of the foggy tropical rainforest, the image is pre-processed through an image filter whose parameters are obtained from a convolutional neural network. And then input it to the YOLO V3 network for Hainan gibbon detection. We produced a dataset containing 6000 images of Hainan gibbons for experimental training. The detection quality of Hainan gibbons based on YOLO V3 and image-filter YOLO (IF-YOLO) was evaluated. The results show that IF-YOLO maintains excellent performance on sunny days while improving the weakness of YOLO. Increasing the content of the dataset will extend the kinds of species detected by the model, which opens up new avenues for detecting others species in harsh conditions.

DETECTION AND LOCALIZATION OF RETINAL BREAKS IN ULTRAWIDEFIELD FUNDUS PHOTOGRAPHY USING a YOLO v3 ARCHITECTURE-BASED DEEP LEARNING MODEL.

We aimed to develop a deep learning model for detecting and localizing retinal breaks in ultrawidefield fundus (UWF) images. We retrospectively enrolled treatment-naive patients diagnosed with retinal break or rhegmatogenous retinal detachment and who had UWF images. The YOLO v3 architecture backbone was used to develop the model, using transfer learning. The performance of the model was evaluated using per-image classification and per-object detection. Overall, 4,505 UWF images from 940 patients were used in the current study. Among them, 306 UWF images from 84 patients were included in the test set. In per-object detection, the average precision for the object detection model considering every retinal break was 0.840. With the best threshold, the overall precision, recall, and F1 score were 0.6800, 0.9189, and 0.7816, respectively. In the per-image classification, the model showed an area under the receiver operating characteristic curve of 0.957 within the test set. The overall accuracy, sensitivity, and specificity in the test data set were 0.9085, 0.8966, and 0.9158, respectively. The UWF image-based deep learning model evaluated in the current study performed well in diagnosing and locating retinal breaks.

Human Detection Algorithm Based on Improved YOLO v4

The human behavior datasets have the characteristics of complex background, diverse poses, partial occlusion, and diverse sizes. Firstly, this paper adopts YOLO v3 and YOLO v4 algorithms to detect human objects in videos, and qualitatively analyzes and compares detection performance of two algorithms on UTI, UCF101, HMDB51 and CASIA datasets. Then, this paper proposed an improved YOLO v4 algorithm since the vanilla YOLO v4 has incomplete human detection in specific video frames. Specifically, the improved YOLO v4 introduces the Ghost module in the CBM module to further reduce the number of parameters. Lateral connection is added in the CSP module to improve the feature representation capability of the network. Furthermore, we also substitute MaxPool with SoftPool in the primary SPP module, which not only avoids the feature loss, but also provides a regularization effect for the network, thus improving the generalization ability of the network. Finally, this paper qualitatively compares the detection effects of the improved YOLO v4 and primary YOLO v4 algorithm on specific datasets. The experimental results show that the improved YOLO v4 can solve the problem of complex targets in human detection tasks effectively, and further improve the detection speed.

Ground Target Detection and Damage Assessment by Patrol Missiles Based on YOLO-VGGNet

Patrol missiles are a common type of unmanned aerial vehicle, which can be efficiently used for reconnaissance and sensing. In this work, target detection and the damage assessment of typical mobile ground targets by patrol missiles are studied, and a new method, combining the YOLO v3 with the VGG networks, is proposed for the problem. Specifically, with YOLO v3 as the precursor, the proposed method can detect, classify, and localize ground targets accurately and quickly. Then, the image blocks of detected targets are fed into the lightweight VGG networks, which can evaluate their damage level coarsely. Meanwhile, based on class activation mapping (CAM) and deconvolution, we further analyse the activation intensity of clustered convolution kernels, which helps determine whether the targets’ key components are destroyed. Unlike traditional image change detection methods, which require images before and after a strike for comparison, the proposed method learns the target model through extensive training and can assess the target’s damage status in a timely and online manner. Compared to previous learning-based methods, our detailed analysis with convolutional feature visualization of the damaged targets and their components gives a more interpretable perspective. Finally, Unity simulation experiments prove the proposed method’s effectiveness, which improves the accuracy of damage level assessment by 16.0% and 8.8% compared with traditional image-change-detection-based methods and the two-CNN learning-based method. The convolutional feature clustering method evaluates the status of the targets’ key components with an accuracy of 72%.

Deep Learning Approach for the Detection of Noise Type in Ancient Images

Recent innovations in digital image capturing techniques facilitate the capture of stationary and moving objects. The images can be easily captured via high-end digital cameras, mobile phones and other handheld devices. Most of the time, captured images vary compared to actual objects. The captured images may be contaminated by dark, grey shades and undesirable black spots. There are various reasons for contamination, such as atmospheric conditions, limitations of capturing device and human errors. There are various mechanisms to process the image, which can clean up contaminated image to match with the original one. The image processing applications primarily require detection of accurate noise type which is used as input for image restoration. There are filtering techniques, fractional differential gradient and machine learning techniques to detect and identify the type of noise. These methods primarily rely on image content and spatial domain information of a given image. With the advancements in the technologies, deep learning (DL) is a technology that can be trained to mimic human intelligence to recognize various image patterns, audio files and text for accuracy. A deep learning framework empowers correct processing of multiple images for object identification and quick decision abilities without human interventions. Here Convolution Neural Network (CNN) model has been implemented to detect and identify types of noise in the given image. Over the multiple internal iterations to optimize the results, the identified noise is classified with 99.25% accuracy using the Proposed System Architecture (PSA) compared with AlexNet, Yolo V5, Yolo V3, RCNN and CNN. The proposed model in this study proved to be suitable for the classification of mural images on the basis of every performance parameter. The precision, accuracy, f1-score and recall of the PSA are 98.50%, 99.25%, 98.50% and 98.50%, respectively. This study contributes to the development of mural art recovery.