You Only Look Once Version 3 Research Articles

A Deep Learning-Based Approach for Real-Time Object Detection and Recognition

Object detection and recognition is an essential task in computer vision with numerous real-world applications such as surveillance, self-driving cars, and robotics. In recent years, deep learning-based approaches have significantly improved the accuracy and speed of object detection and recognition. The You Only Look Once version 3 (YOLOv3) algorithm is a popular deep learning-based approach that can detect and recognize objects in real-time. The Common Objects in Context (COCO) dataset is a large-scale dataset with over 330,000 labeled images and more than 2.5 million object instances, making it a popular choice for object detection and recognition tasks. In this paper, we propose a deep learning-based approach for real-time object detection and recognition using the YOLOv3 architecture and COCO dataset. We evaluate our approach based on several performance metrics, including mean average precision (mAP), frames per second (FPS), total object detection time, object detection accuracy, false positive rate, number of detected objects, and mean intersection over union (mIoU). Our results show that our approach achieves a mean average precision of 0.76 on the COCO dataset and a real-time performance of 40 frames per second on a single GPU. Additionally, our approach achieves an object detection accuracy of 93.5%, a false positive rate of 6.5%, and a mean intersection over union of 0.65. Our proposed approach shows promising results for real-time object detection and recognition and can be applied to various real-world applications.

Scale-Adaptive KCF Mixed with Deep Feature for Pedestrian Tracking

Pedestrian tracking is an important research content in the field of computer vision. Tracking is achieved by predicting the position of a specific pedestrian in each frame of a video. Pedestrian tracking methods include neural network-based methods and traditional template matching-based methods, such as the SiamRPN (Siamese region proposal network), the DASiamRPN (distractor-aware SiamRPN), and the KCF (kernel correlation filter). The KCF algorithm has no scale-adaptive capability and cannot effectively solve the occlusion problem, and because of many defects of the HOG (histogram of oriented gradient) feature that the KCF uses, the tracking target is easy to lose. For those defects of the KCF algorithm, an improved KCF model, the SKCFMDF (scale-adaptive KCF mixed with deep feature) algorithm was designed. By introducing deep features extracted by a newly designed neural network and by introducing the YOLOv3 (you only look once version 3) object detection algorithm, which was also improved for more accurate detection, the model was able to achieve scale adaptation and to effectively solve the problem of occlusion and defects of the HOG feature. Compared with the original KCF, the success rate of pedestrian tracking under complex conditions was increased by 36%. Compared with the mainstream SiamRPN and DASiamRPN models, it was still able to achieve a small improvement.

A computer-aided diagnostic system for mammograms based on YOLOv3

Due to a large amount of noise in medical images, the task of detecting and classifying the lesions of mammograms remains a huge challenge. Based on the existing deep learning methods, focusing on the diversity of breast cancer lesion types, this paper proposes a computer-aided diagnosis system based on YOLOv3 (You Only Look Once version 3) convolutional neural network for mammograms. In this system, we integrate detection and multi-classification problems of breast lesions into a regression problem, thereby simultaneously accomplish the two tasks in one framework. The proposed computer-aided diagnosis system is mainly divided into three components: preprocessing part of the original mammograms, deep convolutional neural network based on YOLOv3, processing and evaluation of the network output. We use the dataset from CBIS-DDSM to train three models: general model, mass model and microcalcification model. These trained models can detect the position of the input mammograms in different situations, and then classify them into mass, microcalcification, benign, malignant, and other categories. After evaluating the performance by using test set images, the accuracy rates of the general model, mass model, and microcalcification model trained by our system reach 93.667 %, 97.767 %, 96.870 % in the detection task, and 93.927 %, 98.121 %, 97.045 % in the classification task. The computer-aided diagnosis system performs well in lesion detection and classification tasks with high-noise mammograms, reflecting well robustness.

Ship Detection and Tracking in Inland Waterways Using Improved YOLOv3 and Deep SORT

Ship detection and tracking is an important task in video surveillance in inland waterways. However, ships in inland navigation are faced with accidents such as collisions. For collision avoidance, we should strengthen the monitoring of navigation and the robustness of the entire system. Hence, this paper presents ship detection and tracking of ships using the improved You Only Look Once version 3 (YOLOv3) detection algorithm and Deep Simple Online and Real-time Tracking (Deep SORT) tracking algorithm. Three improvements are made to the YOLOv3 target detection algorithm. Firstly, the Kmeans clustering algorithm is used to optimize the initial value of the anchor frame to make it more suitable for ship application scenarios. Secondly, the output classifier is modified to a single Softmax classifier to suit our ship dataset which has three ship categories and mutual exclusion. Finally, Soft Non-Maximum Suppression (Soft-NMS) is introduced to solve the deficiencies of the Non-Maximum Suppression (NMS) algorithm when screening candidate frames. Results showed the mean Average Precision (mAP) and Frame Per Second (FPS) of the improved algorithm are increased by about 5% and 2, respectively, compared with the existing YOLOv3 detecting Algorithm. Then the improved YOLOv3 is applied in Deep Sort and the performance result of Deep Sort showed that, it has greater performance in complex scenes, and is robust to interference such as occlusion and camera movement, compared to state of art algorithms such as KCF, MIL, MOSSE, TLD, and Median Flow. With this improvement, it will help in the safety of inland navigation and protection from collisions and accidents.

Application of conventional UAV-based high-throughput object detection to the early diagnosis of pine wilt disease by deep learning

The early diagnosis of pine wilt disease (PWD) is crucial to its management. Substantial effort has been made to develop an accurate early diagnosis method. However, none of the existing methods are suitable for large-scale rapid screening in the field. In this study, an unmanned aerial vehicle (UAV) was used to collect a large number of images from a pine tree canopy in an early stage of infection for the generation of a training dataset. Owing to the inability to develop object detection models using nine regular machine learning classifiers, two advanced deep learning algorithms were employed, namely the Faster Region-based Convolutional Network (Faster R-CNN) and You Only Look Once version 3 (YOLOv3). Model performances were compared based on the precision (mAP), size, and processing speed. All four models possessed similar precision (0.602–0.64), but the YOLO-based models had a smaller size and faster processing speed than the Faster R-CNN-adapted models. The population of infected trees (in the early or late stage of infection) was predicted under different treatments (retaining or removing the dead trees from the forest) using these four models to explore their application. The results indicate that retaining dead trees after chopping them down results in fewer dead trees but more early infected trees the following year. We propose a cost-effective and high-throughput method for the early diagnosis of PWD in the field using UAV-based image processing and object detection (uses YOLOv3) based on deep learning.

Fire Detection Method Based on Depthwise Separable Convolution and YOLOv3

Recently, video-based fire detection technology has become an important research topic in the field of machine vision. This paper proposes a method of combining the classification model and target detection model in deep learning for fire detection. Firstly, the depthwise separable convolution is used to classify fire images, which saves a lot of detection time under the premise of ensuring detection accuracy. Secondly, You Only Look Once version 3 (YOLOv3) target regression function is used to output the fire position information for the images whose classification result is fire, which avoids the problem that the accuracy of detection cannot be guaranteed by using YOLOv3 for target classification and position regression. At the same time, the detection time of target regression for images without fire is greatly reduced saved. The experiments were tested using a network public database. The detection accuracy reached 98% and the detection rate reached 38 fps. This method not only saves the workload of manually extracting flame characteristics, reduces the calculation cost, and reduces the amount of parameters, but also improves the detection accuracy and detection rate.

Deep-learning-based nanowire detection in AFM images for automated nanomanipulation

Atomic force microscope (AFM)-based nanomanipulation has been proved to be a possible method for assembling various nanoparticles into complex patterns and devices. To achieve efficient and fully automated nanomanipulation, nanoparticles on the substrate must be identified precisely and automatically. This work focuses on an autodetection method for flexible nanowires using a deep learning technique. An instance segmentation network based on You Only Look Once version 3 (YOLOv3) and a fully convolutional network (FCN) is applied to segment all movable nanowires in AFM images. Combined with follow-up image morphology and fitting algorithms, this enables detection of postures and positions of nanowires at a high abstraction level. Benefitting from these algorithms, our program is able to automatically detect nanowires of different morphologies with nanometer resolution and has over 90% reliability in the testing dataset. The detection results are less affected by image complexity than the results of existing methods and demonstrate the good robustness of this algorithm.

Improved YOLOv3 with duplex FPN for object detection based on deep learning

YOLOv3 is a deep learning-based real-time object detector and is mainly used in applications such as video surveillance and autonomous vehicles. In this paper, we proposed an improved YOLOv3 (You Only Look Once version 3) applied Duplex FPN, which enhanced large object detection by utilizing low-level feature information. The conventional YOLOv3 improved the small object detection performance by applying FPN (Feature Pyramid Networks) structure to YOLOv2. However, YOLOv3 with an FPN structure specialized in detecting small objects, so it is difficult to detect large objects. Therefore, this paper proposed an improved YOLOv3 applied Duplex FPN, which can utilize low-level location information in high-level feature maps instead of the existing FPN structure of YOLOv3. This improved the detection accuracy of large objects. Also, an extra detection layer was added to the top-level feature map to prevent failure of detection of parts of large objects. Further, dimension clusters of each detection layer were reassigned to learn quickly how to accurately detect objects. The proposed method was compared and analyzed in the PASCAL VOC dataset. The experimental results showed that the bounding box accuracy of large objects improved owing to the Duplex FPN and extra detection layer, and the proposed method succeeded in detecting large objects that the existing YOLOv3 did not.

Multi-Scale Ship Detection From SAR and Optical Imagery Via A More Accurate YOLOv3

Deep learning detection methods use in ship detection remains a challenge, owing to the small scale of the objects and interference from complex sea surfaces. In addition, existing ship detection methods rarely verify the robustness of their algorithms on multisensor images. Thus, we propose a new improvement on the “you only look once” version 3 (YOLOv3) framework for ship detection in marine surveillance, based on synthetic aperture radar (SAR) and optical imagery. First, improved choices are obtained for the anchor boxes by using linear scaling based on the k-means++ algorithm. This addresses the difficulty in reflecting the advantages of YOLOv3's multiscale detection, as the anchor boxes of a single detection target type between different detection scales have small differences. Second, we add uncertainty estimators for the positioning of the bounding boxes by introducing a Gaussian parameter for ship detection into the YOLOv3 framework. Finally, four anchor boxes are allocated to each detection scale in the Gaussian-YOLO layer instead of three as in the default YOLOv3 settings, as there are wide disparities in an object's size and direction in remote sensing images with different resolutions. Applying the proposed strategy to "YOLOv3-spp” and "YOLOv3-tiny,” the results are enhanced by 2%-3%. Compared with other models, the improved-YOLOv3 has the highest average precision on both the optical (93.56%) and SAR (95.52%) datasets. The improved-YOLOv3 is robust, even in the context of a mixed dataset of SAR and optical images comprising images from different satellites and with different scales.

Non-destructive thermal imaging for object detection via advanced deep learning for robotic inspection and harvesting of chili peppers

Deep Learning has been utilized in computer vision for object detection for almost a decade. Real-time object detection for robotic inspection and harvesting has gained interest during this time as a possible technique for high-quality machine assistance during agriculture applications.We utilize RGB and thermal images of chili peppers in an environment of various amounts of debris, pepper overlapping, and ambient lighting, train this dataset, and compare object detection methods. Results are presented from the real-time and less than real-time object detection models. Two advanced deep learning algorithms, Mask-Regional Convolutional Neural Networks (Mask-RCNN) and You Only Look Once version 3 (YOLOv3)are compared in terms of object detection accuracy and computational costs. When utilizing the YOLOv3 architecture, an overall training mean average precision (mAP) value of 1.0 is achieved. Most testing images from this model score within a range from 97 to 100% confidence levels in natural environment. It is shown that the YOLOv3 algorithm has superior capabilities to the Mask-RCNN with over 10 times the computational speed on the chili dataset. However, some of the RGB test images resulted in low classification scores when heavy debris is present in the image. A significant improvement in the real-time classification scores was observed when the thermal images were used, especially with heavy debris present. We found and report improved prediction scores with a thermal imagery dataset where YOLOv3 struggled on the RGB images. It was shown that mapping temperature differences between the pepper and plant/debris can provide significant features for object detection in real-time and can help improve accuracy of predictions with heavy debris, variant ambient lighting, and overlapping of peppers. In addition, successful thermal imaging for real-time robotic harvesting could allow the harvesting period to become more efficient and open up harvesting opportunity in low light situations.

Evaluation of Deep Learning YOLOv3 Algorithm for Object Detection and Classification

You Only Look Once version 3 (YOLOv3) is a deep learning model for object detection and classification. It is a single neural network architecture model that uses features from the feeding images and predicts bounding box for all classes of image simultaneously. This paper descript an experimental work for train the deep learning model based on YOLOv3 architecture implemented using Tensor Flow as a deep learning framework. The training process had been done using the data-set PASCAL VOC 2007 and data-set PASCAL VOC 2012 and using The Adaptive Moment Estimation Optimizer (ADM optimizer). The trained model is then tested by using the VOC 2007 test data-set. The final results evaluate the YOLOv3 deep learning model performance for object detection and classification.

A Novel SAR Image Ship Small Targets Detection Method

To satisfy practical requirements of high real-time accuracy and low computational complexity of synthetic aperture radar (SAR) image ship small target detection, this paper proposes a small ship target detection method based on the improved You Only Look Once Version 3 (YOLOv3). The main contributions of this study are threefold. First, the feature extraction network of the original YOLOV3 algorithm is replaced with the VGG16 network convolution layer. Second, general convolution is transformed into depthwise separable convolution, thereby reducing the computational cost of the algorithm. Third, a residual network structure is introduced into the feature extraction network to reuse the shallow target feature information, which enhances the detailed features of the target and ensures the improvement in accuracy of small target detection performance. To evaluate the performance of the proposed method, many experiments are conducted on public SAR image datasets. For ship targets with complex backgrounds and small ship targets in the SAR image, the effectiveness of the proposed algorithm is verified. Results show that the accuracy and recall rate improved by 5.31% and 2.77%, respectively, compared with the original YOLOV3. Furthermore, the proposed model not only significantly reduces the computational effort, but also improves the detection accuracy of ship small target.

YOLOv3-Based Matching Approach for Roof Region Detection from Drone Images

Due to the large data volume, the UAV image stitching and matching suffers from high computational cost. The traditional feature extraction algorithms—such as Scale-Invariant Feature Transform (SIFT), Speeded Up Robust Features (SURF), and Oriented FAST Rotated BRIEF (ORB)—require heavy computation to extract and describe features in high-resolution UAV images. To overcome this issue, You Only Look Once version 3 (YOLOv3) combined with the traditional feature point matching algorithms is utilized to extract descriptive features from the drone dataset of residential areas for roof detection. Unlike the traditional feature extraction algorithms, YOLOv3 performs the feature extraction solely on the proposed candidate regions instead of the entire image, thus the complexity of the image matching is reduced significantly. Then, all the extracted features are fed into Structural Similarity Index Measure (SSIM) to identify the corresponding roof region pair between consecutive image sequences. In addition, the candidate corresponding roof pair by our architecture serves as the coarse matching region pair and limits the search range of features matching to only the detected roof region. This further improves the feature matching consistency and reduces the chances of wrong feature matching. Analytical results show that the proposed method is 13× faster than the traditional image matching methods with comparable performance.

Vehicle Pedestrian Detection Method Based on Spatial Pyramid Pooling and Attention Mechanism

Object detection for vehicles and pedestrians is extremely difficult to achieve in autopilot applications for the Internet of vehicles, and it is a task that requires the ability to locate and identify smaller targets even in complex environments. This paper proposes a single-stage object detection network (YOLOv3-promote) for the detection of vehicles and pedestrians in complex environments in cities, which improves on the traditional You Only Look Once version 3 (YOLOv3). First, spatial pyramid pooling is used to fuse local and global features in an image to better enrich the expression ability of the feature map and to more effectively detect targets with large size differences in the image; second, an attention mechanism is added to the feature map to weight each channel, thereby enhancing key features and removing redundant features, which allows for strengthening the ability of the feature network to discriminate between target objects and backgrounds; lastly, the anchor box derived from the K-means clustering algorithm is fitted to the final prediction box to complete the positioning and identification of target vehicles and pedestrians. The experimental results show that the proposed method achieved 91.4 mAP (mean average precision), 83.2 F1 score, and 43.7 frames per second (FPS) on the KITTI (Karlsruhe Institute of Technology and Toyota Technological Institute) dataset, and the detection performance was superior to the conventional YOLOv3 algorithm in terms of both accuracy and speed.

Detecting and Measuring Defects in Wafer Die Using GAN and YOLOv3

This research used deep learning methods to develop a set of algorithms to detect die particle defects. Generative adversarial network (GAN) generated natural and realistic images, which improved the ability of you only look once version 3 (YOLOv3) to detect die defects. Then defects were measured based on the bounding boxes predicted by YOLOv3, which potentially provided the criteria for die quality sorting. The pseudo defective images generated by GAN from the real defective images were used as the training image set. The results obtained after training with the combination of the real and pseudo defective images were 7.33% higher in testing average precision (AP) and more accurate by one decimal place in testing coordinate error than after training with the real images alone. The GAN can enhance the diversity of defects, which improves the versatility of YOLOv3 somewhat. In summary, the method of combining GAN and YOLOv3 employed in this study creates a feature-free algorithm that does not require a massive collection of defective samples and does not require additional annotation of pseudo defects. The proposed method is feasible and advantageous for cases that deal with various kinds of die patterns.

CentroidNetV2: A hybrid deep neural network for small-object segmentation and counting

This paper presents CentroidNetV2, a novel hybrid Convolutional Neural Network (CNN) that has been specifically designed to segment and count many small and connected object instances. This complete redesign of the original CentroidNet uses a CNN backbone to regress a field of centroid-voting vectors and border-voting vectors. The segmentation masks of the individual object instances are produced by decoding centroid votes and border votes. A loss function that combines cross-entropy loss and Euclidean-distance loss achieves high quality centroids and borders of object instances. Several backbones and loss functions are tested on three different datasets ranging from precision agriculture to microbiology and pathology. CentroidNetV2 is compared to the state-of-the art networks You Only Look Once Version 3 (YOLOv3) and Mask Recurrent Convolutional Neural Network (MRCNN). On two out of three datasets CentroidNetV2 achieves the highest F1 score and on all three datasets CentroidNetV2 achieves the highest recall. CentroidNetV2 demonstrates the best ability to detect small objects although the best segmentation masks for larger objects are produced by MRCNN.

Plant Counting of Cotton from UAS Imagery Using Deep Learning-Based Object Detection Framework

Assessing plant population of cotton is important to make replanting decisions in low plant density areas, prone to yielding penalties. Since the measurement of plant population in the field is labor intensive and subject to error, in this study, a new approach of image-based plant counting is proposed, using unmanned aircraft systems (UAS; DJI Mavic 2 Pro, Shenzhen, China) data. The previously developed image-based techniques required a priori information of geometry or statistical characteristics of plant canopy features, while also limiting the versatility of the methods in variable field conditions. In this regard, a deep learning-based plant counting algorithm was proposed to reduce the number of input variables, and to remove requirements for acquiring geometric or statistical information. The object detection model named You Only Look Once version 3 (YOLOv3) and photogrammetry were utilized to separate, locate, and count cotton plants in the seedling stage. The proposed algorithm was tested with four different UAS datasets, containing variability in plant size, overall illumination, and background brightness. Root mean square error (RMSE) and R2 values of the optimal plant count results ranged from 0.50 to 0.60 plants per linear meter of row (number of plants within 1 m distance along the planting row direction) and 0.96 to 0.97, respectively. The object detection algorithm, trained with variable plant size, ground wetness, and lighting conditions generally resulted in a lower detection error, unless an observable difference of developmental stages of cotton existed. The proposed plant counting algorithm performed well with 0–14 plants per linear meter of row, when cotton plants are generally separable in the seedling stage. This study is expected to provide an automated methodology for in situ evaluation of plant emergence using UAS data.

Convolutional neural-based algorithm for port occupancy status detection of optical distribution frames

As optical distribution frames (ODFs) carry increasing amounts of data, issues relating to their port management are becoming crucial. Image backhaul inspection has been widely accepted as an effective technology to monitor the occupancy status of ODF ports. However, the massive amount of on-site images requires a large team of off-site technologists to manually identify the occupancy status of each ODF port, which is time-consuming. We employ the you only look once version 3 (YOLOv3) network to automatically recognize ODF port occupancy. The YOLOv3 is a state-of-the-art convolutional neural network that has been shown to be very efficient for object detection in terms of processing speed and accuracy for the common objects in context. To accommodate ODF images with densely arranged small objects, high resolutions, closely spaced adjacent ports, and occlusion, we modified the original YOLOv3 with four-scale feature fusion, anchor box dimension clustering, and soft nonmaximum suppression filtering. Experiments showed a 7.38% increase in the original YOLOv3 detection accuracy rate of 91.45%. The new method can update the image backhaul inspection to automatically realize port resource management. The number of required port management technologists is considerably reduced, and the accuracy of port resources is increased, resulting in significant network investment savings.

Automobile Fine-Grained Detection Algorithm Based on Multi-Improved YOLOv3 in Smart Streetlights

Upgrading ordinary streetlights to smart streetlights to help monitor traffic flow is a low-cost and pragmatic option for cities. Fine-grained classification of vehicles in the sight of smart streetlights is essential for intelligent transportation and smart cities. In order to improve the classification accuracy of distant cars, we propose a reformed YOLOv3 (You Only Look Once, version 3) algorithm to realize the detection of various types of automobiles, such as SUVs, sedans, taxis, commercial vehicles, small commercial vehicles, vans, buses, trucks and pickup trucks. Based on the dataset UA-DETRAC-LITE, manually labeled data is added to improve the data balance. First, data optimization for the vehicle target is performed to improve the generalization ability and position regression loss function of the model. The experimental results show that, within the range of 67 m, and through scale optimization (i.e., by introducing multi-scale training and anchor clustering), the classification accuracies of trucks and pickup trucks are raised by 26.98% and 16.54%, respectively, and the overall accuracy is increased by 8%. Secondly, label smoothing and mixup optimization is also performed to improve the generalization ability of the model. Compared with the original YOLO algorithm, the accuracy of the proposed algorithm is improved by 16.01%. By combining the optimization of the position regression loss function of GIOU (Generalized Intersection Over Union), the overall system accuracy can reach 92.7%, which improves the performance by 21.28% compared with the original YOLOv3 algorithm.

Target Recognition of Infrared Ship Based on Deep Learning

In this study,the You Only Look Once Version 3(YOLOv3)target recognition algorithm in deep learning technology is used to identify the ship in an infrared image collected using an infrared imager from the sea surface.The infrared imager captures images at a frequency of up to 50 frames per second.To reduce network computing time,a few ideas are generated based on YOLOv3;additionally,a full convolution structure and the LeakReLU activation function are used to redesign a lightweight basic network to accelerate detection.The output layer uses the softmax algorithm to regress according to the characteristics of the collected infrared images,which improves the detection speed and accounts for detection accuracy.