YOLOv3-tiny Network Research Articles

Autonomous plume Near-Source search assisted by intermittent visible plume information using finite state Machine and YOLOv3-tiny

Since a plume’s visual characteristics have not been fully considered and utilized, the traditional plume near-source search methods lose a large amount of potentially available information, resulting in the low efficiency and reliability of near-source search. This paper proposes a novel plume near-source search approach assisted by the intermittent visible plume information using finite state machine and YOLOv3-tiny for a robot. A finite state machine is used to realize the autonomous behavioral decision-making of the robot according to the obtained visible plume information, olfactory information, and lidar information. In addition, the YOLOv3-tiny network is used to detect the intermittent visible plume information obtained by a depth camera to capture the color and texture of the plume. In order to improve the search reliability of the robot in the olfactory tracking behavior, a whale optimization algorithm is utilized to enable the robot to simulate the hunting behavior and the social mechanism of the whale group and conduct olfactory tracking of the plume. The experimental results show that this proposed plume near-source search approach achieves the success rates of 91.67% and 83.33% in windless and windy environments respectively, and the average total running time is 113.8 s and 159.6 s in the experimental environment with obstacles, respectively.

FPGA-Based Vehicle Detection and Tracking Accelerator.

A convolutional neural network-based multiobject detection and tracking algorithm can be applied to vehicle detection and traffic flow statistics, thus enabling smart transportation. Aiming at the problems of the high computational complexity of multiobject detection and tracking algorithms, a large number of model parameters, and difficulty in achieving high throughput with a low power consumption in edge devices, we design and implement a low-power, low-latency, high-precision, and configurable vehicle detector based on a field programmable gate array (FPGA) with YOLOv3 (You-Only-Look-Once-version3), YOLOv3-tiny CNNs (Convolutional Neural Networks), and the Deepsort algorithm. First, we use a dynamic threshold structured pruning method based on a scaling factor to significantly compress the detection model size on the premise that the accuracy does not decrease. Second, a dynamic 16-bit fixed-point quantization algorithm is used to quantify the network parameters to reduce the memory occupation of the network model. Furthermore, we generate a reidentification (RE-ID) dataset from the UA-DETRAC dataset and train the appearance feature extraction network on the Deepsort algorithm to improve the vehicles' tracking performance. Finally, we implement hardware optimization techniques such as memory interlayer multiplexing, parameter rearrangement, ping-pong buffering, multichannel transfer, pipelining, Im2col+GEMM, and Winograd algorithms to improve resource utilization and computational efficiency. The experimental results demonstrate that the compressed YOLOv3 and YOLOv3-tiny network models decrease in size by 85.7% and 98.2%, respectively. The dual-module parallel acceleration meets the demand of the 6-way parallel video stream vehicle detection with the peak throughput at 168.72 fps.

Automatic detection of multi-type defects on potatoes using multispectral imaging combined with a deep learning model

Automatic detection of potato multi-type defects remains a challenge because of the diversification in defect size and visual similarity among multi-type defects. In this study, an accurate and fast detection method based on a multispectral (MS) image combined with an improved YOLOv3-tiny model was developed to automatically detect and classify multi-type defects on potatoes. A multispectral imaging system (MSI), covering 25 wavebands with a spatial resolution of 409 × 216 pixels, was used to collect MS images of 428 potato samples, which consisted of defect-free potatoes and defective potatoes (including five types of defects, i.e. germination, common scab, bug-eye, dry-rot, and bruise). By introducing the Res2Net modules into the YOLO v3-tiny network, a detection model called multi-type defects detection network (MDDNet) was developed for detecting multi-type defects on potatoes. Three deep learning models, YOLO-v5x, YOLOv3-tiny, and DY2TNet models were compared with the MDDNet model on 128 testing potato MS images. Experimental results showed that the proposed model achieved the highest mean average precision of 90.26% for potato defects among the four models, with about 75 ms detecting time for each MS image. This research demonstrated that the MDDNet model combined with MSI can be useful for the detection of potato multi-type defects.

A 40.96-GOPS 196.8-mW Digital Logic Accelerator Used in DNN for Underwater Object Recognition

This investigation presents a digital logic accelerator (DLA) design of a neural network hardware that utilizes output reuse. The DLA is used in the detection mechanism of underwater objects that was deployed in an underwater vehicle. A modified YoloV3-tiny network was also implemented to detect more than 20 underwater objects. The proposed DLA uses processing units that have parallel architectures of output windows, and output channels. Moreover, a new Inter-Controller is designed to control the direct memory access (DMA) together with a new Reshape module to improve the performance and power efficiency. A detailed description of the design as well as the measurements on silicon are presented. The chip is realized using a typical 180-nm CMOS process. It showed a performance result of 40.96 GOPS and the power consumption is 196.8 mW. The DLA was tested to demonstrate 19.88 frames per second and 40.96 GOPS.

Partition harvesting of a column-comb litchi harvester based on 3D clustering

Accurate recognition of litchi fruits in orchard environments and acquisition of their coordinate position information are key for realizing successful harvesting using litchi harvesters. However, the existing detection methods are often aimed at large and relatively sparse fruits and thus are inappropriate for small and densely distributed litchi fruits. Therefore, at present, litchi fruit are typically manually harvested, resulting in a low harvesting efficiency that cannot meet the needs of growers. To improve the efficiency of litchi harvesting, this study proposes a column-comb litchi harvesting method based on K-means 3D clustering partitioning, which includes four main steps: (1) Litchi image acquisition and labeling methods are developed. (2) An improved version of the present YOLOv3-tiny network model structure is developed named the YOLOv3-tiny-Litchi network model, and the litchi fruit detection results of five kinds of neural networks, namely, YOLOv3-tiny, YOLOv3-tiny-Litchi, YOLOv4, YOLOv5x and Faster R-CNN, are compared. (3) A depth camera is used to obtain the 3D coordinates of litchi fruits, and the K-means clustering algorithm is used to divide the litchi harvesting area to obtain the optimal partitioning results. (4) Field experiments on litchi harvesting are reported. The experimental results show that the improved YOLOv3-tiny-Litchi model can recognize litchi fruits more accurately; the recall rate is 78.99%, the precision rate is 87.43%, and the F1 score is 0.83. The results of 3D clustering partitioning show that when K is equal to 6, the optimal harvesting rate is 90.03%, which satisfies the theoretical requirements of litchi harvesters. The field experiments show that when the theoretical number of partitions is 6, the average harvesting rate of the litchi harvester is 91.15% and the recognition rate is 88.39%, and the harvesting efficiency of mechanical partition harvesting is 1.4 times that of mechanical harvesting without partitioning and 2 times that of manual harvesting, essentially meeting the operating requirements. These research results can serve as a reference for the mechanization of litchi harvesting.

A real-time railway fastener inspection method using the lightweight depth estimation network

Fasteners are critical components of railways that maintain the rail tracks in a fixed position. Their failure can lead to serious accidents such as train derailments, so their condition needs to be inspected periodically. Conventional image-based inspection methods fail to take full advantage of the structural features of fasteners, making them less robust in real-world environments. This paper presents a new approach for real-time fastener inspection by (1) extracting fastener regions using the YOLOv3-tiny network (2) proposing and pruning a lightweight and encoder–decoder network architecture for inferring depth information from a single RGB image of fasteners (3) fusing the RGB-D features for inspection. Compared with the image-based SVM, the F1 of RGB-D fusion-based SVM increases from 94.34% to 95.83%, illustrating the improvement of additional depth information for fastener defect inspection. The inspection system runs at 11.9 FPS, which enables real-time inspection of railway fasteners.

Diseases Detection of Occlusion and Overlapping Tomato Leaves Based on Deep Learning.

Background: In view of the existence of light shadow, branches occlusion, and leaves overlapping conditions in the real natural environment, problems such as slow detection speed, low detection accuracy, high missed detection rate, and poor robustness in plant diseases and pests detection technology arise.Results: Based on YOLOv3-tiny network architecture, to reduce layer-by-layer loss of information during network transmission, and to learn from the idea of inverse-residual block, this study proposes a YOLOv3-tiny-IRB algorithm to optimize its feature extraction network, improve the gradient disappearance phenomenon during network deepening, avoid feature information loss, and realize network multilayer feature multiplexing and fusion. The network is trained by the methods of expanding datasets and multiscale strategies to obtain the optimal weight model.Conclusion: The experimental results show that when the method is tested on the self-built tomato diseases and pests dataset, and while ensuring the detection speed (206 frame rate per second), the mean Average precision (mAP) under three conditions: (a) deep separation, (b) debris occlusion, and (c) leaves overlapping are 98.3, 92.1, and 90.2%, respectively. Compared with the current mainstream object detection methods, the proposed method improves the detection accuracy of tomato diseases and pests under conditions of occlusion and overlapping in real natural environment.

TempDiff: Feature Map-Level CNN Sparsity Enhancement at Near-Zero Memory Overhead via Temporal Difference

The demand for distributed vision systems at an unceasingly larger scale requires the availability of sensor nodes able to execute high-level visual tasks (e.g., object detection for visual monitoring). Such tasks routinely entail the high computational cost of Convolutional Neural Networks (CNN), which conflicts with the tight power budget and memory capacity of sensor nodes at the edge. This paper introduces an approach to reduce the computational cost of object detection in CNN accelerators available in edge devices. The proposed approach induces additional feature map-level sparsity (i.e., computation skipping) at inference time by exploiting temporal correlation among frames. The proposed approach has an uncommonly favorable computation-memory tradeoff, as significant computation reduction is achieved at the cost of very small additional memory for the storage of intermediate features. As further benefits, no architectural changes or retraining are required, allowing immediate deployment in existing vision frameworks and suppressing the need for storing multiple models. Results show that the proposed TempDiff method achieves up to 37% computation reduction with 1.1% accuracy drop based on the SSD(VGG16) object detection network, under both VIRAT and ImageNet-VID datasets. Similarly, 18.3% (35.8%) computation reduction at 3.3% (3.2%) memory overhead, and 3.8% (6.8%) accuracy drop in YOLOv1 (VGG16) (SSD (VGG16)) is achieved under the CAMEL dataset. Furthermore up to 58% computation reduction with 2% accuracy drop and 3.7% memory overhead were achieved for YOLOv3-Tiny network under the ImageNet-VID dataset.

An Automated Recognition of Work Activity in Industrial Manufacturing Using Convolutional Neural Networks

The automated assessment and analysis of employee activity in a manufacturing enterprise, operating in accordance with the concept of Industry 4.0, is essential for a quick and precise diagnosis of work quality, especially in the process of training a new employee. In the case of industrial solutions, many approaches involving the recognition and detection of work activity are based on Convolutional Neural Networks (CNNs). Despite the wide use of CNNs, it is difficult to find solutions supporting the automated checking of work activities performed by trained employees. We propose a novel framework for the automatic generation of workplace instructions and real-time recognition of worker activities. The proposed method integrates CNN, CNN Support Vector Machine (SVM), CNN Region-Based CNN (Yolov3 Tiny) for recognizing and checking the completed work tasks. First, video recordings of the work process are analyzed and reference video frames corresponding to work activity stages are determined. Next, work-related features and objects are determined using CNN with SVM (achieving 94% accuracy) and Yolov3 Tiny network based on the characteristics of the reference frames. Additionally, matching matrix between the reference frames and the test frames using mean absolute error (MAE) as a measure of errors between paired observations was built. Finally, the practical usefulness of the proposed approach by applying the method for supporting the automatic training of new employees and checking the correctness of their work done on solid fuel boiler equipment in a manufacturing company was demonstrated. The developed information system can be integrated with other Industry 4.0 technologies introduced within an enterprise.

Fast and Accurate Recognition for Codes on Complex Backgrounds for Real-Life Industrial Applications

In the food and beverage industry, the existing recognition of code characters on the surface of complex packaging usually suffers from low accuracy and low speed. This work presents an efficient and accurate inkjet code recognition system based on the combination of the deep learning and traditional image processing methods. The proposed system mainly consists of three sequential modules, i.e., the characters region extraction by modified YOLOv3-tiny network, the character processing by the traditional image processing methods such as binarization and the modified character projection segmentation, and the character recognition by a Convolutional recurrent neural network (CRNN) model based on a modified version of MobileNetV3. In this system, only a small amount of tag data has been made and an effective character data generator is designed to randomly generate different experimental data for the CRNN model training. To the best of our knowledge, this report for the first time describes that deep learning has been applied to the recognition of codes on complex background for the real-life industrial application. Experimental results have been provided to verify the accuracy and effectiveness of the proposed model, demonstrating a recognition accuracy of 0.986 and a processing speed of 100 ms per bottle in the end-to-end character recognition system.

An improved YOLOv3‐tiny algorithm for vehicle detection in natural scenes

YOLO (You Only Look Once), as a target detection algorithm with good speed and precision, is widely used in the industry. In the process of driving, the vehicle image captured by the driving camera is detected and it extracts the license plate and the front part of the vehicle. Compared with the network structure of YOLOv3-tiny algorithm, the acquisition method of anchor box is improved by combining the Birch algorithm. In order to improve the real-time performance, the original two-scale detection is added to the multi-scale prediction of three-scale detection to ensure its accuracy. Finally, the experimental results show that the improved YOLOv3-tiny network structure proposed in this study can improve the performance of mean-average-precision, intersection over union and speed by 5.99%, 17.52% and 48.4%, respectively, and the algorithm has certain robustness.

Pedestrian Motion Path Detection Method Based on Deep Learning and Foreground Detection

For the surveillance video images captured by monocular camera, this paper proposes a method combining foreground detection and deep learning to detect moving pedestrians, making full use of the invariable background of video image. Firstly, the motion region is extracted by the method of interframe difference and background difference. Then, the normalized motion region extracts the feature vectors based on the improved YOLOv3 tiny network. Finally, the trained linear support vector machine is used for pedestrian detection, and the performance of the fusion detection algorithm on caviar dataset is given, which proves the effectiveness of the proposed fusion detection algorithm. Experimental results show that the proposed method not only improves the practical application of pedestrian rerecognition but also reduces the detection range, computational complexity, and false detection rate compared with sliding window method.

An optimized YOLO‐based object detection model for crop harvesting system

The adoption of automated crop harvesting system based on machine vision may improve productivity and optimize the operational cost. The scope of this study is to obtain visual information at the plantation which is crucial in developing an intelligent automated crop harvesting system. This paper aims to develop an automatic detection system with high accuracy performance, low computational cost and lightweight model. Considering the advantages of YOLOv3 tiny, an optimized YOLOv3 tiny network namely YOLO-P is proposed to detect and localize three objects at palm oil plantation which include fresh fruit bunch, grabber and palm tree under various environment conditions. The proposed YOLO-P model incorporated lightweight backbone based on densely connected neural network, multi-scale detection architecture and optimized anchor box size. The experimental results demonstrated that the proposed YOLO-P model achieved good mean average precision and F1 score of 98.68% and 0.97 respectively. Besides, the proposed model performed faster training process and generated lightweight model of 76 MB. The proposed model was also tested to identify fresh fruit bunch of various maturities with accuracy of 98.91%. The comprehensive experimental results show that the proposed YOLO-P model can effectively perform robust and accurate detection at the palm oil plantation.

Enhanced YOLO v3 Tiny Network for Real-Time Ship Detection From Visual Image

Different from ship detection from synthetic aperture radar (SDSAR) and ship detection from spaceborne optical images (SDSOI), ship detection from visual image (SDVI) has better detection accuracy and real-time performance, which can be widely used in port management, cross-border ship detection, autonomous ship, safe navigation, and other real-time applications. In this paper, we proposed a new SDVI algorithm, named enhanced YOLO v3 tiny network for real-time ship detection. The algorithm can be used in video surveillance to realize the accurate classification and positioning of six types of ships (including ore carrier, bulk cargo carrier, general cargo ship, container ship, fishing boat, and passenger ship) in real-time. Based on the original YOLO v3 tiny network, we have made the following fine tunings. 1) The preset anchors trained on Seaship annotation data have the similar “dumpy” shape as the normal ships, helping the network to achieve faster and better training; 2) Convolution layer instead of max-pooling layer and expanding the channels of prediction network improve the small target detection ability of the algorithm. 3) Due to the problem that large-scale ships are easily disturbed by the onshore building, complex waves and light on the water surface, we introduced attention module named CBAM into the backbone network, which make the model more focused on the target. The detection accuracy of the proposed algorism is obviously better than that of the original YOLO v3 tiny work. Although it is slightly inferior to the Yolo v3 network, it has faster speed than Yolo v3. However, the proposed algorithm is a better trade-off between real-time performance and detection accuracy, and is more suitable for actual scenes. Compared with the SOAT algorithm in Z. Shao et al. (2020), our algorithm has a 9.6% improvement in mAP and a faster speed.

LittleYOLO-SPP: A delicate real-time vehicle detection algorithm

Vehicle detection in real-time is a challenging and important task. The existing real-time vehicle detection lacks accuracy and speed. Real-time systems must detect and locate vehicles during criminal activities like theft of vehicle and road traffic violations with high accuracy. Detection of vehicles in complex scenes with occlusion is also extremely difficult. In this study, a lightweight model of deep neural network LittleYOLO-SPP based on the YOLOv3-tiny network is proposed to detect vehicles effectively in real-time. The YOLOv3-tiny object detection network is improved by modifying its feature extraction network to increase the speed and accuracy of vehicle detection. The proposed network incorporated Spatial pyramid pooling into the network, which consists of different scales of pooling layers for concatenation of features to enhance network learning capability. The Mean square error (MSE) and Generalized IoU (GIoU) loss function for bounding box regression is used to increase the performance of the network. The network training includes vehicle-based classes from PASCAL VOC 2007, 2012 and MS COCO 2014 datasets such as car, bus, and truck. LittleYOLO-SPP network detects the vehicle in real-time with high accuracy regardless of video frame and weather conditions. The improved network achieves a higher mAP of 77.44 % on PASCAL VOC and 52.95 % mAP on MS COCO datasets.

MYOLOv3-Tiny: A new convolutional neural network architecture for real-time detection of track fasteners

Real-time detection of fasteners is of great significance for improving the inspection efficiency of railway tracks. However, the task is also challenging due to the limited memory and processing capacity of the embedded maintenance systems. To address these challenges, in this paper, we propose a new detection network architecture called MYOLOv3-Tiny, in which the depthwise and pointwise convolution are used to reduce the computational complexity of the network, and the linear bottleneck structure with inverted residual is employed to design the backbone network for efficiently extracting the fasteners’ features. The MYOLOv3-Tiny is trained and tested on the high-resolution track fastener data collected from the Ring Test Field of National Railway Test Center of China. The extensive experiments show the proposed MYOLOv3-Tiny achieves higher detection precision, lower computational complexity and memory consumption, and faster detection speed compared to state-of-the-art methods. Specifically, its detection precision reaches 99.32% after introducing visually coherent image mixup enhancement technology, and the memory consumption is reduced by 43% compared to the YOLOv3-Tiny network with a high detection speed guarantee. Hence, the proposed MYOLOv3-Tiny has the potential to lay a foundation for the real-time detection of railway track fasteners and to be applied to large-scale fastener detection devices.

EM-Sign: A Non-Contact Recognition Method Based on 24 GHz Doppler Radar for Continuous Signs and Dialogues

We studied continuous sign language recognition using Doppler radar sensors. Four signs in Chinese sign language and American sign language were captured and extracted by complex empirical mode decomposition (CEMD) to obtain spectrograms. Image sharpening was used to enhance the micro-Doppler signatures of the signs. To classify the different signs, we utilized an improved Yolov3-tiny network by replacing the framework with ResNet and fine-tuned the network in advance. This method can remove the epentheses from the training process. Experimental results revealed that the proposed method can surpass the state-of-the-art sign language recognition methods in continuous sign recognition with a precision of 0.924, a recall of 0.993, an F1-measure of 0.957 and a mean average precision (mAP) of 0.99. In addition, dialogue recognition in three daily conversation scenarios was performed and evaluated. The average word error rate (WER) was 0.235, 10% lower than in of other works. Our work provides an alternative form of sign language recognition and a new approach to simplify the training process and achieve a better continuous sign language recognition effect.

Research of status recognition of Fiber transfer box based on machine vision and deep learning

Fiber transfer box is a box containing fiber ports which transfer optical signals for telecommunication networks. It’s hard to check and record the status of hundreds of ports using man power. This paper proposes an intelligent recognition and localization system for fiber transfer box. Pictures of ports are treated and classified with Support Vector Machine (SVM) and Deep learning algorithm. It can identify the status of all ports in the fiber transfer box and recognize the writing letter for each layer. First, the image is converted to binary one by threshold of the Cr channel. A SVM classifier is used to identify the Red-Hat ports by HOG features. Second, an adaptive chessboard segmentation algorithm is designed for segmentation of all ports and character area. In order to improve the identification accuracy, an eleven-layer Convolution Neural Network (CNN) is trained and used to further identify the ports which have not been classified correctly by SVM. Letters are extracted and positioned by using YOLOv3-tiny network. Experiments show that the method achieves great accuracy and efficiency for a variety of scenes on mobile devices.

Search Like an Eagle: A Cascaded Model for Insulator Missing Faults Detection in Aerial Images

Insulator missing fault is a serious accident of high-voltage transmission lines, which can cause abnormal energy supply. Recently, a lot of vision-based methods are proposed for detecting an insulator missing fault in aerial images. However, these methods usually lack efficiency and robustness due to the effect of the complex background interferences in the aerial images. More importantly, most of these methods cannot address the insulator multi-fault detection. This paper proposes an unprecedented cascaded model to detect insulator multi-fault in the aerial images to solve the existing challenges. Firstly, a total of 764 images are adopted to create a novel insulator missing faults dataset ‘IMF-detection’. Secondly, a new network is proposed to locate the insulator string from the complex background. Then, the located region that contains the insulator string is set to be an RoI (region of interest) region. Finally, the YOLO-v3 tiny network is trained and then used to detect the insulator missing faults in the RoI region. Experimental results and analysis validate that the proposed method is more efficient and robust than some previous works. Most importantly, the average running time of the proposed method is about 30ms, which demonstrates that it has the potential to be adopted for the on-line detection of insulator missing faults.

An attribution-based pruning method for real-time mango detection with YOLO network

Real-time fruit detection and localization in orchards are essential for agronomic applications of yield estimation, yield mapping, and automated harvesting. Traditional detection methods based on hand-crafted feature extractors are difficult to adapt to the complicated variations in real orchard environments. Modern deep neural networks (DNNs) need high performance computing units for inference, this is not practical for typical farms and orchards, despite the high detection performance. To reduce the computation cost of DNNs, we propose a generalized attribution method for pruning detection networks which can be easily finetuned to accurately detect mango in real time. By designing the channel and spatial masks to generalize the attribution method, the convolutional kernels that are firmly correlated with specific target output in the original YOLOv3-tiny network can be detected. Then, the uncorrelated kernels are pruned in channel-dimension layer-by-layer. Before finetuning the pruned network, anchor sizes, data augmentation, and learning rate decay were adapted for mango detection. The experimental results show that the proposed pruning method could identify the highly target-related convolutional kernels and that the finetuned network provides better mango detection performance than the original. Our resulting network which is a scale and rotation invariant mango detection network achieved an F1-score of 0.944 with 2.6 GFLOPs (giga-floating point operations). Compared to the finetuned network without pruning, the computation of our network was reduced by 68.7% whereas the accuracy was increased by 0.4%. Compared to a state-of-the-art network trained with the same mango dataset, the computation was reduced by 83.4% with only about 2.4% loss in accuracy. The proposed pruning method can strip a sub-network from a large-scale detection network to meet the real-time requirements of low-power-consumption processors for mobile devices, e.g., ARM Cortex-A8 performs around 4.0 GFLOPS (giga-floating point operations per second). The trained network and test code are available for comparative studies.