Lightweight Network Model Research Articles

Real-Time Recognition and Localization Based on Improved YOLOv5s for Robot's Picking Clustered Fruits of Chilies.

Chili recognition is one of the critical technologies for robots to pick chilies. The robots need locate the fruit. Furthermore, chilies are always planted intensively and their fruits are always clustered. It is a challenge to recognize and locate the chilies that are blocked by branches and leaves, or other chilies. However, little is known about the recognition algorithms considering this situation. Failure to solve this problem will mean that the robot cannot accurately locate and collect chilies, which may even damage the picking robot's mechanical arm and end effector. Additionally, most of the existing ground target recognition algorithms are relatively complex, and there are many problems, such as numerous parameters and calculations. Many of the existing models have high requirements for hardware and poor portability. It is very difficult to perform these algorithms if the picking robots have limited computing and battery power. In view of these practical issues, we propose a target recognition-location scheme GNPD-YOLOv5s based on improved YOLOv5s in order to automatically identify the occluded and non-occluded chilies. Firstly, the lightweight optimization for Ghost module is introduced into our scheme. Secondly, pruning and distilling the model is designed to further reduce the number of parameters. Finally, the experimental data show that compared with the YOLOv5s model, the floating point operation number of the GNPD-YOLOv5s scheme is reduced by 40.9%, the model size is reduced by 46.6%, and the reasoning speed is accelerated from 29 ms/frame to 14 ms/frame. At the same time, the Mean Accuracy Precision (MAP) is reduced by 1.3%. Our model implements a lightweight network model and target recognition in the dense environment at a small cost. In our locating experiments, the maximum depth locating chili error is 1.84 mm, which meets the needs of a chili picking robot for chili recognition.

An Algorithm for Real-Time Aluminum Profile Surface Defects Detection Based on Lightweight Network Structure

Surface defects, which often occur during the production of aluminum profiles, can directly affect the quality of aluminum profiles, and should be monitored in real time. This paper proposes an effective, lightweight detection method for aluminum profiles to realize real-time surface defect detection with ensured detection accuracy. Based on the YOLOv5s framework, a lightweight network model is designed by adding the attention mechanism and depth-separable convolution for the detection of aluminum. The lightweight network model improves the limitations of the YOLOv5s framework regarding to its detection accuracy and detection speed. The backbone network GCANet is built based on the Ghost module, in which the Attention mechanism module is embedded in the AC3Ghost module. A compression of the backbone network is achieved, and more channel information is focused on. The model size is further reduced by compressing the Neck network using a deep separable convolution. The experimental results show that, compared to YOLOv5s, the proposed method improves the mAP by 1.76%, reduces the model size by 52.08%, and increases the detection speed by a factor of two. Furthermore, the detection speed can reach 17.4 FPS on Nvidia Jeston Nano’s edge test, which achieves real-time detection. It also provides the possibility of embedding devices for real-time industrial inspection.

Component Detection of Overhead Transmission Line Based on CBAM-Efficient-YOLOv5

In view of the low efficiency of the staff who need to detect the insulator, grading ring, spacer and other components with their naked eyes when inspecting the transmission line manually, a CBAM Efficient YOLOv5 algorithm is proposed to realize the automatic detection of the above components. The algorithm improves the lightweight of YOLOv5, uses EfficientNet lightweight network to replace the original feature extraction network, and uses CBAM (Convolutional Block Attention Module) with both channel and spatial attention to replace SE (Sequence and Exception) attention module. Finally, a lightweight network model is obtained, in which the model size is reduced by 45.8%, the average precision is 95.2%, and the detection precision of all targets is improved to more than 90%. On the premise of meeting the precision requirements, it is easier to deploy in embedded equipment such as unmanned aerial vehicle, so as to improve the work efficiency of line patrol.

A Lightweight Network Model Based on an Attention Mechanism for Ship-Radiated Noise Classification

Recently, deep learning has been widely used in ship-radiated noise classification. To improve classification efficiency, avoiding high computational costs is an important research direction in ship-radiated noise classification. We propose a lightweight squeeze and excitation residual network 10 (LW-SEResNet10). In ablation experiments of LW-SEResNet10, the use of ResNet10 instead of ResNet18 reduced 56.1% of parameters, while the accuracy is equivalent to ResNet18. The improved accuracy indicates that the ReLU6 enhanced the model stability, and an attention mechanism captured the channel dependence. The ReLU6 activation function does not introduce additional parameters, and the number of parameters introduced by the attention mechanism accounts for 0.2‰ of the model parameters. The 3D dynamic MFCC feature performs better than MFCC, Mel-spectrogram, 3D dynamic Mel-spectrogram, and CQT. Moreover, the LW-SEResNet10 model is also compared with ResNet and two classic lightweight models. The experimental results show that the proposed model achieves higher classification accuracy and is lightweight in terms of not only the model parameters, but also the time consumption. LW-SEResNet10 also outperforms the state-of-the-art model CRNN-9 by 3.1% and ResNet by 3.4% and has the same accuracy as AudioSet pretrained STM, which achieves the trade-off between accuracy and model efficiency.

Monitoring MBE Substrate Deoxidation via RHEED Image-Sequence Analysis by Deep Learning

Reflection high-energy electron diffraction (RHEED) is a powerful tool in molecular beam epitaxy (MBE), but RHEED images are often difficult to interpret, requiring experienced operators. We present an approach for automated surveillance of GaAs substrate deoxidation in MBE reactors using deep-learning-based RHEED image-sequence classification. Our approach consists of an nonsupervised autoencoder (AE) for feature extraction, combined with a supervised convolutional classifier network. We demonstrate that our lightweight network model can accurately identify the exact deoxidation moment. Furthermore, we show that the approach is very robust and allows accurate deoxidation detection for months without requiring retraining. The main advantage of the approach is that it can be applied to raw RHEED images without requiring further information such as the rotation angle, temperature, etc.

Pixel Difference Unmixing Feature Networks for Edge Detection

The edge detection model based on deep learning significantly improves performance, but its generally high model complexity requires a large pretrained Convolutional Neural Networks (CNNs) backbone, and hence large memory and computing power. To solve this problem, we carefully choose proper components for edge detection, introduce a Multiscale Aware Fusion Module based on self-attention and a feature-unmixing loss function, and propose a lightweight network model, Pixel Difference Unmixing Feature Networks (PDUF). The backbone network of proposed model is designed to adopt skip long-short residual connection and does not use pre-trained weights, and requires straightforward hyper-parameter settings. Extensive experiments on the BSDS, NYUD, and Multi-cue datasets, we found that the proposed model has higher F-scores than current state-of-the-art lightweight models (those with fewer than 1 million parameters) on BSDS500 (ODS F-score of 0.818), NYUDv2 depth datasets (ODS F-score of 0.767) and Multi-Cue dataset (ODS F-score 0.871(0.002)), with similar performance compared with some large models (with about 35 million parameters).

Multiple Activation Functions and Data Augmentation-Based Lightweight Network for In Situ Tool Condition Monitoring

Intelligent manufacturing raises higher requirements for tool condition monitoring (TCM) in terms of accuracy, robustness, and adaptability. At present, direct methods based on image processing and deep learning have made breakthroughs in TCM. However, some issues, such as image quality, model parameters, and dataset scale in the abovementioned methods, restrict industrial applications of TCM. Regarding the abovementioned issue, the purpose of this article is to propose a lightweight network model based on multiple activation functions to promote the intelligent industrial application of TCM. First, the image quality mechanism caused by complex working conditions is analyzed in industrial environments. Correspondingly, data augmentation is adopted to solve the problem of data scale under the premise of ensuring data quality and richness. Then, the adaptive activation function and the hard version of swish are introduced at the front and second half of the network to avoid information loss and reduce the activation function cost. Finally, a lightweight network based on cloud-edge collaboration for TCM is constructed. The model is iteratively optimized in the cloud and inferenced on the edge embedded device. The accuracy and adaptability of the proposed network are verified by accelerating milling cutter life under multiple working conditions.

Traffic Sign Detection Based on Lightweight Multiscale Feature Fusion Network

Traffic sign detection is a research hotspot in advanced assisted driving systems, given the complex background, light transformation, and scale changes of traffic sign targets, as well as the problems of slow result acquisition and low accuracy of existing detection methods. To solve the above problems, this paper proposes a traffic sign detection method based on a lightweight multiscale feature fusion network. Since a lightweight network model is simple and has fewer parameters, it can greatly improve the detection speed of a target. To learn more target features and improve the generalization ability of the model, a multiscale feature fusion method can be used to improve recognition accuracy during training. Firstly, MobileNetV3 was selected as the backbone network, a new spatial attention mechanism was introduced, and a spatial attention branch and a channel attention branch were constructed to obtain a mixed attention weight map. Secondly, a feature-interleaving module was constructed to convert the single-scale feature map of the specified layer into a multiscale feature fusion map to realize the combined encoding of high-level semantic information and low-level semantic information. Then, a feature extraction base network for lightweight multiscale feature fusion with an attention mechanism based on the above steps was constructed. Finally, a key-point detection network was constructed to output the location information, bias information, and category probability of the center points of traffic signs to achieve the detection and recognition of traffic signs. The model was trained, validated, and tested using TT100K datasets, and the detection accuracy of 36 common categories of traffic signs reached more than 85%, among which the detection accuracy of five categories exceeded 95%. The results showed that, compared with the traditional methods of Faster R-CNN, CornerNet, and CenterNet, traffic sign detection based on a lightweight multiscale feature fusion network had obvious advantages in the speed and accuracy of recognition, significantly improved the detection performance for small targets, and achieved a better real-time performance.

A Size-Grading Method of Antler Mushrooms Using YOLOv5 and PSPNet

Quality grading in antler mushroom industrial production is a labor-intensive operation. For a long time, manual grading has been used for grading, which produces various problems such as insufficient reliability, low production efficiency, and high mushroom body damage. Automatic grading is a problem to be solved urgently for antler mushroom industrial development with increasing labor costs. To solve the problem, this paper deeply integrates the single-stage object detection of YOLOv5 and the semantic segmentation of PSPNet, and proposes a Y-PNet model for real-time object detection and an image segmentation network. This article also proposes an evaluation model for antler mushroom’s size, which eliminates subjective judgment and achieves quality grading. Moreover, to meet the needs of efficient and accurate hierarchical detection in the factory, this study uses the lightweight network model to construct a lightweight YOLOv5 single-stage object detection model. The MobileNetV3 network model embedded with a CBAM module is used as the backbone extractor in PSPNet to reduce the model’s size and improve the model’s efficiency and accuracy for segmentation. Experiments show that the proposed system can perform real-time grading successfully, which can provide instructive and practical references in industry.

ShuffleCloudNet: A Lightweight Composite Neural Network-Based Method for Cloud Computation in Remote-Sensing Images

The occlusion of cloud layers affects the accurate acquisition of ground object information and causes a large amount of useless remote-sensing data transmission and processing, wasting storage, as well as computing resources. Therefore, in this paper, we designed a lightweight composite neural network model to calculate the cloud amount in high-resolution visible remote-sensing images by training the model using thumbnail images and browsing images in remote-sensing images. The training samples were established using paired thumbnail images and browsing images, and the cloud-amount calculation model was obtained by training a proposed composite neural network. The strategy used the thumbnail images for preliminary judgment and the browsing images for accurate calculation, and this combination can quickly determine the cloud amount. The multi-scale confidence fusion module and bag-of-words loss function were redesigned to achieve fast and accurate calculation of cloud-amount data from remote-sensing images. This effectively alleviates the problem of low cloud-amount calculation, thin clouds not being counted as clouds, and that of ice and clouds being confused as in existing methods. Furthermore, a complete dataset of cloud-amount calculation for remote-sensing images, CTI_RSCloud, was constructed for training and testing. The experimental results show that, with less than 13 MB of parameters, the proposed lightweight network model greatly improves the timeliness of cloud-amount calculation, with a runtime is in the millisecond range. In addition, the calculation accuracy is better than the classic lightweight networks and backbone networks of the best cloud-detection models.

A Lightweight Real-Time Rice Blast Disease Segmentation Method Based on DFFANet

The evaluation of rice disease severity is a quantitative indicator for precise disease control, which is of great significance for ensuring rice yield. In the past, it was usually done manually, and the judgment of rice blast severity can be subjective and time-consuming. To address the above problems, this paper proposes a real-time rice blast disease segmentation method based on a feature fusion and attention mechanism: Deep Feature Fusion and Attention Network (abbreviated to DFFANet). To realize the extraction of the shallow and deep features of rice blast disease as complete as possible, a feature extraction (DCABlock) module and a feature fusion (FFM) module are designed; then, a lightweight attention module is further designed to guide the features learning, effectively fusing the extracted features at different scales, and use the above modules to build a DFFANet lightweight network model. This model is applied to rice blast spot segmentation and compared with other existing methods in this field. The experimental results show that the method proposed in this study has better anti-interference ability, achieving 96.15% MioU, a speed of 188 FPS, and the number of parameters is only 1.4 M, which can achieve a high detection speed with a small number of model parameters, and achieves an effective balance between segmentation accuracy and speed, thereby reducing the requirements for hardware equipment and realizing low-cost embedded development. It provides technical support for real-time rapid detection of rice diseases.

Lightweight Smoke Recognition Based on Deep Convolution and Self-Attention

Deep convolutional networks have better smoke recognition performance. However, a lightweight network model and high recognition accuracy cannot be balanced when deployed on hardware with limited computing resources such as edge computing. Based on this background, we propose a novel smoke recognition network that combines convolutional networks (CNN) and self-attention. The core ideas of this framework are as follows: (1) Combine the depthwise convolution and asymmetric convolution of large convolution kernels to construct a lightweight CNN model, and realize multiscale extraction of feature information with slight model complexity. (2) Combined with the self-attention in transformer, a skip-connection branch is designed, which improves the feature extraction capability of the backbone network through parallel processing and fusion of feature map information. (3) Fusion multicomponent discrete cosine transform (DCT) is used to compress channel information and expand the ability of global average pooling (GAP) to aggregate feature maps. The proposed DCT-GAP improves the accuracy of the network without adding additional computational costs. Experimental results show that the proposed CSANet achieves an average accuracy of over 98.3% with 238 M FLOPs and 5.8 M parameters on the homemade smoke dataset, outperforming state-of-the-art competitors.

Driving Behavior Recognition Algorithm Combining Attention Mechanism and Lightweight Network.

In actual driving scenes, recognizing and preventing drivers’ non-standard driving behavior is helpful in reducing traffic accidents. To resolve the problems of various driving behaviors, a large range of action, and the low recognition accuracy of traditional detection methods, in this paper, a driving behavior recognition algorithm was proposed that combines an attention mechanism and lightweight network. The attention module was integrated into the YOLOV4 model after improving the feature extraction network, and the structure of the attention module was also improved. According to the 20,000 images of the Kaggle dataset, 10 typical driving behaviors were analyzed, processed, and recognized. The comparison and ablation experimental results showed that the fusion of an improved attention mechanism and lightweight network model had good performance in accuracy, model size, and FLOPs.

Satellite Image for Cloud and Snow Recognition Based on Lightweight Feature Map Attention Network

Cloud and snow recognition technology is of great significance in the field of meteorology, and is also widely used in remote sensing mapping, aerospace, and other fields. Based on the traditional method of manually labeling cloud-snow areas, a method of labeling cloud and snow areas using deep learning technology has been gradually developed to improve the accuracy and efficiency of recognition. In this paper, from the perspective of designing an efficient and lightweight network model, a cloud snow recognition model based on a lightweight feature map attention network (Lw-fmaNet) is proposed to ensure the performance and accuracy of the cloud snow recognition model. The model is improved based on the ResNet18 network with the premise of reducing the network parameters and improving the training efficiency. The main structure of the model includes a shallow feature extraction module, an intrinsic feature mapping module, and a lightweight adaptive attention mechanism. Overall, in the experiments conducted in this paper, the accuracy of the proposed cloud and snow recognition model reaches 95.02%, with a Kappa index of 93.34%. The proposed method achieves an average precision rate of 94.87%, an average recall rate of 94.79%, and an average F1-Score of 94.82% for four sample recognition classification tasks: no snow and no clouds, thin cloud, thick cloud, and snow cover. Meanwhile, our proposed network has only 5.617M parameters and takes only 2.276 s. Compared with multiple convolutional neural networks and lightweight networks commonly used for cloud and snow recognition, our proposed lightweight feature map attention network has a better performance when it comes to performing cloud and snow recognition tasks.

SVT-Net: Super Light-Weight Sparse Voxel Transformer for Large Scale Place Recognition

Simultaneous Localization and Mapping (SLAM) and Autonomous Driving are becoming increasingly more important in recent years. Point cloud-based large scale place recognition is the spine of them. While many models have been proposed and have achieved acceptable performance by learning short-range local features, they always skip long-range contextual properties. Moreover, the model size also becomes a serious shackle for their wide applications. To overcome these challenges, we propose a super light-weight network model termed SVT-Net. On top of the highly efficient 3D Sparse Convolution (SP-Conv), an Atom-based Sparse Voxel Transformer (ASVT) and a Cluster-based Sparse Voxel Transformer (CSVT) are proposed respectively to learn both short-range local features and long-range contextual features. Consisting of ASVT and CSVT, SVT-Net can achieve state-of-the-art performance in terms of both recognition accuracy and running speed with a super-light model size (0.9M parameters). Meanwhile, for the purpose of further boosting efficiency, we introduce two simplified versions, which also achieve state-of-the-art performance and further reduce the model size to 0.8M and 0.4M respectively.

Convolutional Neural Network-Based Virtual Reality Real-Time Interactive System Design for Unity3D.

In this paper, the algorithm of the convolutional neural network is used for in-depth research and analysis of real-time interactivity, and a virtual reality real-time interactive system is designed based on Unity3D. Aiming at the problems of high computation and low efficiency of existing high-precision models, based on the lightweight network model, this paper proposes a real-time semantic segmentation method based on an asymmetric codec. The encoder part of the network adopts a newly designed bottleneck residual module based on depth-separable convolution, null convolution, and decomposition convolution to extract local and contextual information without increasing the computational effort. At the same time, channel rearrangement is introduced in the module to facilitate the interaction of information between channels. The approach proposed in this paper can present the architectural system simply and clearly and is more flexible to match various complex business relationships. The system design technology is based on the Spring MVC framework technology, using visualization technology design and implementation. The combination of the Spring MVC framework and Unity3D achieves the separation of the front and back ends of the system, which makes the system stable, real-time, visible, and efficient. Meanwhile, a newly designed global attention guidance module based on the attention mechanism is introduced at the jump connection between codecs to guide the low-level features of encoder structure and high-level features of decoder structure for better integration and accuracy improvement.

Object Detection and Ranging System Based on Binocular Vision

For the Census transform stereo matching algorithm is susceptible to noise leading to mis-matching, a stereo matching algorithm based on the improved Census transform is proposed, which improves the Census transform reference value selection and incorporates the image gradient information to form the initial cost. Train the lightweight network model NanoDet to quickly detect and obtain the object position, and combine the object disparity and camera parameters to obtain the distance of the object, and transplant the system to an embedded device for practicality verification. After analyzing and verifying the experimental data, it shows that the anti-noise capability of the improved algorithm is significantly improved, and the proposed method can quickly complete the detection and ranging tasks on embedded devices with high reliability and practicality.

Teacher-student knowledge distillation for real-time correlation tracking

The performance of correlation filter (CF) based visual trackers has been greatly improved with pretrained deep convolutional neural networks. However, these networks limit the application scope of CF based trackers because of high feature dimension, high time consumption of feature extraction and huge memory storage. To alleviate this problem, we introduce a teacher-student knowledge distillation framework to obtain a lightweight network to speed up CF based trackers. Specifically, we take a pretrained deep convolutional neural network from the image classification task as a teacher network, and distill this teacher network into a lightweight student network. During offline distillation training process, we propose an attention transfer loss to ensure the lightweight student network maintains feature representation of the large-capacity teacher network. Meanwhile, we propose a correlation tracking loss to transfer the student network from image classification task to correlation tracking task, which improves the discriminant ability of the student network. Experiments on OTB, VOT2017 and Temple Color show that, using the learned lightweight network model as the feature extractor, the state-of-the-art CF based tracker achieves real-time speed on a single CPU, while maintaining almost the same tracking performance.

Face Detection Method based on Lightweight Network and Weak Semantic Segmentation Attention Mechanism

A face detection method based on lightweight network and weak semantic segmentation attention mechanism is proposed in this paper, aiming at the problems of low detection accuracy and slow detection speed in face detection in complex scenes. K-means++ algorithm is employed to perform clustering analysis on YOLOv4 model prior frames in this paper, and smaller size prior frames are set to capture small face information to solve the missing detection problem of small face targets in scenes. The backbone network structure is improved by introducing Mobile Net lightweight network model, to reduce the number of parameters and calculation of the model and improve the detection speed. The convolutional block attention module model with dual attention mechanism is embedded to improve the sensitivity of the model to target features, which can suppress interference information and improve the accuracy of target detection. A dynamic enhancement attachment based on weak semantic segmentation is added in front of the detector head, whose output is used as the spatial weight distribution to correct the activation area, to suppress the false detection and missed detection caused by the decrease of extraction ability evoked by the pursuit of lightweight. The experimental results on WIDEFACE dataset indicate that this method not only can detect face in real time and with high accuracy, but also has better performance than other existing methods.

A Combined Detection Algorithm for Personal Protective Equipment Based on Lightweight YOLOv4 Model

Improper wearing of personal protective equipment may lead to safety incidents; this paper proposes a combined detection algorithm for personal protective equipment based on the lightweight YOLOv4 model for mobile terminals. To ensure high detection accuracy, a channel and layer pruning method (CLSlim) to lightweight algorithm is used to reduce computing power consumption and improve the detection speed on the basis of the YOLOv4 network. This method applies L1 regularization and gradient sparse training on the scaling factor of the BN layer in the convolutional module: global pruning threshold and local safety threshold are used to eliminate redundant channels, the layer pruning threshold is used to prune the structure of the shortcuts in the Cross Stage Partial (CSP) module for inference speed improvement, and finally, a lightweight network model is obtained. The experiment improves the YOLOv4 and YOLOv4-Tiny models for CLSlim lightweight separately in GTX2080ti environment. Results show that (1) CLSlim-YOLOv4 compresses the YOLOv4 model parameters by 98.2% and increases the inference speed by 1.8 times with mAP loss of only 2.1% and (2) CLSlim-YOLOv4-Tiny compresses the original model parameters by 74.3% and increases the inference speed by 1.1 times with mAP increase of 0.8%, which certificates that this improved lightweight algorithm serves better for the real-time ability and accuracy of combined detection on PPE with mobile terminals.