Lightweight Attention Research Articles

A lightweight fault diagnosis model for planetary gearbox using domain adaptation and model compression

This article proposes a novel lightweight attention spatiotemporal joint distribution adaptation network fault diagnosis model to address the key challenges of domain transfer and high model complexity in traditional methods. The novelty lies in 1. Using model compression techniques to reduce the complexity of the network model and improve its computational efficiency; 2. Introducing new domain adaptation and adversarial methods to solve the domain transfer problem. The effectiveness of the proposed model is verified through a transfer experiment of planetary gearbox vibration data. The experimental results show that the proposed model reduces the parameters and computational complexity to 18 % and 15 % of the original model, respectively, and has a diagnostic accuracy of over 98 % in cross-condition transfer tasks, and still maintains an accuracy of over 88 % even under high noise levels. This indicates that the proposed model is an efficient and accurate fault diagnosis model.

The Study on Recognition and Detection of Express Package Grabbing Based on Machine Vision

This research addresses the automated supply of parcels in logistics by employing a robotic arm to replace manual intervention. The process involves the suction-based retrieval of disorderedly stacked parcels and their placement onto a conveyor belt, presenting an economically significant engineering challenge. This paper designs a visual recognition system to assist the robotic arm in identifying the grasping surface of disorderedly stacked parcels. Leveraging transfer learning, a pre-trained model constructs a multimodal network framework. Pre-trained models of ResNet 169, DenseNet121, ResNet101, and ResNet50 serve as backbone networks to train and test on custom parcel datasets. Post comparative testing of model performance, DenseNet169 is chosen to construct the visual recognition network. Specifically, the RGB and Depth data of parcels are separately fed into DenseNet169 for feature extraction. Post-feature extraction, multimodal fusion is applied, integrating a lightweight attention mechanism, CBAM, to enhance semantic segmentation accuracy. Subsequently, post-processing of image features filters out the background, achieving precise identification of the parcel grasping region. Ultimately, the constructed network achieves an average TOP1 true class accuracy of 95.86% on the test set. Thus, the designed parcel visual recognition system based on this methodology exhibits robustness, meeting the demand for autonomous parcel retrieval by the robotic arm. It offers significant support towards resolving challenges in automated parcel supply within logistics.

Human Pose Estimation Based on Efficient and Lightweight High-Resolution Network (EL-HRNet).

As an important direction in computer vision, human pose estimation has received extensive attention in recent years. A High-Resolution Network (HRNet) can achieve effective estimation results as a classical human pose estimation method. However, the complex structure of the model is not conducive to deployment under limited computer resources. Therefore, an improved Efficient and Lightweight HRNet (EL-HRNet) model is proposed. In detail, point-wise and grouped convolutions were used to construct a lightweight residual module, replacing the original 3 × 3 module to reduce the parameters. To compensate for the information loss caused by the network's lightweight nature, the Convolutional Block Attention Module (CBAM) is introduced after the new lightweight residual module to construct the Lightweight Attention Basicblock (LA-Basicblock) module to achieve high-precision human pose estimation. To verify the effectiveness of the proposed EL-HRNet, experiments were carried out using the COCO2017 and MPII datasets. The experimental results show that the EL-HRNet model requires only 5 million parameters and 2.0 GFlops calculations and achieves an AP score of 67.1% on the COCO2017 validation set. In addition, PCKh@0.5mean is 87.7% on the MPII validation set, and EL-HRNet shows a good balance between model complexity and human pose estimation accuracy.

DPNet: Dual-Path Network for Real-Time Object Detection With Lightweight Attention.

The recent advances in compressing high-accuracy convolutional neural networks (CNNs) have witnessed remarkable progress in real-time object detection. To accelerate detection speed, lightweight detectors always have few convolution layers using a single-path backbone. Single-path architecture, however, involves continuous pooling and downsampling operations, always resulting in coarse and inaccurate feature maps that are disadvantageous to locate objects. On the other hand, due to limited network capacity, recent lightweight networks are often weak in representing large-scale visual data. To address these problems, we present a dual-path network, named DPNet, with a lightweight attention scheme for real-time object detection. The dual-path architecture enables us to extract in parallel high-level semantic features and low-level object details. Although DPNet has a nearly duplicated shape with respect to single-path detectors, the computational costs and model size are not significantly increased. To enhance representation capability, a lightweight self-correlation module (LSCM) is designed to capture global interactions, with only a few computational overheads and network parameters. In the neck, LSCM is extended into a lightweight cross correlation module (LCCM), capturing mutual dependencies among neighboring scale features. We have conducted exhaustive experiments on MS COCO, Pascal VOC 2007, and ImageNet datasets. The experimental results demonstrate that DPNet achieves a state-of-the-art trade off between detection accuracy and implementation efficiency. More specifically, DPNet achieves 31.3% AP on MS COCO test-dev, 82.7% mAP on Pascal VOC 2007 test set, and 41.6% mAP on ImageNet validation set, together with nearly 2.5M model size, 1.04 GFLOPs, and 164 and 196 frames/s (FPS) FPS for [Formula: see text] input images of three datasets.

FL-ToLeD: An Improved Lightweight Attention Convolutional Neural Network Model for Tomato Leaf Diseases Classification for Low-end Devices

FL-ToLeD: An Improved Lightweight Attention Convolutional Neural Network Model for Tomato Leaf Diseases Classification for Low-end Devices

Hierarchical Transformer With Lightweight Attention for Radar-Based Precipitation Nowcasting

Hierarchical Transformer With Lightweight Attention for Radar-Based Precipitation Nowcasting

Dual-resolution transformer combined with multi-layer separable convolution fusion network for real-time semantic segmentation

Environmental perception is crucial for unmanned mobile platforms such as autonomous vehicles and robots. Precise and fast semantic segmentation of the surrounding scene is a key task to enhance this capability. Existing real-time semantic segmentation networks are typically based on convolutional neural networks (CNNs), which have achieved good results, but they still lack control over global context features. In recent years, the Transformer architecture has achieved significant success in capturing global context, which is beneficial for improving segmentation accuracy. However, Transformers tend to ignore local connections, and their computational complexity makes real-time segmentation challenging. We propose a lightweight real-time semantic segmentation network called DTMC-Net, which combines the advantages of CNNs and Transformers. We design a special residual convolution module called the Lightweight Multi-layer Separable Convolution Attention module (LMSCA) to reduce the parameter count and perform multi-scale feature fusion to capture local features effectively. We introduce the Simple Dual-Resolution Transformer (SDR Transformer) that utilizes lightweight attention mechanisms and residual feed forward networks to capture and maintain features, with multiple bilateral fusions between two branches to exchange information. The proposed Anti-artifact Aggregation Pyramid Pooling Module (AAPPM) optimizes the upsampling process, refines features, and performs multi-scale feature fusion again. DTMC-Net only contains 4.2M parameters and achieves good performance on multiple public datasets with different scenarios.

Research on improved YOLOx weed detection based on lightweight attention module

Accurate weed detection is essential for precise weed control in farmland, and machine vision serves as an effective method for identifying and targeting these unwanted plants. An advanced YOLOx weed detection model, incorporating a deep network connection lightweight attention mechanism, has been proposed to effectively identify distinct types of weeds in maize seedling fields. The lightweight attention module is connected to the deep network of YOLOx-Darknet, which weakens the channel noise effect of residual computation and thus makes the detection model more efficient. A deconvolution layer has been introduced in the residual module to improve the small size feature extraction capability. The Generalized Intersection over Union (GIoU) has replaced the Intersection over Union (IoU) to minimize the positional discrepancy between the predicted frame and the actual frame, thereby enhancing the learning capacity of the detection model. Compared with the original algorithm, the improved one achieved an average detection accuracy of 94.86% in the performance evaluation, 0.07% better in the F1 value, and 1.16% better in the AP value. The weeding robot with an improved YOLOx algorithm had 92.45% detection rate of maize seedlings and 88.94% detection rate of weed recognition at 0.2 m s−1. The results of this study can provide technical references for real-time weed detection and robotic precision weeding.

Multi-lead-time short-term runoff forecasting based on Ensemble Attention Temporal Convolutional Network

In the realm of ecological management and human activities within river basins, short-term runoff forecasting plays a pivotal role. Addressing this need, this paper introduces an innovative framework for short-term runoff forecasting: the Ensemble Attention Temporal Convolutional Network (EA-TCN). The cornerstone of this innovation lies in the effective amalgamation of Temporal Convolutional Network (TCN), lightweight attention mechanism, and ensemble learning strategy. This integration synergistically enhances the model’s overall performance in terms of accuracy, efficiency, and robustness. TCN forms the foundation of this framework, where its efficient architecture, characterized by shared parameters and parallel computation, significantly boosts computational efficiency. Its employment of causal and dilated convolutions adeptly captures long-term dependencies within time series inputs. The incorporated lightweight attention mechanism further augments the TCN, enabling EA-TCN to precisely discern complex relationship in temporal data, particularly exhibiting remarkable temporal robustness across various forecasting horizons—a feat challenging for conventional forecasting approaches. Additionally, the integration of the Snapshot ensemble method within the framework allows for simulating the effect of training multiple models through one single training process, thus further elevating the model’s accuracy and robustness. Rigorous ablation and comparative experiments conducted on the US Columbia River dataset substantiate our claims. The results not only validate the individual merits of each component within EA-TCN but also illuminate the significant advantages of their collective application. Our comprehensive assessment unequivocally demonstrates the framework’s exceptional performance in short-term runoff forecasting, positioning it as a state-of-the-art solution in this field. We will further discuss its impact of vocation education in this industry.

TeenyNet: a novel lightweight attention model for sunflower disease detection

Sunflower is one of the oilseed crops which is popularly and widely cultivated globally and contributes significantly to human health. Leaf diseases of sunflower seriously affect the growth and yield of sunflower, which directly leads to the loss of agricultural economy. However, existing machine learning algorithms and deep learning techniques are mainly based on large models with attention mechanisms, which lack considerations in computational performance, especially model size. Therefore, this study proposes a lightweight model called TeenyNet to break through the dilemma. First, the designed global multi-frequency feature extraction module decomposes the image to extract multi-frequency multi-scale features. Then, a parameter-free maximum pooling layer further extracts edge and texture features and simplifies the network complexity through downsampling, after which the proposed lightweight dual fusion attention and multi-branching structure fuses all the feature vectors to enhance multidimensional feature learning and accelerate the model convergence. Finally, the fully connected linear layer solves the multi-classification problem of sunflower disease under natural illumination background conditions. The experimental results show that TeenyNet obtains the highest accuracy of 98.94% for sunflower disease recognition with a minimum size of 143 KB and has better recognition performance in comparison experiments. TeenyNet can be effectively used for the detection of sunflower leaf diseases to achieve disease prevention and control.

Research on tile surface defect detection by improved YOLOv5

Traditional methods for detecting surface defects on ceramic tiles result in misdetection and missed detection, which makes it difficult to guarantee product stability and consistency within the same batch. Therefore, this article proposes an improved YOLOv5 algorithm for detecting surface defects on ceramic tiles. Firstly, the Res2Net module is combined with self-attention to fully utilize local and global information and improve the feature extraction effect of defects. Secondly, the GS-BiFPN neck network is designed to enhance the fusion capability of shallow detail and deep semantic information and alleviate ambiguity and redundancy on the feature map. Then, a lightweight attention module is introduced to improve the detection capability of difficult-to-recognize defects and anti-background interference. Finally, the SIoU loss function improves the model’s convergence speed and accuracy. Experimental results demonstrate that the improved algorithm’s mean average precision (mAP) reaches 73.3%, 6.3% higher than the baseline model. Even when compared with YOLOv7-tiny, the mAP of the improved algorithm has increased by 8.7%. Additionally, the detection speed of the model can reach 92 frames per second, which can meet the requirements of ceramic tile surface defect detection in industrial scenarios.

MSA R-CNN: A comprehensive approach to remote sensing object detection and scene understanding

Remote sensing scene understanding is crucial for extracting valuable information from high-resolution images, including object detection and classification. Traditional object detection methods face challenges in handling the diverse scales, orientations, and complex backgrounds present in remote sensing data. In this paper, we propose a novel remote sensing scene understanding system called multiscale-attention R-CNN (MSA R-CNN), which incorporates a super multiscale feature extraction network (SMENet) for enhanced feature extraction from multiscale images, an adaptive dynamic inner lateral (ADIL) connection module to tackle information loss in feature pyramid networks (FPN), and a distributed lightweight attention module (DLAM) to refine feature information processing. Furthermore, a new dataset combining the DIOR and DOTA datasets is introduced to extract the background information of detected objects and evaluate the proposed system’s performance. MSA R-CNN achieved an mAP of 74.37% on the DIOR dataset when the gamma value was set to 0.2 and 81.97% on the DOTA dataset when the gamma value was set to 0.1 with the same learning rate, outperforming state-of-the-art models on both datasets. The proposed system demonstrates significant improvements in both object detection and background information extraction, providing a comprehensive solution for remote sensing scene understanding.

WaterFormer: A coupled transformer and CNN network for waterbody detection in optical remotely-sensed imagery

As one of the most significant components of the ecosystem, waterbody needs to be highly monitored at different spatial and temporal scales. Nevertheless, waterbody variations in shape, size, and reflectivity, complicated and varied types of land covers, and environmental scene diversity, present colossal challenges in achieving accurate waterbody detection (WD). In this paper, we propose a novel network coupled with the Transformer and convolutional neural network (CNN), termed WaterFormer, to automatically, efficiently, and accurately delineate waterbodies from optical high-resolution remotely sensed (HR-RS) images. This network mainly includes a dual-stream CNN, a cross-level Vision Transformer, a light-weight attention module, and a sub-pixel up-sampling module. First, the dual-stream network abstracts waterbody features at multi-views and different levels. Then, to exploit the long-range dependencies between low-level spatial information and high-order semantic features, the cross-level Vision Transformer is embedded into the dual-stream, aiming at improving WD accuracy. Afterwards, the light-weight attention module is adopted to provide semantically strong feature abstractions by enhancing discrimination neurons, and the sub-pixel up-sampling module is employed to further generate high-resolution and high-quality class-specific representations. Quantitative and qualitative evaluations demonstrated that the WaterFormer provided a promising means for detecting waterbody areas in satellite images under complex scene conditions. Moreover, comparative analyses with the state-of-the-art (SOTA) alternatives, e.g., MSFENet, MSAFNet, and BiSeNet, also verified the generalization and superiority of the WaterFormer in WD tasks. The assessment results exhibited that the WaterFormer gained an average accuracy of 97.24%, average precision of 94.59%, average recall of 91.95%, average F1-score of 93.24%, and average Kappa index of 0.9133, respectively. Additionally, we presented an open-access HR satellite imagery waterbody dataset, a mesoscale dataset with high-quality and high-precision waterbody annotation to facilitate future research in this field. The dataset has been released at https://github.com/NJdeuK/WD_Dataset.

Lite-YOLOv3: a real-time object detector based on multi-scale slice depthwise convolution and lightweight attention mechanism

Lite-YOLOv3: a real-time object detector based on multi-scale slice depthwise convolution and lightweight attention mechanism

Research on Human Pose Estimation Based on High-resolution Network

The problem addressed in this paper is the increase in the parameter quantity of existing human body pose estimation network models when the depth of the network is scaled up to improve forecast precision. To tackle this issue, an optimization network model of the new human body pose estimation called BDENet is proposed, inspired by high-resolution detection networks. This model incorporates a bottleneck structure and dilated convolution to reduce parameters and incorporates the ECA lightweight attention mechanism to enhance precision. Compared with HRNet, the proposed model achieves a 21.4% reduction in parameter quantity on the MSCOCO dataset while scaling up precision by 1.4%. The experimental findings strongly suggest that the modified network significantly improves network precision along with lowering the number of parameters.

Badminton video action recognition based on time network

With the continuous development of artificial intelligence research, computer vision research has shifted from traditional “feature engineering”-based methods to deep learning-based “network engineering” methods, which automatically extracts and classifies features by using deep neural networks. Traditional methods based on artificial design features are computationally expensive and are usually used to solve simple research problems, which is not conducive for large-scale data feature extraction. Deep learning-based methods greatly reduce the difficulty of artificial features by learning features from large-scale data and are successfully applied in many visual recognition tasks. Video action recognition methods also shift from traditional methods based on artificial design features to deep learning-based methods, which is oriented to building more effective deep neural network models. Through collecting and sorting related research results found that academic for timing segment network of football and basketball video action research is relatively rich, but lack of badminton research given the above research results, this study based on timing segment network of badminton video action identification can enrich the research results, provide reference for follow-up research. This paper introduces the lightweight attention mechanism into the temporal segmentation network, forming the attention mechanism-timing segmentation network, and trains the neural network to get the classifier of badminton stroke action, which can be predicted as four common types: forehand stroke, backhand stroke, overhead stroke and pick ball. The experimental results show that the recognition recall and accuracy of various stroke movements reach more than 86%, and the average size of recall and accuracy is 91.2% and 91.6% respectively, indicating that the method based on timing segmentation network can be close to the human judgment level and can effectively conduct the identification task of badminton video strokes.

GAMNet: Global attention via multi‐scale context for depth estimation algorithm and application

AbstractDeep neural networks significantly enhance the accuracy of the stereo‐based disparity estimation. Some current methods suffer from inefficient use of the global context information, which will lead to the loss of structural details in ill‐posed areas. To this end, a novel stereo network GAMNet is designed, composed of three core components (GDA, MPF, DCA) for estimating the depth prediction in challenging real‐world environments. First, a lightweight attention module is presented, integrating the global semantic cues for every feature position across the channel and spatial dimensions. Next, the MPF module is constructed to fuse the diverse semantic and contextual information from different levels of the feature pyramid. Finally, cost volume is aggregated with a stacked encoder‐decoder composed of the DCA module and 3D convolutions, filtering the transmission of matching clues and capturing the rich global contexts. Substantial experiments conducted on KITTI 2012, KITTI 2015, SceneFlow, and Middlebury‐v3 datasets manifest that GAMNet surpasses preceding methods with contour‐preserving disparity predictions. In addition, the first 3D scene reconstruction linear evaluation strategy on spatial grasping points for the end‐to‐end stereo networks in an unsupervised mode is proposed, and it is deployed on the designed robot vision‐guided system. In application experiments, the method can produce densely high‐precision 3D reconstructions to implement the grasping task in complex real‐world scenes, and achieve excellent robust performance with competitive inference efficiency.

YOLO-DRS: A Bioinspired Object Detection Algorithm for Remote Sensing Images Incorporating a Multi-Scale Efficient Lightweight Attention Mechanism.

Bioinspired object detection in remotely sensed images plays an important role in a variety of fields. Due to the small size of the target, complex background information, and multi-scale remote sensing images, the generalized YOLOv5 detection framework is unable to obtain good detection results. In order to deal with this issue, we proposed YOLO-DRS, a bioinspired object detection algorithm for remote sensing images incorporating a multi-scale efficient lightweight attention mechanism. First, we proposed LEC, a lightweight multi-scale module for efficient attention mechanisms. The fusion of multi-scale feature information allows the LEC module to completely improve the model's ability to extract multi-scale targets and recognize more targets. Then, we propose a transposed convolutional upsampling alternative to the original nearest-neighbor interpolation algorithm. Transposed convolutional upsampling has the potential to greatly reduce the loss of feature information by learning the feature information dynamically, thereby reducing problems such as missed detections and false detections of small targets by the model. Our proposed YOLO-DRS algorithm exhibits significant improvements over the original YOLOv5s. Specifically, it achieves a 2.3% increase in precision (P), a 3.2% increase in recall (R), and a 2.5% increase in mAP@0.5. Notably, the introduction of the LEC module and transposed convolutional results in a respective improvement of 2.2% and 2.1% in mAP@0.5. In addition, YOLO-DRS only increased the GFLOPs by 0.2. In comparison to the state-of-the-art algorithms, namely YOLOv8s and YOLOv7-tiny, YOLO-DRS demonstrates significant improvements in the mAP@0.5 metrics, with enhancements ranging from 1.8% to 7.3%. It is fully proved that our YOLO-DRS can reduce the missed and false detection problems of remote sensing target detection.

EL-NAS: Efficient Lightweight Attention Cross-Domain Architecture Search for Hyperspectral Image Classification

Deep learning (DL) algorithms have demonstrated important breakthroughs for hyperspectral image (HSI) classification. Despite the remarkable success of DL, the burden of a manually designed DL structure with increased depth and size aroused the difficulty for the application in the mobile and embedded devices in a real application. To tackle this issue, in this paper, we proposed an efficient lightweight attention network architecture search algorithm (EL-NAS) for realizing an efficient automatic design of a lightweight DL structure as well as improving the classification performance of HSI. First, aimed at realizing an efficient search procedure, we construct EL-NAS based on a differentiable network architecture search (NAS), which can greatly accelerate the convergence of the over-parameter supernet in a gradient descent manner. Second, in order to realize lightweight search results with high accuracy, a lightweight attention module search space is designed for EL-NAS. Finally, further for alleviating the problem of higher validation accuracy and worse classification performance, the edge decision strategy is exploited to perform edge decisions through the entropy of distribution estimated over non-skip operations to avoid further performance collapse caused by numerous skip operations. To verify the effectiveness of EL-NAS, we conducted experiments on several real-world hyperspectral images. The results demonstrate that the proposed EL-NAS indicates a more efficient search procedure with smaller parameter sizes and high accuracy performance for HSI classification, even under data-independent and sensor-independent scenarios.

YOLO v7-ECA-PConv-NWD Detects Defective Insulators on Transmission Lines

This paper proposes an enhanced YOLO v7-based method for detecting insulator defects in transmission lines, addressing the challenges of low accuracy and high leakage rates caused by complex backgrounds and electric poles alongside varying sizes of insulator targets in the image. Firstly, to address the issue of background interference and improve the importance of insulator features, a lightweight attention mechanism named Efficient Channel Attention (ECA) was introduced. With the incorporation of ECA, this model could effectively suppress background noise and provide more focus to insulator regions, thus enhancing its ability to detect insulator defects accurately. Secondly, a partial convolution (PConv) approach was employed in the backbone network instead of conventional convolution, which learned some important channels. This substitution improved both the network model’s accuracy and the training speed. Finally, the Normalized Wasserstein Distance (NWD) prevented insulator features from being lost during pre-feature extraction, which reduced the leakage rate and improved the detection accuracy of small target insulators and defective insulators. The experimental results demonstrate that the improved YOLO v7 network model achieved an average detection accuracy (mAP) of 98.1%, recall of 93.7%, and precision of 96.8% on the TISLTR dataset. On the FISLTR dataset, the average detection accuracy (mAP) for flashover insulators was 93%, with a recall of 92.3% and precision of 87.1%. The average detection accuracy (mAP) for broken insulators was 92.2%, with a recall of 90.3% and a precision of 95.2%. These metrics demonstrate significant improvements in both datasets, highlighting the proposed algorithms’ strong generalization capability and practicable potential to detect insulator targets.