Lightweight Deep Learning Model Research Articles

Detection of QR Code-based Cyberattacks using a Lightweight Deep Learning Model

Detection of QR Code-based Cyberattacks using a Lightweight Deep Learning Model

MobileAmcT: A Lightweight Mobile Automatic Modulation Classification Transformer in Drone Communication Systems

With the rapid advancement of wireless communication technology, automatic modulation classification (AMC) plays a crucial role in drone communication systems, ensuring reliable and efficient communication in various non-cooperative environments. Deep learning technology has demonstrated significant advantages in the field of AMC, effectively and accurately extracting and classifying modulation signal features. However, existing deep learning models often have high computational costs, making them difficult to deploy on resource-constrained drone communication devices. To address this issue, this study proposes a lightweight Mobile Automatic Modulation Classification Transformer (MobileAmcT). This model combines the advantages of lightweight convolutional neural networks and efficient Transformer modules, incorporating the Token and Channel Conv (TCC) module and the EfficientShuffleFormer module to enhance the accuracy and efficiency of the automatic modulation classification task. The TCC module, based on the MetaFormer architecture, integrates lightweight convolution and channel attention mechanisms, significantly improving local feature extraction efficiency. Additionally, the proposed EfficientShuffleFormer innovatively improves the traditional Transformer architecture by adopting Efficient Additive Attention and a novel ShuffleConvMLP feedforward network, effectively enhancing the global feature representation and fusion capabilities of the model. Experimental results on the RadioML2016.10a dataset show that compared to MobileNet-V2 (CNN-based) and MobileViT-XS (ViT-based), MobileAmcT reduces the parameter count by 74% and 65%, respectively, and improves classification accuracy by 1.7% and 1.09% under different SNR conditions, achieving an accuracy of 62.93%. This indicates that MobileAmcT can maintain high classification accuracy while significantly reducing the parameter count and computational complexity, clearly outperforming existing state-of-the-art AMC methods and other lightweight deep learning models.

Deep learning-based electricity theft prediction in non-smart grid environments

In developing countries, smart grids are nonexistent, and electricity theft significantly hampers power supply. This research introduces a lightweight deep-learning model using monthly customer readings as input data. By employing careful direct and indirect feature engineering techniques, including Principal Component Analysis (PCA), t-distributed Stochastic Neighbor Embedding (t-SNE), UMAP (Uniform Manifold Approximation and Projection), and resampling methods such as Random-Under-Sampler (RUS), Synthetic Minority Over-sampling Technique (SMOTE), and Random-Over-Sampler (ROS), an effective solution is proposed. Previous studies indicate that models achieve high precision, recall, and F1 score for the non-theft (0) class, but perform poorly, even achieving 0 %, for the theft (1) class. Through parameter tuning and employing Random-Over-Sampler (ROS), significant improvements in accuracy, precision (89 %), recall (94 %), and F1 score (91 %) for the theft (1) class are achieved. The results demonstrate that the proposed model outperforms existing methods, showcasing its efficacy in detecting electricity theft in non-smart grid environments.

Balancing accuracy and efficiency: A lightweight deep learning model for COVID 19 detection

The novel coronavirus disease in human was detected in Wuhan, China and spread rapidly across countries each time mutating itself as a new variant. It has impacted not only the patients but also medical infrastructure a lot. Several artificial intelligence-based systems for detection of covid-19 in chest images have been proposed. This study aims to develop a deep learning-based approach that efficiently detect Covid-19 using chest Computed Tomography (CT) images. In this work, we propose an efficient and lightweight deep-learning architecture based on the concept of a densely connected network for Covid-19 detection. The proposed architecture is designed by stacking dense modules in which each layer is directly connected to the other layers, with very fewer number of neurons at each layer. The dense connection is used to improve the learning capability of the proposed model and the fewer number of neurons make network computationally efficient. CT images for covid-19 detection are effectively used for the detection of Covid-19 patients. We train and test our model on SARS-CoV-2 CT-scan datasets. The experimental results are evaluated on accuracy, sensitivity and F1 score. The Adam optimizer with binary cross entropy loss function gives 97.2% accuracy with 0.98 sensitivity, 0.97 precision and 0.97 F1 score. The model achieves maximum accuracy of 88% at precision 0.97%, specificity 0.82 and F1 score 0.89 with Covid-CT dataset. The proposed model is compared with other state-of-the-art methods. The results of the proposed methods and its comparison shows acceptable level of performance on chest CT images.

Lightweight deep learning model for automated STN localization using MER in Parkinson’s disease

Deep brain stimulation (DBS) of Subthalamic nucleus (STN), using a permanent electrode has emerged as an effective treatment for Parkinson’s disease patients. However, accurately localizing STN region during DBS surgery remains a challenging task. Microelectrode recording (MER) is used to identify STN by neurosurgeons. Nevertheless, manual interpretation of STN functional activity might prompt false alarms. Although the existing technology based on deep learning (DL) can outline the STN anatomical borders, it suffers from computational complexity and skewed findings. An end-to-end attention-based lightweight model with residual connections and soft thresholding is designed specifically for STN region detection. Scalogram and spectrogram images are generated from MER to mimic the observation of neurosurgeons and gather contextual information in a more intuitive manner. Proposed model exhibits a simplified architecture with only 15 layers, distinguishing it from previous DL models that tend to be more complex. Spatial attention helped to focus on relevant patterns within the data, while residual connections enhanced the gradient flow throughout the developed network. Soft thresholding offered noise reduction and robustness to signal variability. This approach eliminated the requirement for excessively deep layers and the necessity for separate feature extraction and classification stages by integrating them into a single pipeline. Our model identified STN with an accuracy of 97.42%. Proposed model outperformed 15 existing cutting-edge techniques. Our findings highlight the effectiveness of lightweight DL framework to achieve precise STN localization. The proposed method has significant potential in improving the efficiency of DBS surgery by reducing reliance on the neurosurgeon experience.

Image Forgery Detection Based on Fusion of Light Weight Deep Learning Models

Image Forgery Detection Based on Fusion of Light Weight Deep Learning Models

Bearing-DETR: A Lightweight Deep Learning Model for Bearing Defect Detection Based on RT-DETR.

Detecting bearing defects accurately and efficiently is critical for industrial safety and efficiency. This paper introduces Bearing-DETR, a deep learning model optimised using the Real-Time Detection Transformer (RT-DETR) architecture. Enhanced with Dysample Dynamic Upsampling, Efficient Model Optimization (EMO) with Meta-Mobile Blocks (MMB), and Deformable Large Kernel Attention (D-LKA), Bearing-DETR offers significant improvements in defect detection while maintaining a lightweight framework suitable for low-resource devices. Validated on a dataset from a chemical plant, Bearing-DETR outperformed the standard RT-DETR, achieving a mean average precision (mAP) of 94.3% at IoU = 0.5 and 57.5% at IoU = 0.5-0.95. It also reduced floating-point operations (FLOPs) to 8.2 G and parameters to 3.2 M, underscoring its enhanced efficiency and reduced computational demands. These results demonstrate the potential of Bearing-DETR to transform maintenance strategies and quality control across manufacturing environments, emphasising adaptability and impact on sustainability and operational costs.

경량화 MobileNet을 활용한 축산 데이터 음성 분석

Pigs express their reactions to their environment and health status through a variety of sounds, such as grunting, coughing, and screaming. Given the significance of pig vocalizations, their study has recently become a vital source of data for livestock industry workers. To facilitate this, we propose a lightweight deep learning model based on MobileNet that analyzes pig vocal patterns to distinguish pig voices from farm noise and differentiate between vocal sounds and coughing. This model was able to accurately identify pig vocalizations amidst a variety of background noises and cough sounds within the pigsty. Test results demonstrated that this model achieved a high accuracy of 98.2%. Based on these results, future research is expected to address issues such as analyzing pig emotions and identifying stress levels.

A Forest Fire Smoke Monitoring System Based on a Lightweight Neural Network for Edge Devices

Forest resources are one of the indispensable resources of the earth, which are the basis for the survival and development of human society. With the swift advancements in computer vision and artificial intelligence technology, the utilization of deep learning for smoke detection has achieved remarkable results. However, the existing deep learning models have poor performance in forest scenes and are difficult to deploy because of numerous parameters. Hence, we introduce an optimized forest fire smoke monitoring system for embedded edge devices based on a lightweight deep learning model. The model makes full use of the multi-scale variable attention mechanism of Transformer architecture to strengthen the ability of image feature extraction. Considering the needs of application scenarios, we propose an improved lightweight network model LCNet for feature extraction, which can reduce the parameters and enhance detecting ability. In order to improve running speed, a simple semi-supervised label knowledge distillation scheme is used to enhance the overall detection capability. Finally, we design and implement a forest fire smoke detection system on an embedded device, including the Jetson NX hardware platform, high-definition camera, and detection software system. The lightweight model is transplanted to the embedded edge device to achieve rapid forest fire smoke detection. Also, an asynchronous processing framework is designed to make the system highly available and robust. The improved model reduces three-fourths of the parameters and increases speed by 3.4 times with similar accuracy to the original model. This demonstrates that our system meets the precision demand and detects smoke in time.

A lightweight deep learning model with knowledge distillation for pulmonary diseases detection in chest X-rays

A lightweight deep learning model with knowledge distillation for pulmonary diseases detection in chest X-rays

A Study on Maritime Object Image Classification Using a Pruning-Based Lightweight Deep-Learning Model

Deep learning models require high computing power due to a substantial amount of computation. It is difficult to use them in devices with limited computing environments, such as coastal surveillance equipments. In this study, a lightweight model is constructed by analyzing the weight changes of the convolutional layers during the training process based on MobileNet and then pruning the layers that affects the model less. The performance comparison results show that the lightweight model maintains performance while reducing computational load, parameters, model size, and data processing speed. As a result of this study, an effective pruning method for constructing lightweight deep learning models and the possibility of using equipment resources efficiently through lightweight models in limited computing environments such as coastal surveillance equipments are presented.

SKINC-NET: an efficient Lightweight Deep Learning Model for Multiclass skin lesion classification in dermoscopic images

SKINC-NET: an efficient Lightweight Deep Learning Model for Multiclass skin lesion classification in dermoscopic images

Real-time tracking method for motion spatter in high-power laser welding of stainless steel plate based on a lightweight deep learning model

The high-power laser welding of stainless steel is prone to spatter defects. The dynamic behavior of motion spatter is closely related to the welding state, but the relevant features are difficult to obtain. This paper extensively describes a motion spatter tracking method based on a lightweight deep learning model and a multi-target tracking algorithm. Firstly, an in-situ observation platform with a high-speed camera-based high spatiotemporal resolution for motion spatters is established. This platform can record the generation and motion process of different spatters during welding. To reduce the interferences and highlight the spatter characteristics, a linear point operation-based image low-brightness metal vapor plume removal method is also proposed. Then, a deep learning-based lightweight spatter detection model is established, and an automatic spatter label generation method is proposed to quickly obtain a sufficient amount of highly discriminative training data to train the established model comprehensively. Compared with the basic model, the validation results show that the lightweight spatter detection model runs 5.11 times faster, with a spatter detection accuracy of 96.71%. Finally, a motion spatter tracking method based on the DeepSORT algorithm is proposed to distinguish different spatters and associate the same spatters. The spatter size, velocity, quantity, and other features are also able to be accurately extracted via this method, achieving spatter defect monitoring. In summary, this study describes a novel and robust method for motion spatter real-time tracking during laser welding and this method can provide reliable data support for accurate and fast process condition monitoring as well as weld quality assessment.

TriCh-LKRepNet: A large kernel convolutional malicious code classification network for structure reparameterisation and triple-channel mapping

In response to escalating cybersecurity threats, this study aims to develop a lightweight deep learning model for efficient and accurate detection and classification of malware. To achieve this goal, the study introduces RGB-MalNet, a novel network architecture that balances performance and resource utilization. The method innovatively transforms malware representations into image channels through RGB three-channel mapping, which improves information richness and discriminative power. RGB-MalNet creates a streamlined framework that optimizes network connections, reduces memory access overhead, and boosts overall efficiency. The model achieves accuracy rates of 99.47% and 97.55% on the Kaggle and DataCon datasets, respectively. Compared to existing methodologies, this approach stands out in terms of performance, resource consumption, and versatility. It offers a viable and efficient solution for malicious code detection and classification.

An Improved Lightweight Deep Learning Model and Implementation for Track Fastener Defect Detection with Unmanned Aerial Vehicles

Track fastener defect detection is an essential component in ensuring railway safety operations. Traditional manual inspection methods no longer meet the requirements of modern railways. The use of deep learning image processing techniques for classifying and recognizing abnormal fasteners is faster, more accurate, and more intelligent. With the widespread use of unmanned aerial vehicles (UAVs), conducting railway inspections using lightweight, low-power devices carried by UAVs has become a future trend. In this paper, we address the characteristics of track fastener detection tasks by improving the YOLOv4-tiny object detection model. We improved the model to output single-scale features and used the K-means++ algorithm to cluster the dataset, obtaining anchor boxes that were better suited to the dataset. Finally, we developed the FPGA platform and deployed the transformed model on this platform. The experimental results demonstrated that the improved model achieved an mAP of 95.1% and a speed of 295.9 FPS on the FPGA, surpassing the performance of existing object detection models. Moreover, the lightweight and low-powered FPGA platform meets the requirements for UAV deployment.

A lightweight deep learning approach for detecting electrocardiographic lead misplacement

Objective. Electrocardiographic (ECG) lead misplacement can result in distorted waveforms and amplitudes, significantly impacting accurate interpretation. Although lead misplacement is a relatively low-probability event, with an incidence ranging from 0.4% to 4%, the large number of ECG records in clinical practice necessitates the development of an effective detection method. This paper aimed to address this gap by presenting a novel lead misplacement detection method based on deep learning models. Approach. We developed two novel lightweight deep learning model for limb and chest lead misplacement detection, respectively. For limb lead misplacement detection, two limb leads and V6 were used as inputs, while for chest lead misplacement detection, six chest leads were used as inputs. Our models were trained and validated using the Chapman database, with an 8:2 train-validation split, and evaluated on the PTB-XL, PTB, and LUDB databases. Additionally, we examined the model interpretability on the LUDB databases. Limb lead misplacement simulations were performed using mathematical transformations, while chest lead misplacement scenarios were simulated by interchanging pairs of leads. The detection performance was assessed using metrics such as accuracy, precision, sensitivity, specificity, and Macro F1-score. Main results. Our experiments simulated three scenarios of limb lead misplacement and nine scenarios of chest lead misplacement. The proposed two models achieved Macro F1-scores ranging from 93.42% to 99.61% on two heterogeneous test sets, demonstrating their effectiveness in accurately detecting lead misplacement across various arrhythmias. Significance. The significance of this study lies in providing a reliable open-source algorithm for lead misplacement detection in ECG recordings. The source code is available at https://github.com/wjcai/ECG_lead_check.

RTSeg-net: A lightweight network for real-time segmentation of fetal head and pubic symphysis from intrapartum ultrasound images

The segmentation of the fetal head (FH) and pubic symphysis (PS) from intrapartum ultrasound images plays a pivotal role in monitoring labor progression and informing crucial clinical decisions. Achieving real-time segmentation with high accuracy on systems with limited hardware capabilities presents significant challenges. To address these challenges, we propose the real-time segmentation network (RTSeg-Net), a groundbreaking lightweight deep learning model that incorporates innovative distribution shifting convolutional blocks, tokenized multilayer perceptron blocks, and efficient feature fusion blocks. Designed for optimal computational efficiency, RTSeg-Net minimizes resource demand while significantly enhancing segmentation performance. Our comprehensive evaluation on two distinct intrapartum ultrasound image datasets reveals that RTSeg-Net achieves segmentation accuracy on par with more complex state-of-the-art networks, utilizing merely 1.86 M parameters—just 6 % of their hyperparameters—and operating seven times faster, achieving a remarkable rate of 31.13 frames per second on a Jetson Nano, a device known for its limited computing capacity. These achievements underscore RTSeg-Net's potential to provide accurate, real-time segmentation on low-power devices, broadening the scope for its application across various stages of labor. By facilitating real-time, accurate ultrasound image analysis on portable, low-cost devices, RTSeg-Net promises to revolutionize intrapartum monitoring, making sophisticated diagnostic tools accessible to a wider range of healthcare settings.

Enhanced fingerprint pattern classification: Integrating attention modules with lightweight deep learning models

AbstractLarge fingerprint databases can make the automated search process tedious and time‐consuming. Fingerprint pattern classification is a significant step in the identification system's complexity in terms of time and speed. Although several fingerprint algorithms have been developed for classification tasks, further improvements in performance and efficiency are still required. Most of the fingerprint algorithms use deep learning techniques. However, some deep learning techniques can be resource‐intensive and computationally expensive, while others can disregard the spatial relationships between the features used in classifying fingerprint patterns. This study proposes using lightweight deep learning models (i.e., MobileNet and EfficientNet‐B0) integrated with attention modules to classify fingerprint patterns. The two lightweight models are modified, yielding MobileNet+ and EfficientNet‐B0+ models. The lightweight deep learning models can help achieve optimal performance and reduce computational complexity. The attention modules focus on distinctive features for classification. Our proposed approach integrates four attention modules for fingerprint pattern classification into two lightweight deep learning models, that is, MobileNet+ and EfficientNet‐B0+. To evaluate our approach, we use two publicly available fingerprint datasets, that is, the NIST special database 301 dataset and the LivDet dataset. The evaluation results show that the EfficientNet‐B0+ model achieves the highest classification accuracy of 97% with only 854,086 training parameters. As a conclusion, we consider the training parameters small enough for the EfficientNet‐B0+ model to be deployed on low‐resource devices.

Mapping large-scale pine wilt disease trees with a lightweight deep-learning model and very high-resolution UAV images

ABSTRACT Pine wilt disease (PWD), caused by pine wood nematodes, has brought a great loss in ecology and economy all over the world. In China, the forest health status is also significantly affected by PWD since 1980, especially in coniferous forests and mixed forest regions. The PWD spreads very fast and can cause a healthy pine wood tree to die within a very short time. An effective way to protect other healthy pine wood trees is to discover PWD trees early. Using unmanned aerial vehicle (UAV) images can help people discover the PWD tree quickly and accurately, and a few automatical methods have been developed to monitor PWD trees including the deep learning methods. Because of the robust spatial-temporal transferability, deep learning methods have become the mainstream algorithms to monitor PWD trees. As we know, the training dataset is the most important material to train a deep learning model. However, there is still a lack of a PWD segmentation dataset so far. To fill this gap, in this paper, we have generated the first open-sourced PWD segmentation dataset based on very high-resolution UAV images to help the community conduct PWD monitor research conveniently. This dataset has 994 training samples, and each sample has visible bands with 512 × 512 pixels, and the spatial resolution of this dataset is 0.05 m. In order to train an advanced segmentation model, we have designed a lightweight deep-learning model for mobile devices or edge devices in this manuscript, named MobileSeg. The main feature of MobileSeg is its decoupling, which uses the re-parameterization technology to improve the model performance. Finally, a large-scale real-world scenario experiment with very high-resolution UAV images was utilized to validate the performance of MobileSeg, the experiment result indicated that MobileSeg has achieved the best performance compared with the recent lightweight segmentation models, and the experiment also proved the effectiveness of the proposed PWD dataset.

Research on non-intrusive load identification method based on multi-feature fusion with improved shufflenetv2

Non-intrusive load monitoring is a significant advancement in energy conservation and smart electricity usage as it enables the identification of load status and equipment type without the need for extensive sensing instruments. As one of the important steps of non-intrusive load monitoring, non-intrusive load identification plays a crucial role in the correct identification of electrical appliances. In this paper, a novel approach is presented to amalgamate multiple appliance features into a new identification feature, which is then used for appliance recognition through an improved lightweight deep learning model. This addresses the issue of low accuracy when using a single feature for appliance identification. The core concept involves initially capturing the current data of an appliance during steady state operation and fusing the image features of the appliance encoded by Gramian summation angular field, Gramian difference angular field, and Markov transition field into new recognition features for each appliance using average weighting. Subsequently, a shufflenetv2 lightweight deep learning model based on the squeeze -and-excitation module is used to mine the constructed load feature information for the load classification task. The method is then experimentally validated on the Self-test Dataset, PLAID, and WHITED dataset, resulting in recognition accuracies of 100%, 98.214%, and 99.745%, respectively. These results demonstrate the effectiveness of the proposed method in significantly improving recognition performance compared to the original approach.