Lightweight Model Research Articles

Towards lightweight military object detection

Military object military object detection technology serves as the foundation and critical component for reconnaissance and command decision-making, playing a significant role in information-based and intelligent warfare. However, many existing military object detection models focus on exploring deeper and more complex architectures, which results in models with a large number of parameters. This makes them unsuitable for inference on mobile or resource-constrained combat equipment, such as combat helmets and reconnaissance Unmanned Aerial Vehicles (UAVs). To tackle this problem, this paper proposes a lightweight detection framework. A CSP-GhostnetV2 module is proposed in our method to make the feature extraction network more lightweight while extracting more effective information. Furthermore, to fuse multiscale information in low-computational scenarios, GSConv and the proposed CSP-RepGhost are used to form a lightweight feature aggregation network. The experimental results demonstrate that our proposed lightweight model has significant advantages in detection accuracy and efficiency compared to other detection algorithms.

Attention-based digital filter with anchor-free feature pyramid learning model for pedestrian detection

Ensuring real-time performance while leveraging pedestrian detection is a crucial prerequisite for intelligent driving technology. The development of lightweight models with good detection accuracy is also crucial. This work proposes a novel method, the Attention Digital Filter with Anchor-Free Feature Pyramid Learning Model (ADFAFPLM), to meet these needs. The suggested method consists of combining two networks: one is a digital filter based on an attention network that eliminates noise and other picture distortions. The attention-based residual network digital filters are chosen for their enhanced filtering performance, adaptability, efficient learning through residual connections, noise suppression, interpretability, and generalization capabilities. Next, from the input crowded and occluded photos, the pedestrian is identified using an anchor-free feature pyramid network. The Eurocity person dataset was used to train the model, and it was also tested on other datasets like CityPersons, INRIA, PennFudan, and Eurocity. The investigation was expanded to include images in hazy, noisy, and occlusion environments, among other environmental conditions. The image resolutions were also considered for analysis and it was observed that with increasing image resolution, the mAP increases. Based on the ablation study, the ADF-AFPLM adopted YOLOv8n with batch size 16, and image size 640 is considered for efficient result with different testing datasets. The model achieved a mean average precision (mAP) of approx. 87% and shows its efficacy over state-of-art models.

A Deep Analysis of Printed Image Forensics with Auto Machine Learning

This research undertakes an exhaustive exploration of the Auto Machine Learning-based approach (AutoML) to ascertain the source printer for microscopic printed image forensics. The primary focus is on enhancing precision and efficacy in a scenario with limited sample data, achieved by integrating traditional image processing methods with diverse AutoML representatives. The investigation involves a meticulous evaluation and comparative analysis of classical Machine Learning (ML) models and AutoML models in microscopic printed image forensics. Through extensive experiments, we highlight the inherent limitations of traditional models and underscore the compelling advantages offered by AutoML. Notably, we recognize AutoML’s unique capability to discern varying degrees of uncertainty within printed patterns. The outcomes emphasize the prowess of Auto Machine Learning in improving the accuracy of printed image forensics, making it particularly promising for future research, especially given the tabular nature of the data and the imperative for a lightweight model suitable for embedded systems.

A serial semantic segmentation model based on encoder-decoder architecture

The thriving progress of Convolutional Neural Networks (CNNs) and the outstanding efficacy of Visual Transformers (ViTs) have delivered impressive outcomes in the domain of semantic segmentation. However, each model in isolation entails a trade-off between high computational complexity and compromised computational efficiency. To address this challenge, we effectively combine the CNN and encoder-decoder structures in a Transformer-inspired fashion, presenting the Serial Semantic Segmentation Trans via CNN Former (SSS-Former) model. To augment the feature extraction capability, we utilize the meticulously crafted SSS-CSPNet, resulting in a well-designed architecture for the holistic model. We propose a novel SSS-PN attention network that enhances the spatial topological connections of features, leading to improved overall performance. Additionally, the integration of SASPP bridges the semantic gap between multi-scale features and enhances segmentation ability for overlapping objects. To fulfill the requirement of real-time segmentation, we leverage a novel restructuring technique to devise a more lightweight and faster ResSSS-Former model. Abundant experimental results demonstrate that both SSS-Former and ResSSS-Former outperform existing state-of-the-art methods in terms of computational efficiency, result precision, and speed. Remarkably, SSS-Former achieves a mIoU of 58.63 % at 89.1FPS on the ADE20K dataset. On the validation and testing datasets of CityScapes, it obtains mIoU scores of 85.1 % and 85.2 % respectively, with a speed of 94.1FPS. Our optimized ResSSS-Former achieves impressive real-time segmentation results, with an astonishing 100+FPS while maintaining high segmentation accuracy. The compelling results from the ISPRS datasets further validate the effectiveness of our proposed models in segmenting multi-scale and overlapping objects.

Enhanced fingerprint pattern classification: Integrating attention modules with lightweight deep learning models

AbstractLarge fingerprint databases can make the automated search process tedious and time‐consuming. Fingerprint pattern classification is a significant step in the identification system's complexity in terms of time and speed. Although several fingerprint algorithms have been developed for classification tasks, further improvements in performance and efficiency are still required. Most of the fingerprint algorithms use deep learning techniques. However, some deep learning techniques can be resource‐intensive and computationally expensive, while others can disregard the spatial relationships between the features used in classifying fingerprint patterns. This study proposes using lightweight deep learning models (i.e., MobileNet and EfficientNet‐B0) integrated with attention modules to classify fingerprint patterns. The two lightweight models are modified, yielding MobileNet+ and EfficientNet‐B0+ models. The lightweight deep learning models can help achieve optimal performance and reduce computational complexity. The attention modules focus on distinctive features for classification. Our proposed approach integrates four attention modules for fingerprint pattern classification into two lightweight deep learning models, that is, MobileNet+ and EfficientNet‐B0+. To evaluate our approach, we use two publicly available fingerprint datasets, that is, the NIST special database 301 dataset and the LivDet dataset. The evaluation results show that the EfficientNet‐B0+ model achieves the highest classification accuracy of 97% with only 854,086 training parameters. As a conclusion, we consider the training parameters small enough for the EfficientNet‐B0+ model to be deployed on low‐resource devices.

A light-weight defect detection model for capacitor appearance based on the Yolov5

As one of the most important electronic components, capacitors are very important for appearance inspection in the production process. However, the current production process mainly relies on manual inspection, which not only reduces product quality and production efficiency but also increases production costs. The automated detection is limited by the computational resources of the equipment, which is difficult to apply in practice. Therefore, in this paper, we propose a lightweight method for capacitor appearance inspection. We use the YOLOv5 (You Only Look Once Version 5) framework, MobileNet as the backbone network, and GSConv (Ghost convolution) and GSCSP module as the neck depth compression network model to reduce the computational cost. In addition, we incorporate the CBAM (Convolutional Block Attention Module) into the backbone network to improve the network's ability to extract features. For the problem of difficult detection of small targets, we use CIoU (Complete Intersection Union) and NWD (Normalised Gaussian Wasserstein Distance) to design a new loss function. By testing our method on capacitor appearance defect data, compared to the baseline, the model computational cost FLOPs was reduced by 130 %, the model size was reduced by 94 %, the accuracy reached 92.5 %, and the mAP (mean average precision) reached 92.3 %, while the number of frames detected per second was up to 58 frames. The experimental results show that our method is capable of real-time detection of capacitor appearance defects, providing strong theoretical support for practical applications.

Knowledge Distillation Based on Fitting Ground-Truth Distribution of Images

Knowledge distillation based on the features from the penultimate layer allows the student (lightweight model) to efficiently mimic the internal feature outputs of the teacher (high-capacity model). However, the training data may not conform to the ground-truth distribution of images in terms of classes and features. We propose two knowledge distillation algorithms to solve the above problem from the directions of fitting the ground-truth distribution of classes and fitting the ground-truth distribution of features, respectively. The former uses teacher labels to supervise student classification output instead of dataset labels, while the latter designs feature temperature parameters to correct teachers’ abnormal feature distribution output. We conducted knowledge distillation experiments on the ImageNet-2012 and Cifar-100 datasets using seven sets of homogeneous models and six sets of heterogeneous models. The experimental results show that our proposed algorithms improve the performance of penultimate layer feature knowledge distillation and outperform other existing knowledge distillation methods in terms of classification performance and generalization ability.

GCT-YOLOv5: a lightweight and efficient object detection model of real-time side-scan sonar image

GCT-YOLOv5: a lightweight and efficient object detection model of real-time side-scan sonar image

Quality Detection and Grading of Rose Tea Based on a Lightweight Model.

Rose tea is a type of flower tea in China's reprocessed tea category, which is divided into seven grades, including super flower, primary flower, flower bud, flower heart, yellow flower, scattered flower, and waste flower. Grading rose tea into distinct quality levels is a practice that is essential to boosting their competitive advantage. Manual grading is inefficient. We provide a lightweight model to advance rose tea grading automation. Firstly, four kinds of attention mechanisms were introduced into the backbone and compared. According to the experimental results, the Convolutional Block Attention Module (CBAM) was chosen in the end due to its ultimate capacity to enhance the overall detection performance of the model. Second, the lightweight module C2fGhost was utilized to change the original C2f module in the neck to lighten the network while maintaining detection performance. Finally, we used the SIoU loss in place of the CIoU loss to improve the boundary regression performance of the model. The results showed that the mAP, precision (P), recall (R), FPS, GFLOPs, and Params values of the proposed model were 86.16%, 89.77%, 83.01%, 166.58, 7.978, and 2.746 M, respectively. Compared with the original model, the mAP, P, and R values increased by 0.67%, 0.73%, and 0.64%, the GFLOPs and Params decreased by 0.88 and 0.411 M, respectively, and the speed was comparable. The model proposed in this study also performed better than other advanced detection models. It provides theoretical research and technical support for the intelligent grading of roses.

Mapping large-scale pine wilt disease trees with a lightweight deep-learning model and very high-resolution UAV images

ABSTRACT Pine wilt disease (PWD), caused by pine wood nematodes, has brought a great loss in ecology and economy all over the world. In China, the forest health status is also significantly affected by PWD since 1980, especially in coniferous forests and mixed forest regions. The PWD spreads very fast and can cause a healthy pine wood tree to die within a very short time. An effective way to protect other healthy pine wood trees is to discover PWD trees early. Using unmanned aerial vehicle (UAV) images can help people discover the PWD tree quickly and accurately, and a few automatical methods have been developed to monitor PWD trees including the deep learning methods. Because of the robust spatial-temporal transferability, deep learning methods have become the mainstream algorithms to monitor PWD trees. As we know, the training dataset is the most important material to train a deep learning model. However, there is still a lack of a PWD segmentation dataset so far. To fill this gap, in this paper, we have generated the first open-sourced PWD segmentation dataset based on very high-resolution UAV images to help the community conduct PWD monitor research conveniently. This dataset has 994 training samples, and each sample has visible bands with 512 × 512 pixels, and the spatial resolution of this dataset is 0.05 m. In order to train an advanced segmentation model, we have designed a lightweight deep-learning model for mobile devices or edge devices in this manuscript, named MobileSeg. The main feature of MobileSeg is its decoupling, which uses the re-parameterization technology to improve the model performance. Finally, a large-scale real-world scenario experiment with very high-resolution UAV images was utilized to validate the performance of MobileSeg, the experiment result indicated that MobileSeg has achieved the best performance compared with the recent lightweight segmentation models, and the experiment also proved the effectiveness of the proposed PWD dataset.

Towards interpretability lightweight semantic segmentation model for waterbody extraction in large-scale high resolution remote sensing images

ABSTRACT In recent years, high resolution (HR) remote sensing images have brought significant changes to land cover monitoring. Monitoring water resources, a vital and scarce commodity for human survival, is a crucial aspect of land cover monitoring and forms the foundation for water resource allocation in regions facing shortages. Hence, a universal method is necessary to address this problem and handle large-scale water resources monitoring. We explore the network structure via improving the interpretability to reduce the network layers and propose a lightweight pixel-wise semantic segmentation model, which achieves waterbody extraction from GF-1 remote sensing images with high accuracy and high speed. The proposed model structure addresses both semantic segmentation for large and small waterbodies, extracting fine-grained edge and contour information from high-resolution feature maps. Additionally, it efficiently extracts multi-scale high-level semantic information using residual convolutional blocks and dilated convolutional blocks. The multi-scale feature maps are fused, and binary classification is predicted through Support Vector Machines (SVM). Furthermore, the paper introduces a model training method with an adaptive learning rate, reducing the overall training time. To validate the model’s performance, remote sensing images from GF-1 are utilized to construct a dataset. Experimental results, compared with five models, demonstrate that the proposed method achieves the highest accuracy and least training time.

Simplified neural architecture for efficient human motion prediction in human-robot interaction

Human motion prediction is essential for safe and effective human-robot interaction, but modeling the intricate spatio-temporal dynamics inherent in human movement remains challenging. While recent methods have advanced motion prediction accuracy, they rely on complex neural architectures such as Graph Convolutional Networks and Recurrent Neural Networks, demanding extensive hyperparameter tuning. To overcome this limitation, we propose a streamlined multi-stage layer incremental perceptron (MSLP) architecture that achieves competitive results with far fewer parameters, improving efficiency. The MSLP eschews the complexity of prevalent networks and instead takes a stepped refinement approach to predict intricate motions using a lightweight model. This simplified yet effective design enables nuanced learning of spatio-temporal relationships without exhaustive tuning. Specifically, the MSLP incorporates two key components: a multi-channel feature extraction and enhancement block (MFE-block) and a temporal feature extraction module (TFE-block). The MFE-block strengthens the representation of each action node by integrating multi-dimensional action features. The TFE-block then captures the contextual relationships between actions over time. Together, the MFE-block and TFE-block allow the MSLP to model the complex spatial and temporal dynamics of human movement using a streamlined architecture and minimal parameter tuning. When evaluated on established datasets including Human3.6 M, CMU, 3DPW, and AMASS, the proposed MSLP method achieves improved accuracy gains of 16.7%-67.0% over existing state-of-the-art techniques. Additionally, the MSLP significantly reduces the number of parameters by 35.7%-99.5% compared to prior architectures. We find that the proposed MSLP significantly improves both long-term prediction and generalization capabilities of the model.

A deep neural network for vehicle detection in aerial images

This research paper highlights the significance of vehicle detection in aerial images for surveillance systems, focusing on deep learning methods that outperform traditional approaches. However, the challenge of high computation complexity due to diverse vehicle appearances persists. The motivation behind this study is to highlight the crucial role of vehicle detection in aerial images for surveillance systems, emphasizing the superior performance of deep learning methods compared to traditional approaches. To address this, a lightweight deep neural network-based model is developed, striking a balance between accuracy and efficiency enabling real-time operation. The model is trained and evaluated on a standardized dataset, with extensive experiments demonstrating its ability to achieve accurate vehicle detection with significantly reduced computation costs, offering a practical solution for real-world aerial surveillance scenarios.

Video-based face outline recognition

We propose a novel approach for individual recognition that uses the motion traits from face outline-anonymised videos as the identity signatures; we call this type of signature a Temporal-iD. To extract a robust Temporal-iD, a highly lightweight transformer-based model, namely the Temporal-iD-ViT (TiDViT), is devised to capture and aggregate Temporal-iD features from videos. The TiDViT is equipped with a custom-designed multi-head temporal–spatial joint attention module that establishes interaction between the current frame input and the previous hidden state, thereby temporally aggregating the temporal features. The TiDViT processes the face video frame by frame instead of in a fixed batch-wise manner, which requires less computational memory. Moreover, the TiDViT can extract the temporal features of unconstrained face videos and considers ethical and privacy concerns. Extensive experimental results show that the proposed TiDViT model achieves a decent performance on this highly challenging task.

Detection of dental periapical lesions using retinex based image enhancement and lightweight deep learning model

Dental periapical lesions, commonly associated with inflammation around the tooth apex, pose a significant challenge in early diagnosis and treatment. This study introduces a novel approach for the detection of dental periapical lesions through the integration of Retinex-based image enhancement techniques and a lightweight deep learning model. The Retinex algorithm is employed to enhance the radiographic images, addressing issues related to inconsistent illumination and contrast. Subsequently, a tailored lightweight deep learning model is designed to efficiently extract relevant features from the enhanced images. Present methodology leverages a dataset of dental radiographs to train and evaluate the deep learning model, incorporating a diverse range of periapical lesion cases. The model is optimized for computational efficiency while maintaining high accuracy, making it suitable for deployment in resource-constrained environment. To enhance precision in lesion detection, the U-Net segmentation technique has been incorporated, providing a sophisticated approach to delineate and analyze specific areas of interest within the radiographic images. This addition further refines our diagnostic framework, contributing to the robustness of lesion identification. Experimental results demonstrate the effectiveness of the Retinex-based image enhancement in improving the visibility of periapical lesions. The lightweight deep learning model exhibits promising performance in accurately detecting and classifying dental periapical lesions, showcasing its potential for early and efficient diagnosis. The results obtained from the present model are compared with those from Convolutional Neural Network (CNN) as well as with diagnosis of expert practitioners and the model is observed to perform very well. The study contributes to the advancement of computer-aided diagnostic tools in dentistry, offering a scalable and accessible solution for the identification of dental periapical lesions through the fusion of image enhancement and lightweight deep learning techniques.

Fast and Accurate Multi-Task Learning for Encrypted Network Traffic Classification

The classification of encrypted traffic plays a crucial role in network management and security. As encrypted network traffic becomes increasingly complicated and challenging to analyze, there is a growing need for more efficient and comprehensive analytical approaches. Our proposed method introduces a novel approach to network traffic classification, utilizing multi-task learning to simultaneously train multiple tasks within a single model. To validate the proposed method, we conducted experiments using the ISCX 2016 VPN/Non-VPN dataset, consisting of three tasks. The proposed method outperformed the majority of existing methods in classification with 99.29%, 97.38%, and 96.89% accuracy in three tasks (i.e., encapsulation, category, and application classification, respectively). The efficiency of the proposed method also demonstrated outstanding performance when compared to methods excluding lightweight models. The proposed approach demonstrates accurate and efficient multi-task classification on encrypted traffic.

Grow-light smart monitoring system leveraging lightweight deep learning for plant disease classification

This work focuses on a novel lightweight machine learning approach to the task of plant disease classification, posing as a core component of a larger grow-light smart monitoring system. To the extent of our knowledge, this work is the first to implement lightweight convolutional neural network architectures leveraging down-scaled versions of inception blocks, residual connections, and dense residual connections applied without pre-training to the PlantVillage dataset. The novel contributions of this work include the proposal of a smart monitoring framework outline; responsible for detection and classification of ailments via the devised lightweight networks as well as interfacing with LED grow-light fixtures to optimize environmental parameters and lighting control for the growth of plants in a greenhouse system. Lightweight adaptation of dense residual connections achieved the best balance of minimizing model parameters and maximizing performance metrics with accuracy, precision, recall, and F1-scores of 96.75%, 97.62%, 97.59%, and 97.58% respectively, while consisting of only 228,479 model parameters. These results are further compared against various full-scale state-of-the-art model architectures trained on the PlantVillage dataset, of which the proposed down-scaled lightweight models were capable of performing equally to, if not better than many large-scale counterparts with drastically less computational requirements.

Light3DHS: A lightweight 3D hippocampus segmentation method using multiscale convolution attention and vision transformer

The morphological analysis and volume measurement of the hippocampus are crucial to the study of many brain diseases. Therefore, an accurate hippocampal segmentation method is beneficial for the development of clinical research in brain diseases. U-Net and its variants have become prevalent in hippocampus segmentation of Magnetic Resonance Imaging (MRI) due to their effectiveness, and the architecture based on Transformer has also received some attention. However, some existing methods focus too much on the shape and volume of the hippocampus rather than its spatial information, and the extracted information is independent of each other, ignoring the correlation between local and global features. In addition, many methods cannot be effectively applied to practical medical image segmentation due to many parameters and high computational complexity. To this end, we combined the advantages of CNNs and ViTs (Vision Transformer) and proposed a simple and lightweight model: Light3DHS for the segmentation of the 3D hippocampus. In order to obtain richer local contextual features, the encoder first utilizes a multi-scale convolutional attention module (MCA) to learn the spatial information of the hippocampus. Considering the importance of local features and global semantics for 3D segmentation, we used a lightweight ViT to learn high-level features of scale invariance and further fuse local-to-global representation. To evaluate the effectiveness of encoder feature representation, we designed three decoders of different complexity to generate segmentation maps. Experiments on three common hippocampal datasets demonstrate that the network achieves more accurate hippocampus segmentation with fewer parameters. Light3DHS performs better than other state-of-the-art algorithms.

DuKA: A Dual-Keyless-Attention Model for Multi-Modality EHR Data Fusion and Organ Failure Prediction.

Organ failure is a leading cause of mortality in hospitals, particularly in intensive care units. Predicting organ failure is crucial for clinical and social reasons. This study proposes a dual-keyless-attention (DuKA) model that enables interpretable predictions of organ failure using electronic health record (EHR) data. Three modalities of medical data from EHR, namely diagnosis, procedure, and medications, are selected to predict three types of vital organ failures: heart failure, respiratory failure, and kidney failure. DuKA utilizes pre-trained embeddings of medical codes and combines them using a modality-wise attention module and a medical concept-wise attention module to enhance interpretation. Three organ failure tasks are addressed using two datasets to verify the effectiveness of DuKA. The proposed multi-modality DuKA model outperforms all reference and baseline models. The diagnosis history, particularly the presence of cachexia and previous organ failure, emerges as the most influential feature in organ failure prediction. DuKA offers competitive performance, straightforward model interpretations and flexibility in terms of input sources, as the input embeddings can be trained using different datasets and methods. DuKA is a lightweight model that innovatively uses dual attention in a hierarchical way to fuse diagnosis, procedure and medication information for organ failure predictions. It also enhances disease comprehension and supports personalized treatment.

Combined Anomaly Aware Weakly Supervised Lightweight Model for Surface Defect Inspection

Combined Anomaly Aware Weakly Supervised Lightweight Model for Surface Defect Inspection