Model Size Reduction Research Articles

Improved lightweight infrared road target detection method based on YOLOv8

Infrared-based road scene object detection algorithms often face issues with excessive parameters and computational demands, making them incompatible with edge devices having constrained computational capabilities. This paper introduces an enhanced lightweight infrared-based road object detection algorithm based on YOLOv8n. Firstly, a streamlined network architecture is devised by merging YOLOv8n’s C2f module with PConv, creating a lighter module and reducing the neural network’s downsampling rate of infrared images. This strategy reduces redundant computations and memory access, preventing the loss of fine details in infrared images caused by deep convolutional neural networks. Additionally, the model’s accuracy in detecting infrared targets is significantly enhanced through the integration of the coordinate attention mechanism. Finally, replacing CIoU with Wise-IoU for bounding box regression in YOLOv8n accelerates the model’s convergence. Empirical findings indicate that in contrast to the YOLOv8n algorithm, the optimized model showcases a 34.17 % reduction in model size, a 40.35 % decrease in parameters, and a 4.8 % increase in average detection accuracy. This enhanced algorithm not only achieves a lightweight profile but also delivers superior performance on embedded edge devices.

TP-Transfiner: high-quality segmentation network for tea pest.

Detecting and controlling tea pests promptly are crucial for safeguarding tea production quality. Due to the insufficient feature extraction ability of traditional CNN-based methods, they face challenges such as inaccuracy and inefficiency of detecting pests in dense and mimicry scenarios. This study proposes an end-to-end tea pest detection and segmentation framework, TeaPest-Transfiner (TP-Transfiner), based on Mask Transfiner to address the challenge of detecting and segmenting pests in mimicry and dense scenarios. In order to improve the feature extraction inability and weak accuracy of traditional convolution modules, this study proposes three strategies. Firstly, a deformable attention block is integrated into the model, which consists of deformable convolution and self-attention using the key content only term. Secondly, the FPN architecture in the backbone network is improved with a more effective feature-aligned pyramid network (FaPN). Lastly, focal loss is employed to balance positive and negative samples during the training period, and parameters are adapted to the dataset distribution. Furthermore, to address the lack of tea pest images, a dataset called TeaPestDataset is constructed, which contains 1,752 images and 29 species of tea pests. Experimental results on the TeaPestDataset show that the proposed TP-Transfiner model achieves state-of-the-art performance compared with other models, attaining a detection precision (AP50) of 87.211% and segmentation performance of 87.381%. Notably, the model shows a significant improvement in segmentation average precision (mAP) by 9.4% and a reduction in model size by 30% compared to the state-of-the-art CNN-based model Mask R-CNN. Simultaneously, TP-Transfiner's lightweight module fusion maintains fast inference speeds and a compact model size, demonstrating practical potential for pest control in tea gardens, especially in dense and mimicry scenarios.

PCB Defect Detection Based on Improved Deep Learning Model

Printed circuit boards (PCBs) play a critical role in electronic products. Ensuring these products’ long-term reliability and consistent performance requires effective PCB defect detection. Although existing deep learning models for PCB defect detection are not highly accurate, they often neglect capability considerations. This paper introduces a precise, fast, and lightweight defect detection model, CCG-YOLO, based on an enhanced YOLOv5 model to address this issue. The enhancements in CCG-YOLO can be summarized as follows: (1) Improved Backbone network: The feature extraction ability of the Backbone network is enhanced by introducing a C3HB module, which fosters spatial interaction capabilities. (2) Lightweight feature fusion network: A lightweight convolution structure called Ghost-Shuffle Convolution is incorporated in the feature fusion network, remarkably reducing model parameters while maintaining performance. (3) Efficient residual networking: To enhance model performance further, a CNeB module is introduced based on the ConvNeXt network, which replaces the C3 module in the Neck. CNeB improves model detection accuracy and reduces the number of model parameters. The combination of these enhancements results in impressive performance. CCG-YOLO achieves mean average precision (mAP@0.5) of 99.5% and 88.75% in mAP@0.5:0.95 on the TDD-Net public dataset. Compared with the original YOLOv5s algorithm, CCG-YOLO offers a 4.24% improvement in mAP@0.5:0.95, a 1[Formula: see text]MB reduction in model size, a 0.472[Formula: see text]M decrease in the number of parameters, a 0.6G floating point operation reduction in computational complexity, and a 120 frames per second real-time inference speed. These experimental results underscore that the proposed model excels in accuracy and speed and has a compact size for PCB defect detection. Moreover, CCG-YOLO is easily deployable on low-end devices, making it well-suited for meeting the real-time requirements of industrial defect detection.

Edge human activity recognition using federated learning on constrained devices

Human Activity Recognition (HAR) using wearable Internet of Things (IoT) devices represents a well investigated researched field encompassing various application domains. Many current approaches rely on cloud-based methodologies for gathering data from diverse users, resulting in the creation of extensive training datasets. Although this strategy facilitates the application of powerful Machine Learning (ML) techniques, it raises significant privacy concerns, which can become particularly severe given the sensitivity of HAR data. Moreover, the labeling process can be extremely time-consuming and even more challenging for IoT wearable devices due to the absence of efficient input systems. In this paper, we address both aforementioned challenges by designing, implementing, and validating edge-based Human Activity Recognition (HAR) systems that operate on resource-constrained IoT devices, which relies on the utilization of Self-Organizing Maps (SOM) for activity detection. We incorporate a feature selection process before training to reduce data dimensionality and, consequently, the SOM size, aligning with the resource limitations of wearable IoT devices. Additionally, we explore the application of Federated Learning (FL) techniques for HAR tasks, enabling new users to leverage SOM models trained by others on their respective datasets. Our federated Extreme Edge (EE)-aware HAR system is implemented on a wearable IoT device and rigorously tested against state-of-the-art and experimental datasets. The results demonstrate that our C++-based SOM implementation achieves a consistent reduction in model size compared to state-of-the-art approaches. Furthermore, our findings highlight the effectiveness of the FL-based approach in overcoming personalized training challenges, particularly in onboarding scenarios.

Lightweight-detection: The strip steel surface defect identification based on improved YOLOv5d

The surface defect of strip steel seriously affects the product quality, the current identification methods have the inherent defects of low detection efficiency and poor detection accuracy, and it is important an improved an algorithm based on the modified YOLOv5 framework to enhance the defect detection effectiveness and accuracy. To create a more lightweight network model, the C3 module and part of the convolutional structure in the original YOLOv5 network were replaced with the GhostBottleneck structure. Additionally, the Coordinate Attention (CA) mechanism was incorporated into the Backbone section to compensate for the original model's lack of attention mechanisms. To address the angular mismatch between the actual frame and the predicted bounding box, a limitation overlooked by the CIOU loss function was further refined. Experimental results demonstrate that the enhanced YOLOv5s-CSG model outperformed the original YOLOv5s by 7.3 %, achieving a significant 37.9 % reduction in model size and an mAP value of 77.5 %. Furthermore, the detection precision and speed were notably higher compared to several widely-used algorithms. The proposed YOLOv5s-CSG model is well-suited for deployment on mobile devices and shows great potential for rapidly and accurately detecting both the type and location of strip surface defects, surpassing the performance of existing methods.

Gesture Recognition of Filipino Sign Language Using Convolutional and Long Short-Term Memory Deep Neural Networks

In response to the recent formalization of Filipino Sign Language (FSL) and the lack of comprehensive studies, this paper introduces a real-time FSL gesture recognition system. Unlike existing systems, which are often limited to static signs and asynchronous recognition, it offers dynamic gesture capturing and recognition of 10 common expressions and five transactional inquiries. To this end, the system sequentially employs cropping, contrast adjustment, grayscale conversion, resizing, and normalization of input image streams. These steps serve to extract the region of interest, reduce the computational load, ensure uniform input size, and maintain consistent pixel value distribution. Subsequently, a Convolutional Neural Network and Long-Short Term Memory (CNN-LSTM) model was employed to recognize nuances of real-time FSL gestures. The results demonstrate the superiority of the proposed technique over existing FSL recognition systems, achieving an impressive average accuracy, recall, and precision rate of 98%, marking an 11.3% improvement in accuracy. Furthermore, this article also explores lightweight conversion methods, including post-quantization and quantization-aware training, to facilitate the deployment of the model on resource-constrained platforms. The lightweight models show a significant reduction in model size and memory utilization with respect to the base model when executed in a Raspberry Pi minicomputer. Lastly, the lightweight model trained with the quantization-aware technique (99%) outperforms the post-quantization approach (97%), showing a notable 2% improvement in accuracy.

LungVision: X-ray Imagery Classification for On-Edge Diagnosis Applications

This study presents a comprehensive analysis of utilizing TensorFlow Lite on mobile phones for the on-edge medical diagnosis of lung diseases. This paper focuses on the technical deployment of various deep learning architectures to classify nine respiratory system diseases using X-ray imagery. We propose a simple deep learning architecture that experiments with six different convolutional neural networks. Various quantization techniques are employed to convert the classification models into TensorFlow Lite, including post-classification quantization with floating point 16 bit representation, integer quantization with representative data, and quantization-aware training. This results in a total of 18 models suitable for on-edge deployment for the classification of lung diseases. We then examine the generated models in terms of model size reduction, accuracy, and inference time. Our findings indicate that the quantization-aware training approach demonstrates superior optimization results, achieving an average model size reduction of 75.59%. Among many CNNs, MobileNetV2 exhibited the highest performance-to-size ratio, with an average accuracy loss of 4.1% across all models using the quantization-aware training approach. In terms of inference time, TensorFlow Lite with integer quantization emerged as the most efficient technique, with an average improvement of 1.4 s over other conversion approaches. Our best model, which used EfficientNetB2, achieved an F1-Score of approximately 98.58%, surpassing state-of-the-art performance on the X-ray lung diseases dataset in terms of accuracy, specificity, and sensitivity. The model experienced an F1 loss of around 1% using quantization-aware optimization. The study culminated in the development of a consumer-ready app, with TensorFlow Lite models tailored to mobile devices.

Deep Learning Model Compression Techniques Performance on Edge Devices

Artificial intelligence at the edge can help solve complex tasks faced by various sectors such as automotive, healthcare and surveillance. However, challenged by the lack of computational power from the edge devices, artificial intelligence models are forced to adapt. Many have developed and quantified model compres-sion approaches over the years to tackle this problem. However, not many have considered the overhead of on-device model compression, even though model compression can take a considerable amount of time. With the added metric, we provide a more complete view on the efficiency of model compression on the edge. The objective of this research is identifying the benefit of compression methods and it’s tradeoff between size and latency reduction versus the accuracy loss as well as compression time in edge devices. In this work, quantitative method is used to analyze and rank three common ways of model compression: post-training quantization, unstructured pruning and knowledge distillation on the basis of accuracy, latency, model size and time to compress overhead. We concluded that knowledge distillation is the best, with potential of up to 11.4x model size reduction, and 78.67% latency speed up, with moderate loss of accura-cy and compression time.

ResPrune: An energy-efficient restorative filter pruning method using stochastic optimization for accelerating CNN

Convolutional Neural Networks (CNNs) are frequently employed for image pattern recognition and other computer vision tasks. When over-parameterized deep learning models are used for inference, resource-constrained edge devices may struggle. As a result, model compression, particularly filter pruning, has become critical. A reduction in model size might result in less calculation, resulting in faster hardware execution and lower energy consumption. One of the drawbacks of current pruning strategies is that once the filters are pruned, their weights are permanently lost. To address this constraint, we propose a unique two-phase pruning technique in which the filters to be pruned are selected using two criteria: l2-norm and redundancy. Second, rather than omitting the selected filters for all future epochs, we restore them to their original value with some stochasticity. Retaining the most optimal filter weights in earlier epochs enables the survival of the fittest filters, resulting in higher model convergence. Experiments on three benchmark datasets, CIFAR-10, CIFAR-100, and ILSVRC-2012, reveal that our strategy outperforms other state-of-the-art pruning methods by a minimum reduction of 57% FLOPs with an accuracy loss as minimal as 0.08 %.

An adaptive modeling method with a local choice of optimal displacement fields for finite element analysis of structures

In the standard finite element procedure, the user chooses himself which mechanical theory will be used for a given application. To this end, he relies on some rules acquired through experience or some theoretical consideration. But choosing an appropriate theory may be a difficult task when geometry, boundary conditions, loadings, and materials are complex. This paper aims to define an adaptive methodology to identify, in the context of linear static analysis, optimal finite element models from a theory choice point of view. A criterion is defined to choose, in each part of the structure, the relevant mechanical theory: solid, shell or beam. A solid mesh is defined for the whole structure while specific solid-shell or solid-beam approaches are used in shell or beam areas respectively. This avoids the construction of mid-surface or mid-axis geometries from solid ones, which is a complicated task, in particular for industrial applications. Kinematic relations between nodes are imposed to apply the displacement fields of shell or beam theories. This leads to a set of linear equations which are used for eliminating slave degrees of freedom. The methodology proposed can also be interpreted as a model size reduction method, compared to a complete solid approach. The effectiveness of this approach is demonstrated through two numerical examples, including academic cantilever structures and an industrial multilayered composite structure.

A Review of Abstraction Methods Toward Verifying Neural Networks

Neural networks as a machine learning technique are increasingly deployed in various domains. Despite their performance and their continuous improvement, the deployment of neural networks in safety-critical systems, in particular for autonomous mobility, remains restricted. This is mainly due to the lack of (formal) specifications and verification methods and tools that allow for having sufficient confidence in the behavior of the neural-network-based functions. Recent years have seen neural network verification getting more attention; many verification methods were proposed, yet the practical applicability of these methods to real-world neural network models remains limited. The main challenge of neural network verification methods is related to the computational complexity and the large size of neural networks pertaining to complex functions. As a consequence, applying abstraction methods for neural network verification purposes is seen as a promising mean to cope with such issues. The aim of abstraction is to build an abstract model by omitting some irrelevant details or some details that are not highly impacting w.r.t some considered features. Thus, the verification process is made faster and easier while preserving, to some extent, the relevant behavior regarding the properties to be examined on the original model. In this article, we review both the abstraction techniques for activation functions and model size reduction approaches, with a particular focus on the latter. The review primarily discusses the application of abstraction techniques on feed-forward neural networks and explores the potential for applying abstraction to other types of neural networks. Throughout the article, we present the main idea of each approach and then discuss its respective advantages and limitations in detail. Finally, we provide some insights and guidelines to improve the discussed methods.

CAC-YOLOv8: real-time bearing defect detection based on channel attenuation and expanded receptive field strategy

Bearing defect detection plays a crucial role in the intelligent production of chemical transmission equipment, where timely identification and handling of defective bearings are essential. However, in practical large-scale industrial production, product surface defects are often complex, diverse, and exhibit significant variations in appearance, posing severe challenges to the discriminative ability and detection efficiency of bearing defect detection algorithms. This paper proposes a real-time bearing surface defect detection algorithm, CAC-YOLOv8, which designs the Channel Attenuation Network (CAN) and Compound Pooling Pyramid Spatial Pyramid Pooling Fast (CPPSPPF) structure. Specifically, the model introduces the Channel Attenuation Network to achieve parallel feature extraction, deep feature processing, and feature fusion under different channel numbers, capturing critical features related to bearing defects and thereby improving the inference speed. Subsequently, based on the concept of overlapped receptive fields, a CPPSPPF structure is constructed, utilizing multiple iterations of max-pooling operations with smaller pooling kernel sizes to prevent information loss while expanding the receptive field, thereby strengthening the capturing ability of features at different scales. The experimental results indicate that the proposed CAC-YOLOv8 bearing surface defect detection algorithm, compared to the YOLOv8 model, achieved a 0.3% improvement in mAP@0.5, reduced model size by 14.4%, and enhanced model inference speed by 33.3%. This enables the CAC-YOLOv8 model to significantly improve the real-time performance of bearing defect detection while maintaining high-precision detection. The performance in practical industrial detection demonstrates that the proposed approach has achieved outstanding results in both speed and accuracy.

Shufflemono: Rethinking Lightweight Network for Self-Supervised Monocular Depth Estimation

Abstract Self-supervised monocular depth estimation has been widely applied in autonomous driving and automated guided vehicles. It offers the advantages of low cost and extended effective distance compared with alternative methods. However, like automated guided vehicles, devices with limited computing resources struggle to leverage state-of-the-art large model structures. In recent years, researchers have acknowledged this issue and endeavored to reduce model size. Model lightweight techniques aim to decrease the number of parameters while maintaining satisfactory performance. In this paper, to enhance the model’s performance in lightweight scenarios, a novel approach to encompassing three key aspects is proposed: (1) utilizing LeakyReLU to involve more neurons in manifold representation; (2) employing large convolution for improved recognition of edges in lightweight models; (3) applying channel grouping and shuffling to maximize the model efficiency. Experimental results demonstrate that our proposed method achieves satisfactory outcomes on KITTI and Make3D benchmarks while having only 1.6M trainable parameters, representing a reduction of 27% compared with the previous smallest model, Lite-Mono-tiny, in monocular depth estimation.

Calibrated confidence learning for large-scale real-time crash and severity prediction

Real-time crash and severity prediction is a complex task, and there is no existing framework to predict crash likelihood and severity together. Creating such a framework poses numerous challenges, particularly not independent and identically distributed (non-IID) data, large model sizes with high computational costs, missing data, sensitivity vs. false alarm rate (FAR) trade-offs, and real-world deployment strategies. This study introduces a novel modeling technique to address these challenges and develops a deployable real-world framework. We used extensive real-time traffic and weather data to develop a crash likelihood prediction modeling prototype, leveraging our preliminary work of spatial ensemble modeling. Next, we equipped this spatial ensemble model with local model regularization to calibrate model confidence training. The investigated regularizations include weight decay, label smoothing and knowledge distillation. Furthermore, post-calibration on model outputs was conducted to improve severity rating identification. We tested the framework to predict crashes and severity in real-time, categorizing crashes into four levels. Results were compared with benchmark models, real-world deployment mechanisms were explained, traffic safety improvement potential and sustainability aspects of the study were discussed. Modeling results demonstrated excellent performance, and fatal, severe, minor and PDO crash severities were predicted with 91.7%, 83.3%, 85.6%, and 87.7% sensitivity, respectively, and with very low FAR. Similarly, the viability of our model to predict different severity levels for specific crash types, i.e., all-crash types, rear-end crashes, and sideswipe/angle crashes, were examined, and it showed excellent performance. Our modeling technique showed great potential for reducing model size, lowering computational costs, improving sensitivity, and, most importantly, reducing FAR. Finally, the deployment strategy for the proposed crash and severity prediction technique is discussed, and its potential to predict crashes with severity levels in real-time will make a substantial contribution to tailoring specific strategies to prevent crashes.

Multi‐armed bandit based online model selection for concept‐drift adaptation

AbstractEnsemble methods are among the most effective concept‐drift adaptation techniques due to their high learning performance and flexibility. However, they are computationally expensive and pose a challenge in applications involving high‐speed data streams. In this paper, we present a computationally efficient heterogeneous classifier ensemble entitled OMS‐MAB which uses online model selection for concept‐drift adaptation by posing it as a non‐stationary multi‐armed bandit (MAB) problem. We use a MAB to select a single adaptive learner within the ensemble for learning and prediction while systematically exploring promising alternatives. Each ensemble member is made drift resistant using explicit drift detection and is represented as an arm of the MAB. An exploration factor controls the trade‐off between predictive performance and computational resource requirements, eliminating the need to continuously train and evaluate all the ensemble members. A rigorous evaluation on 20 benchmark datasets and 9 algorithms indicates that the accuracy of OMS‐MAB is statistically at par with state‐of‐the‐art (SOTA) ensembles. Moreover, it offers a significant reduction in execution time and model size in comparison to several SOTA ensemble methods, making it a promising ensemble for resource constrained stream‐mining problems.

Sauron U-Net: Simple automated redundancy elimination in medical image segmentation via filter pruning

We introduce Sauron, a filter pruning method that eliminates redundant feature maps of convolutional neural networks (CNNs). Sauron optimizes, jointly with the loss function, a regularization term that promotes feature maps clustering at each convolutional layer by reducing the distance between feature maps. Sauron then eliminates the filters corresponding to the redundant feature maps by using automatically adjusted layer-specific thresholds. Unlike most filter pruning methods, Sauron requires minimal changes to typical neural network optimization because it prunes and optimizes CNNs jointly, which, in turn, accelerates the optimization over time. Moreover, unlike with other cluster-based approaches, the user does not need to specify the number of clusters in advance, a hyperparameter that is difficult to tune. We evaluated Sauron and five state-of-the-art filter pruning methods on four medical image segmentation tasks. This is an area where little attention has been paid to filter pruning, but where smaller CNN models are desirable for local deployment, mitigating privacy concerns associated with cloud-based solutions. Sauron was the only method that achieved a reduction in model size of over 90% without deteriorating substantially the performance. Sauron also achieved, overall, the fastest models at inference time in machines with and without GPUs. Finally, we show through experiments that the feature maps of models pruned with Sauron are highly interpretable, which is essential for medical image segmentation.

A bearing surface defect detection method based on multi-attention mechanism Yolov8

Surface defects in bearings not only affect the appearance but also impact the service life and performance. Therefore, it is imperative for bearing manufacturers to conduct quality inspections before bearings leave the factory. However, traditional visual inspection methods exhibit shortcomings such as high omission rates, insufficient feature fusion and oversized models when dealing with multiple target defects in bearings. To address these challenges, this paper proposes a surface defect detection method for bearings based on an improved Yolov8 algorithm (G-Yolov8). Firstly, a C3Ghost convolutional module based on the Ghost module is constructed in YOLOv8 to simplify model computational costs. Secondly, a global attention mechanism module is designed at the end of the backbone network to increase sensitivity to implicit small target area features and optimize feature extraction efficiency. Subsequently, a deep deformable convolution feature pyramid network is constructed by introducing the deformable convolutional networks version 2 (DCNv2) and the lightweight content-aware reassembly of features upsampling operator to reduce sampling information loss and improve the fusion of multi-scale target defects. Finally, different attention mechanisms are embedded in the detection network to construct a multi-attention detection head to replace the decoupled head, refining classification and localization tasks, reducing feature confusion, and improving the model’s detection accuracy. Experimental results demonstrate that the improved algorithm achieves a 3.5% increase in mean average precision on a self-made small-scale train bearing surface defect dataset, with a 17.3% reduction in model size. This improvement not only enhances accuracy but also addresses the requirement for lightweight deployment in subsequent stages.

ML3CNet: Non-local means-assisted automatic framework for lung cancer subtypes classification using histopathological images

Background and objective:Lung cancer (LC) has a high fatality rate that continuously affects human lives all over the world. Early detection of LC prolongs human life and helps to prevent the disease. Histopathological inspection is a common method to diagnose LC. Visual inspection of histopathological diagnosis necessitates more inspection time, and the decision depends on the subjective perception of clinicians. Usually, machine learning techniques mostly depend on traditional feature extraction which is labor-intensive and may not be appropriate for enormous data. In this work, a convolutional neural network (CNN)-based architecture is proposed for the more effective classification of lung tissue subtypes using histopathological images. Methods:Authors have utilized the first-time nonlocal mean (NLM) filter to suppress the effect of noise from histopathological images. NLM filter efficiently eliminated noise while preserving the edges of images. Then, the obtained denoised images are given as input to the proposed multi-headed lung cancer classification convolutional neural network (ML3CNet). Furthermore, the model quantization technique is utilized to reduce the size of the proposed model for the storage of the data. Reduction in model size requires less memory and speeds up data processing. Results:The effectiveness of the proposed model is compared with the other existing state-of-the-art methods. The proposed ML3CNet achieved an average classification accuracy of 99.72%, sensitivity of 99.66%, precision of 99.64%, specificity of 99.84%, F-1 score of 0.9965, and area under the curve of 0.9978. The quantized accuracy of 98.92% is attained by the proposed model. To validate the applicability of the proposed ML3CNet, it has also been tested on the colon cancer dataset. Conclusion:The findings reveal that the proposed approach can be beneficial to automatically classify LC subtypes that might assist healthcare workers in making decisions more precisely. The proposed model can be implemented on the hardware using Raspberry Pi for practical realization.

GDPs-YOLO: an improved YOLOv8s for coal gangue detection

ABSTRACT In response to the issues of low detection accuracy, slow speed, excessively large models, and difficult deployment in existing coal gangue recognition algorithms, a coal gangue target detection network based on an inverted residual structure is proposed. By conducting in-depth research on advanced edge computing networks, DPsP structure and DsP structure have been devised in this paper, while incorporating GhostModule to construct the GDPs-YOLO network within the YOLOv8s. The experimental results demonstrate the superior performance of the GDPs-YOLO network compared to both the baseline network and the control network. In comparison with the YOLOv5 series, YOLOv8 series, and four advanced edge computing networks, an increase in detection accuracy for coal gangue targets by a maximum of 2.5%, 1.6%, and 3.3% is observed, respectively. The model simultaneously exhibits enhanced speed and reduced model size, with a speed increase ranging from 12.5% to 86.85% compared to the control network. The compressibility of the model ranges from 24.07% to 94%. The inference latency measures approximately 2.8 ms, while the image processing speed reaches around 357 images per second, thereby satisfying the requirements for real-time detection.

A modified vision transformer architecture with scratch learning capabilities for effective fire detection

Fire is considered to be one of the major influencing factors that cause fatalities, property damage, and economic, and ecological disruption. To perform the early detection of fire using data from vision sensors and prevent or reduce the damage that fires cause, deep learning models have been widely adopted to overcome the limitations of conventional methods. However, mainstream convolutional neural network (CNN) models have limited generalisation abilities in unseen scenarios and struggle to obtain a good trade-off among the accuracy, inference speed, and model size. Currently, vision transformers (ViT) outperform conventional CNN models; however, they are computationally expensive and required more data for training. They provide a limited performance for small and medium-sized datasets, which are very common in the fire scene classification domain. In this work, we employ a novel ViT architecture by combining shifted patch tokenisation and local self-attention modules for efficient fire scene classification and enable the model to learn from scratch even on small and medium-sized datasets. Furthermore, to make the model suitable for real-time inferencing, we modify the transformer encoder and eventually achieve a reduced number of floating-point operations and a reduced model size. Additionally, in this work, a medium-scale fire dataset is developed that contains complex real-world scenarios. Our model is assessed on three benchmark and a self-created datasets using several evaluation metrics, including a novel cross-corpse evaluation metric, as well as a robustness evaluation metric. The experimental results indicate that our model achieved an overwhelmingly better performance compared to existing methods in terms of the accuracy and model complexity.