Lightweight Technique Research Articles

Early fire detection using wavelet based features

In recent years, millions of hectares of vegetation worldwide have been destroyed by wildfires and forest fires. Computer vision-based fire classification, which separates pixels from image or video datasets into fire and non-fire categories, has gained more attention recently due to technological innovations. Fire pixels in an image or video can be classified using either a deep learning strategy or a traditional machine learning approach. Deep learning algorithms could process enormous volumes of data, but their training model performance is constrained because they fail to consider the differences in complexity between training samples. Moreover, it is not obvious to train a deep learning model without a dedicated GPU unit.Similarly, deep learning techniques that have a scarcity of training data and insufficient features exhibit poor performance in intricate real-world fire situations. Consequently, to categorize fire and non-fire pixels from the processed photos from the publicly accessible datasets, Corsican and FLAME, as well as the aerial private dataset Firefront_Gestosa, the current study uses a lightweight technique based on SVM and a refined set of features.The present research implemented a novel framework for fire detection and classification from a variety of RGB and Infra-red images acquired during real missions, addressing the significant requirement for swift and accurate recognition of diverse types of flames, ranging from wildfires to industrial and domestic fires. The framework employs wavelet decomposition-based features, including wavelet length, standard deviation, variance, energy, and Shannon’s entropy, extracted through a sliding window sampling method within a machine learning approach. It should be noted that managing multidimensional data to train a model is difficult in machine learning applications. This issue is solved adopting a feature selection approach, which eliminates redundant or unnecessary data that affects the functionality of the model. Thus, to enhance model performance, feature selection using ranking algorithms based on theoretical mutual information is applied in combination to the Support Vector Machine (SVM) with Radial Basis Function (RBF) kernel.Extensive experiments demonstrate exceptional results, notably with Haar wavelets, achieving an impressive overall accuracy of 99.43% and remarkable performance across specificity, precision, recall, F-measure, and G-mean metrics. Thus, the present study showcases the potential of advanced image processing techniques to significantly advance fire detection and classification, thereby contributing to fire prevention, management, and research in various contexts.

Advancing cryptography: a novel hybrid cipher design merging Feistel and SPN structures

In the dynamic field of cryptography, lightweight ciphers play a pivotal role in overcoming resource constraints in modern applications. This paper introduces a lightweight cryptographic algorithm by seamlessly merging the proven characteristics of the Feistel cipher CLEFIA with the advanced substitution-permutation network (SPN) framework of RECTANGLE for key generation. The algorithm incorporates a specially optimized feather S-box, balancing efficiency and security in both CLEFIA and RECTANGLE components. The RECTANGLE key generation, vital for the proposed lightweight technique, enhances overall cryptographic security and efficiency. Meticulous consideration of resource limitations maintains the algorithm's lightweight nature, making it well-suited for applications with restricted computational resources. To validate the efficacy of the lightweight algorithm, extensive evaluation on encrypted data is conducted using National Institute of Standards and Technology (NIST) tools, known for assessing cryptographic algorithm quality. Results reveal a high degree of randomness, indicative of robust resistance against cryptographic attacks. This manuscript provides a comprehensive examination of the lightweight algorithm, emphasizing key attributes, security enhancements, and successful integration of the optimized feather S-box. Rigorous testing, particularly NIST tool-based randomness analysis, offers empirical evidence of the algorithm's resilience against attacks, establishing its suitability for secure data encryption in resource-limited environments.<p> </p>

PatchSkip: A lightweight technique for effectively alleviating over-smoothing in vision transformers

Recently vision transformers (ViTs) have encountered the over-smoothing problem, which reduces their capacity by mapping input patches into a similar latent representation. Existing methods introduce regularization terms to alleviate over-smoothing but often increase computational costs. To address this, this paper proposes PatchSkip, a novel and flexible dropout variant, alleviating the over-smoothing problem of ViTs in a lightweight manner. Specifically, PatchSkip draws inspiration from the fact that a similar over-smoothing problem in GNNs is primarily caused by static adjacent matrices leading to solitary message passing mode between nodes. PatchSkip constructs graphs with patch embeddings and analyzes the adjacent matrices in ViTs. By randomly selecting specific patch embeddings to bypass transformer blocks, PatchSkip is proved to generate various adjacent matrices and acts as a multi-mode message passing engine, providing diverse modes of message passing between patches. The effectiveness of PatchSkip in preventing over-smoothing is demonstrated through theoretical proofs and empirical visualizations. Furthermore, PatchSkip is evaluated on various datasets and backbones, showing significant performance improvements while reducing computational costs. For example, when trained on Tiny-ImageNet from scratch, PatchSkip improves the performance of the vanilla CrossViT by 3.85% while reducing computational costs by over 20%.

Shufflemono: Rethinking Lightweight Network for Self-Supervised Monocular Depth Estimation

Abstract Self-supervised monocular depth estimation has been widely applied in autonomous driving and automated guided vehicles. It offers the advantages of low cost and extended effective distance compared with alternative methods. However, like automated guided vehicles, devices with limited computing resources struggle to leverage state-of-the-art large model structures. In recent years, researchers have acknowledged this issue and endeavored to reduce model size. Model lightweight techniques aim to decrease the number of parameters while maintaining satisfactory performance. In this paper, to enhance the model’s performance in lightweight scenarios, a novel approach to encompassing three key aspects is proposed: (1) utilizing LeakyReLU to involve more neurons in manifold representation; (2) employing large convolution for improved recognition of edges in lightweight models; (3) applying channel grouping and shuffling to maximize the model efficiency. Experimental results demonstrate that our proposed method achieves satisfactory outcomes on KITTI and Make3D benchmarks while having only 1.6M trainable parameters, representing a reduction of 27% compared with the previous smallest model, Lite-Mono-tiny, in monocular depth estimation.

Computational kinematics of dance: distinguishing hip hop genres.

Dance plays a vital role in human societies across time and culture, with different communities having invented different systems for artistic expression through movement (genres). Differences between genres can be described by experts in words and movements, but these descriptions can only be appreciated by people with certain background abilities. Existing dance notation schemes could be applied to describe genre-differences, however they fall substantially short of being able to capture the important details of movement across a wide spectrum of genres. Our knowledge and practice around dance would benefit from a general, quantitative and human-understandable method of characterizing meaningful differences between aspects of any dance style; a computational kinematics of dance. Here we introduce and apply a novel system for encoding bodily movement as 17 macroscopic, interpretable features, such as expandedness of the body or the frequency of sharp movements. We use this encoding to analyze Hip Hop Dance genres, in part by building a low-cost machine-learning classifier that distinguishes genre with high accuracy. Our study relies on an open dataset (AIST++) of pose-sequences from dancers instructed to perform one of ten Hip Hop genres, such as Breakdance, Popping, or Krump. For comparison we evaluate moderately experienced human observers at discerning these sequence's genres from movements alone (38% where chance = 10%). The performance of a baseline, Ridge classifier model was fair (48%) and that of the model resulting from our automated machine learning pipeline was strong (76%). This indicates that the selected features represent important dimensions of movement for the expression of the attitudes, stories, and aesthetic values manifested in these dance forms. Our study offers a new window into significant relations of similarity and difference between the genres studied. Given the rich, complex, and culturally shaped nature of these genres, the interpretability of our features, and the lightweight techniques used, our approach has significant potential for generalization to other movement domains and movement-related applications.

A lightweight performance proxy for deep‐learning model training on Amazon SageMaker

SummaryCloud computing has become popular for training deep‐learning (DL) models, avoiding the costs of acquiring and maintaining on‐premise systems. SageMaker is a cloud service that automates the execution of DL workloads. Its features include automatic hyperparameter optimization and use of spot instances. Nonetheless, it does not assist in selecting the right instance type for a workload. In public clouds, rent price depends on the configuration of the chosen instance type. Advanced and faster instances are typically more expensive, but not always the best choice. To select the optimal instance type, users must compare the workload's relative performance (and hence cost) on several candidates. Building on the execution profiles of multiple DL applications, we model the performance and cost of training DL applications on SageMaker and propose a lightweight technique to estimate these at low temporal and monetary cost. This method is a performance proxy that can be used to replace more expensive performance measurement procedures. So, it could speed up any technique that relies on such measurements. We show how it can help cloud customers seeking suitable instance types to train DL models, and that it can accurately predict the performance of different instance types when training these models on SageMaker.

Dias: Dynamic Rewriting of Pandas Code

In recent years, dataframe libraries, such as pandas have exploded in popularity. Due to their flexibility, they are increasingly used in ad-hoc exploratory data analysis (EDA) workloads. These workloads are diverse, including custom functions which can span libraries or be written in pure Python. The majority of systems available to accelerate EDA workloads focus on bulk-parallel workloads, which contain vastly different computational patterns, typically within a single library. As a result, they can introduce excessive overheads for ad-hoc EDA workloads due to their expensive optimization techniques. Instead, we identify source-to-source, external program rewriting as a lightweight technique which can optimize across representations, and offer substantial speedups while also avoiding slowdowns. We implemented Dias, which rewrites notebook cells to be more efficient for ad-hoc EDA workloads. We develop techniques for efficient rewrites in Dias, including checking the preconditions under which rewrites are correct, dynamically, at fine-grained program points. We show that Dias can rewrite individual cells to be 57× faster compared to pandas and 1909× faster compared to optimized systems such as modin. Furthermore, Dias can accelerate whole notebooks by up to 3.6× compared to pandas and 27.1× compared to modin.

A Lightweight Method for Detecting and Correcting Errors in Low-Frequency Measurements for In-Orbit Demonstrators.

In the pursuit of enhancing the technological maturity of innovative magnetic sensing techniques, opportunities presented by in-orbit platforms (IOD/IOV experiments) provide a means to evaluate their in-flight capabilities. The Magnetic Experiments for the Laser Interferometer Space Antenna (MELISA) represent a set of in-flight demonstrators designed to characterize the low-frequency noise performance of a magnetic measurement system within a challenging space environment. In Low Earth Orbit (LEO) satellites, electronic circuits are exposed to high levels of radiation coming from energetic particles trapped by the Earth's magnetic field, solar flares, and galactic cosmic rays. A significant effect is the accidental bit-flipping in memory registers. This work presents an analysis of memory data redundancy resources using auxiliary second flash memory and exposes recovery options to retain critical data utilizing a duplicated data structure. A new and lightweight technique, CCM (Cross-Checking and Mirroring), is proposed to verify the proper performance of these techniques. Four alternative algorithms included in the original version of the MELISA software (Version v0.0) are presented. All the versions have been validated and evaluated according to various merit indicators. The evaluations showed similar performances for the proposed techniques, and they are valid for situations in which the flash memory suffers from more than one bit-flip. The overhead due to the introduction of additional instructions to the main code is negligible, even in the target experiment based on an 8-bit microcontroller.

Lightweight Techniques to Improve Generalization and Robustness of U-Net Based Networks for Pulmonary Lobe Segmentation.

Lung lobe segmentation in chest CT is relevant to a wide range of clinical applications. However, existing segmentation pipelines often exhibit vulnerabilities and performance degradations when applied to external datasets. This is usually attributed to the size of the available dataset or model. We show that it is possible to enhance generalizability without huge resources by carefully curating the dataset and combining machine learning with medical expertise. Multiple machine learning techniques (self-supervision (SSL), attention (A), and data augmentation (DA)) are used to train a fast and fully-automated lung lobe segmentation model based on 2D U-Net. Our study involved evaluating these techniques on a diverse dataset collected under the RACOON project, encompassing 100 CT chest scans from patients with bacterial, viral, or SARS-CoV2 infections. We compare our model to a baseline U-Net trained on the same dataset. Our approach significantly improved segmentation accuracy (Dice score of 92.8% vs. 82.3%, p < 0.001). Moreover, our model achieved state-of-the-art performance (Dice score of 92.8% vs. 90.8% for the literature's state-of-the-art, p = 0.102) with reduced training examples (69 vs. 231 CT Scans). Among the techniques, data augmentation with expert knowledge displayed the most significant impact, enhancing the Dice score by +0.056. Notably, these enhancements are not limited to lobe segmentation but can be seamlessly integrated into various medical imaging segmentation tasks, demonstrating their versatility and potential for broader applications.

Extracting laboratory test information from paper-based reports

BackgroundIn the healthcare domain today, despite the substantial adoption of electronic health information systems, a significant proportion of medical reports still exist in paper-based formats. As a result, there is a significant demand for the digitization of information from these paper-based reports. However, the digitization of paper-based laboratory reports into a structured data format can be challenging due to their non-standard layouts, which includes various data types such as text, numeric values, reference ranges, and units. Therefore, it is crucial to develop a highly scalable and lightweight technique that can effectively identify and extract information from laboratory test reports and convert them into a structured data format for downstream tasks.MethodsWe developed an end-to-end Natural Language Processing (NLP)-based pipeline for extracting information from paper-based laboratory test reports. Our pipeline consists of two main modules: an optical character recognition (OCR) module and an information extraction (IE) module. The OCR module is applied to locate and identify text from scanned laboratory test reports using state-of-the-art OCR algorithms. The IE module is then used to extract meaningful information from the OCR results to form digitalized tables of the test reports. The IE module consists of five sub-modules, which are time detection, headline position, line normalization, Named Entity Recognition (NER) with a Conditional Random Fields (CRF)-based method, and step detection for multi-column. Finally, we evaluated the performance of the proposed pipeline on 153 laboratory test reports collected from Peking University First Hospital (PKU1).ResultsIn the OCR module, we evaluate the accuracy of text detection and recognition results at three different levels and achieved an averaged accuracy of 0.93. In the IE module, we extracted four laboratory test entities, including test item name, test result, test unit, and reference value range. The overall F1 score is 0.86 on the 153 laboratory test reports collected from PKU1. With a single CPU, the average inference time of each report is only 0.78 s.ConclusionIn this study, we developed a practical lightweight pipeline to digitalize and extract information from paper-based laboratory test reports in diverse types and with different layouts that can be adopted in real clinical environments with the lowest possible computing resources requirements. The high evaluation performance on the real-world hospital dataset validated the feasibility of the proposed pipeline.

Multi-Plant Disease Identification Based on Lightweight ResNet18 Model

Deep-learning-based methods for plant disease recognition pose challenges due to their high number of network parameters, extensive computational requirements, and overall complexity. To address this issue, we propose an improved residual-network-based multi-plant disease recognition method that combines the characteristics of plant diseases. Our approach introduces a lightweight technique called maximum grouping convolution to the ResNet18 model. We made three enhancements to adapt this method to the characteristics of plant diseases and ultimately reduced the convolution kernel requirements, resulting in the final model, Model_Lite. The experimental dataset comprises 20 types of plant diseases, including 13 selected from the publicly available Plant Village dataset and seven self-constructed images of apple leaves with complex backgrounds containing disease symptoms. The experimental results demonstrated that our improved network model, Model_Lite, contains only about 1/344th of the parameters and requires 1/35th of the computational effort compared to the original ResNet18 model, with a marginal decrease in the average accuracy of only 0.34%. Comparing Model_Lite with MobileNet, ShuffleNet, SqueezeNet, and GhostNet, our proposed Model_Lite model achieved a superior average recognition accuracy while maintaining a much smaller number of parameters and computational requirements than the above models. Thus, the Model_Lite model holds significant potential for widespread application in plant disease recognition and can serve as a valuable reference for future research on lightweight network model design.

Towards Lightweight Deep Classification for Low-Resolution Synthetic Aperture Radar (SAR) Images: An Empirical Study

Numerous works have explored deep models for the classification of high-resolution natural images. However, limited investigation has been made into a deep classification for low-resolution synthetic aperture radar (SAR) images, which is a challenging yet important task in the field of remote sensing. Existing work adopted ROC–VGG, which has a huge amount of parameters, thus limiting its application in practical deployment. It remains unclear whether the techniques developed in high-resolution natural images to make the model lightweight can be effective for low-resolution SAR images. Therefore, with prior work as the baseline, this work conducts an empirical study, testing three popular lightweight techniques: (1) channel attention module; (2) spatial attention module; (3) multi-stream head. Our empirical results show that these lightweight techniques in the high-resolution natural image domain can also be effective in the low-resolution SAR domain. We reduce the parameters from 9.2M to 0.17M while improving the performance from 94.8% to 96.8%.

Performance analysis of electric motors, comparing lightweight techniques and cooling methods: A review

The paper gives a comprehensive review of lightweight materials/design of permanent-magnet-synchronous-motors and different temperature-reducing (cooling) techniques. The paper discusses and compares various simulations and performance analyses of different practically implemented solutions. Temperature and performance analyses for cooling systems using phase-changing materials, cooling jackets, and other coolants are investigated, and the use of lightweight materials and composites is compared. A discussion on numerical methods and results (finite element method analysis), in order to get a more-precise thermal mapping model for different machine structures is presented. Finally, the best temperature-reducing-cooling systems methods along with the good lightweight designs which increase the overall efficiency of the motor are discussed and commented on.

Unsupervised domain adaptation via reliable pseudolabeling based memory module and dynamic distance threshold learning

An important challenge in unsupervised domain adaptation (UDA) is how to capture domain-specific discriminative information in the target domain data. For the labels in the target domain are unavailable, some methods uncover the intrinsic discriminative information by clustering the target domain features in the embedding space. However, most of existing works only consider filtering reliable target domain samples by setting a fixed distance threshold, ignoring the potential intraclass and interclass relationships and cannot set appropriate distance thresholds for different transfer tasks and different classes. To this end, we propose a new pseudolabeling framework named the dynamic distance threshold learning framework (DDT) for UDA. In our framework, we construct distance memory module to encode intraclass and interclass distribution information, and select reliable target domain samples satisfying intraclass compactness and interclass independence via dynamic intraclass and interclass distance thresholds. Specifically, we define class centroid memory modules, intraclass and interclass distance memory modules in the embedding space. Then, we retrieve the intraclass compactness distance threshold and interclass independence distance threshold from the distance memory based on the class and classpair information of target domain sample. Finally, we use reliable target domain samples to update the class centroid and the distance memory module, so that the distance memory module is dynamically updated following the model training. Benefiting from DDT, the UDA model can be updated with reliable target domain samples for training, and the joint distribution can be further aligned at the category level. As a lightweight technique, DDT constructs class center and distance memory modules to transfer the intraclass and interclass distribution relationships from the source domain to the target domain, and sets dynamic intraclass and interclass distances for different classes in different transfer tasks. Without any additional training parameters, the framework can be easily plugged into various domain adaptation paradigms. Experiments on benchmark datasets show that our method can filter the noise in pseudolabels and confers a competitive advantage compared with existing state-of-the-art domain adaptation methods.

A privacy preservation data collection and access control using entropy-based conic curve

Because significantly complex crypto procedures such as holomorphic encryption are robotically applied, despite the fact that consumer gadgets under our software circumstances are not, computational overhead is outrageously high. Simply hiding customers with the aid of nameless communications to act to protect the server and adversaries from linking suggestions made with the aid of the same customer makes the traditional method, which computes with the aid of any server based on the amount of provided services, impossible, and customers with charge features widely publicised with the aid of the server cause additional security concerns, impossible. To overcome the above existing drawbacks, this research study presents a Privacy Preservation Data Collection and Access Control Using Entropy-Based Conic Curve. To safeguard the identity of clients and their requests, EBCC employs a unique group signature technic and an asymmetric cryptosystem. First, we ought to implement our EBCC method for data acquisition while maintaining privacy. Second, we consider looking at the properties of secure multiparty computation. EBCC employs lightweight techniques in encryption, aggregation, and decryption, resulting in little computation and communication overhead. Security research suggests that the EBCC is safe, can withstand collision attacks, and can conceal consumer distribution, which is required for fair balance checks in credit card payments. Finally, the results are analysed to illustrate the proposed method performance in addition to the more traditional ABC, AHRPA, ECC, and RSA methods. The proposed work should be implemented in JAVA.

PolypSeg+: A Lightweight Context-Aware Network for Real-Time Polyp Segmentation.

Automatic polyp segmentation from colonoscopy videos is a prerequisite for the development of a computer-assisted colon cancer examination and diagnosis system. However, it remains a very challenging task owing to the large variation of polyps, the low contrast between polyps and background, and the blurring boundaries of polyps. More importantly, real-time performance is a necessity of this task, as it is anticipated that the segmented results can be immediately presented to the doctor during the colonoscopy intervention for his/her prompt decision and action. It is difficult to develop a model with powerful representation capability, yielding satisfactory segmentation results and, simultaneously, maintaining real-time performance. In this article, we present a novel lightweight context-aware network, namely, PolypSeg+, attempting to capture distinguishable features of polyps without increasing network complexity and sacrificing time performance. To achieve this, a set of novel lightweight techniques is developed and integrated into the proposed PolypSeg+, including an adaptive scale context (ASC) module equipped with a lightweight attention mechanism to tackle the large-scale variation of polyps, an efficient global context (EGC) module to promote the fusion of low-level and high-level features by excluding background noise and preserving boundary details, and a lightweight feature pyramid fusion (FPF) module to further refine the features extracted from the ASC and EGC. We extensively evaluate the proposed PolypSeg+ on two famous public available datasets for the polyp segmentation task: 1) Kvasir-SEG and 2) CVC-Endoscenestill. The experimental results demonstrate that our PolypSeg+ consistently outperforms other state-of-the-art networks by achieving better segmentation accuracy in much less running time. The code is available at https://github.com/szu-zzb/polypsegplus.

Securely Outsourcing Neural Network Inference to the Cloud With Lightweight Techniques

Neural network (NN) inference services enrich many applications, like image classification, object recognition, facial verification, and more. These NN inference services are increasingly becoming an essential offering from cloud computing providers, where end-users’ data are offloaded to the cloud for inference under a customized model. However, current cloud-based inference services operate on clear inputs and NN models, raising paramount privacy concerns. Individual user data may contain private information that should always remain confidential. Meanwhile, the NN model is deemed proprietary to the model owner as model training requires substantial resources. In this article, we present, tailor, and evaluate <monospace>Sonic</monospace> , a lightweight secure NN inference service delegated in the cloud. <monospace>Sonic</monospace> leverages the cloud computing paradigm to fully outsource the secure inference, freeing end devices and model owners from being actively online for assistance. <monospace>Sonic</monospace> guards both user input and model privacy along the whole service flow. We design a series of secure and efficient NN layer functions purely using lightweight cryptographic primitives. Extensive evaluations demonstrate that <monospace>Sonic</monospace> achieves up to <inline-formula><tex-math notation="LaTeX">$60\times$</tex-math></inline-formula> bandwidth saving in online inference compared to prior art.

A Light-Weight Technique to Detect GPS Spoofing Using Attenuated Signal Envelopes

A Light-Weight Technique to Detect GPS Spoofing Using Attenuated Signal Envelopes

Robust Hierarchical Federated Learning with Anomaly Detection in Cloud-Edge-End Cooperation Networks

Federated learning (FL) enables devices to collaborate on machine learning (ML) model training with distributed data while preserving privacy. However, the traditional FL is inefficient and costly in cloud–edge–end cooperation networks since the adopted classical client-server communication framework fails to consider the real network structure. Moreover, malicious attackers and malfunctioning clients may be implied in all participators to exert adverse impacts as abnormal behaviours on the FL process. To address the above challenges, we leverage cloud–edge–end cooperation to propose a robust hierarchical federated learning (R-HFL) framework to enhance inherent system resistance to abnormal behaviours while improving communication efficiency in practical networks and keeping the advantages of the traditional FL. Specifically, we introduce a hierarchical cloud–edge–end collaboration-based FL framework to reduce communication costs. For the framework, we design a detection mechanism as partial cosine similarity (PCS) to filter adverse clients to improve performance, where the proposed lightweight technique has high computation parallelization. Besides, we theoretically discuss the influence of the proposed PCS on the convergence and stabilization of FL. Finally, the experimental results show that the proposed R-HFL always outperforms baselines in general cases under malicious attacks, which further shows the effectiveness of our scheme.

Hybrid algorithms based on historical accuracy for forecasting particulate matter concentrations

Air pollution has become one of the most significant problems impacting human health. Particulate matter (PM) 2.5 is usually used as an identifier of the intensity of the pollution. The PM2.5 forecasting is essential and gainful for reducing health risks. The efficient model for forecasting PM2.5 concentration can be used in determining the period of outdoor activities, thereby reducing the impact on health. In addition, the government sector can use the forecasting model as a tool for laying down measures a burning control. In this work, the hybrid forecasting algorithms for improving accuracy are presented. The hybrid forecasting algorithms combine neural network models with historical predictive data for improving the accuracy of forecasting. The experimental results show that the proposed algorithms can reduce the mean absolute error and root mean square error of forecasting at 36% and 45%. Therefore, the proposed algorithms are not only effectively used to forecast the PM2.5 concentrations but also apply the lightweight technique based on historical accuracy to forecast other complex problems efficiently.