Resource-constrained Edge Research Articles

Resource-constrained edge-based deep learning for real-time person-identification using foot-pad

To build a smart home that assists people with disabilities, we urgently need a non-invasive user identification system capable of recognizing family members in real-time and being easy to use. Existing systems face issues such as privacy concerns from cameras, inconvenience from needing physical contact with sensors, and the requirement to always carry a specific device. Additionally, these systems typically depend on edge nodes to collect data and then transfer it to cloud servers for high-performance inference. This dependency leads to network delays, hindering real-time service, and introducing security issues. To address these requirements, this paper presents a real-time non-invasive user identification system that recognizes users as they step on a foot pad. This study introduces an edge node designed to measure real-time foot pressure distribution data, along with a preprocessing system for data generalization. Additionally, we propose a system that performs real-time user inference using only resource-constrained edge nodes to overcome the challenges of cloud-based systems, including addressing security issues without specific protocols. To achieve this, we optimized various deep learning-based user identification models to be executable on edge nodes and then compared their performance. As a result, using a ResNet18 model through pruning and post quantization training with integer, we achieved inference within 1.5 s with 85% accuracy. Compared to the worst-performing AlexNet, the ResNet18 model shows a substantial reduction in model size by approximately 33%, a decrease in memory usage by about 80%, and a significant increase in inference speed by over tenfold.

Dual-Grained Lightweight Strategy.

Removing redundant parameters and computations before the model training has attracted a great interest as it can effectively reduce the storage space of the model, speed up the training and inference of the model, and save energy consumption during the running of the model. In addition, the simplification of deep neural network models can enable high-performance network models to be deployed to resource-constrained edge devices, thus promoting the development of the intelligent world. However, current pruning at initialization methods exhibit poor performance at extreme sparsity. In order to improve the performance of the model under extreme sparsity, this paper proposes a dual-grained lightweight strategy-TEDEPR. This is the first time that TEDEPR has used tensor theory in the pruning at initialization method to optimize the structure of a sparse sub-network model and improve its performance. Specifically, first, at the coarse-grained level, we represent the weight matrix or weight tensor of the model as a low-rank tensor decomposition form and use multi-step chain operations to enhance the feature extraction capability of the base module to construct a low-rank compact network model. Second, unimportant weights are pruned at a fine-grained level based on the trainability of the weights in the low-rank model before the training of the model, resulting in the final compressed model. To evaluate the superiority of TEDEPR, we conducted extensive experiments on MNIST, UCF11, CIFAR-10, CIFAR-100, Tiny-ImageNet and ImageNet datasets with LeNet, LSTM, VGGNet, ResNet and Transformer architectures, and compared with state-of-the-art methods. The experimental results show that TEDEPR has higher accuracy, faster training and inference, and less storage space than other pruning at initialization methods under extreme sparsity.

RT-Cabi: an Internet of Things based framework for anomaly behavior detection with data correction through edge collaboration and dynamic feature fusion

The rapid advancement of Internet of Things (IoT) technologies brings forth new security challenges, particularly in anomaly behavior detection in traffic flow. To address these challenges, this study introduces RT-Cabi (Real-Time Cyber-Intelligence Behavioral Anomaly Identifier), an innovative framework for IoT traffic anomaly detection that leverages edge computing to enhance the data processing and analysis capabilities, thereby improving the accuracy and efficiency of anomaly detection. RT-Cabi incorporates an adaptive edge collaboration mechanism, dynamic feature fusion and selection techniques, and optimized lightweight convolutional neural network (CNN) frameworks to address the limitations of traditional models in resource-constrained edge devices. Experiments conducted on two public datasets, Edge-IIoT and UNSW_NB15, demonstrate that RT-Cabi achieves a detection accuracy of 98.45% and 90.94%, respectively, significantly outperforming existing methods. These contributions not only validate the effectiveness of the RT-Cabi model in identifying anomalous behaviors in IoT traffic but also offer new perspectives and technological pathways for future research in IoT security.

LFIR-YOLO: Lightweight Model for Infrared Vehicle and Pedestrian Detection

The complexity of urban road scenes at night and the inadequacy of visible light imaging in such conditions pose significant challenges. To address the issues of insufficient color information, texture detail, and low spatial resolution in infrared imagery, we propose an enhanced infrared detection model called LFIR-YOLO, which is built upon the YOLOv8 architecture. The primary goal is to improve the accuracy of infrared target detection in nighttime traffic scenarios while meeting practical deployment requirements. First, to address challenges such as limited contrast and occlusion noise in infrared images, the C2f module in the high-level backbone network is augmented with a Dilation-wise Residual (DWR) module, incorporating multi-scale infrared contextual information to enhance feature extraction capabilities. Secondly, at the neck of the network, a Content-guided Attention (CGA) mechanism is applied to fuse features and re-modulate both initial and advanced features, catering to the low signal-to-noise ratio and sparse detail features characteristic of infrared images. Third, a shared convolution strategy is employed in the detection head, replacing the decoupled head strategy and utilizing shared Detail Enhancement Convolution (DEConv) and Group Norm (GN) operations to achieve lightweight yet precise improvements. Finally, loss functions, PIoU v2 and Adaptive Threshold Focal Loss (ATFL), are integrated into the model to better decouple infrared targets from the background and to enhance convergence speed. The experimental results on the FLIR and multispectral datasets show that the proposed LFIR-YOLO model achieves an improvement in detection accuracy of 4.3% and 2.6%, respectively, compared to the YOLOv8 model. Furthermore, the model demonstrates a reduction in parameters and computational complexity by 15.5% and 34%, respectively, enhancing its suitability for real-time deployment on resource-constrained edge devices.

Real-Time Detection and Localization of Weeds in Dictamnus dasycarpus Fields for Laser-Based Weeding Control

Traditional Chinese medicinal herbs have strict environmental requirements and are highly susceptible to weed damage, while conventional herbicides can adversely affect their quality. Laser weeding has emerged as an effective method for managing weeds in precious medicinal herbs. This technique allows for precise weed removal without chemical residue and protects the surrounding ecosystem. To maximize the effectiveness of this technology, accurate detection and localization of weeds in the medicinal herb fields are crucial. This paper studied seven species of weeds in the field of Dictamnus dasycarpus, a traditional Chinese medicinal herb. We propose a lightweight YOLO-Riny weed-detection algorithm and develop a YOLO-Riny-ByteTrack Multiple Object Tracking method by combining it with the ByteTrack algorithm. This approach enables accurate detection and localization of weeds in medicinal fields. The YOLO-Riny weed-detection algorithm is based on the YOLOv7-tiny network, which utilizes the FasterNet lightweight structure as the backbone, incorporates a lightweight upsampling operator, and adds structure reparameterization to the detection network for precise and rapid weed detection. The YOLO-Riny-ByteTrack Multiple Object Tracking method provides quick and accurate feedback on weed identification and location, reducing redundant weeding and saving on laser weeding costs. The experimental results indicate that (1) YOLO-Riny improves detection accuracy for Digitaria sanguinalis and Acalypha australis, ultimately amounting to 5.4% and 10%, respectively, compared to the original network. It also diminishes the model size by 2 MB and inference time by 10 ms, making it more suitable for resource-constrained edge devices. (2) YOLO-Riny-ByteTrack enhances Multiple Object Tracking accuracy by 3%, reduces ID switching by 14 times, and improves overall tracking accuracy by 3.4%. The proposed weed-detection and localization method for Dictamnus dasycarpus offers fast detection speed, high localization accuracy, and stable tracking, supporting the implementation of laser weeding during the seedling stage of Dictamnus dasycarpus.

Task-Level Customized Pruning for Image Classification on Edge Devices

Convolutional neural networks (CNNs) are widely utilized in image classification. Nevertheless, CNNs typically require substantial computational resources, posing challenges for deployment on resource-constrained edge devices and limiting the spread of AI-driven applications. While various pruning approaches have been proposed to mitigate this issue, they often overlook a critical fact that edge devices are typically tasked with handling only a subset of classes rather than the entire set. Moreover, the specific combinations of subcategories that each device must discern vary, highlighting the need for fine-grained task-specific adjustments. Unfortunately, these oversights result in pruned models that still contain unnecessary category redundancies, thereby impeding the potential for further model optimization and lightweight design. To bridge this gap, we propose a task-level customized pruning (TLCP) method via utilizing task-level information, i.e., class combination information relevant to edge devices. Specifically, TLCP first introduces channel control gates to assess the importance of each convolutional channel for individual classes. These class-level control gates are then aggregated through linear combinations, resulting in a pruned model customized to the specific tasks of edge devices. Experiments on various customized tasks demonstrate that TLCP can significantly reduce the number of parameters, by up to 33.9% on CIFAR-10 and 14.0% on CIFAR-100, compared to other baseline methods, while maintaining almost the same inference accuracy.

Binarized ResNet: Enabling Robust Automatic Modulation Classification at the Resource-Constrained Edge

Binarized ResNet: Enabling Robust Automatic Modulation Classification at the Resource-Constrained Edge

Enabling efficient and low-effort decentralized federated learning with the EdgeFL framework

Context:Federated Learning (FL) has gained prominence as a solution for preserving data privacy in machine learning applications. However, existing FL frameworks pose challenges for software engineers due to implementation complexity, limited customization options, and scalability issues. These limitations prevent the practical deployment of FL, especially in dynamic and resource-constrained edge environments, preventing its widespread adoption. Objective:To address these challenges, we propose EdgeFL, an efficient and low-effort FL framework designed to overcome centralized aggregation, implementation complexity and scalability limitations. EdgeFL applies a decentralized architecture that eliminates reliance on a central server by enabling direct model training and aggregation among edge nodes, which enhances fault tolerance and adaptability to diverse edge environments. Methods:We conducted experiments and a case study to demonstrate the effectiveness of EdgeFL. Our approach focuses on reducing weight update latency and facilitating faster model evolution on edge devices. Results:Our findings indicate that EdgeFL outperforms existing FL frameworks in terms of learning efficiency and performance. By enabling quicker model evolution on edge devices, EdgeFL enhances overall efficiency and responsiveness to changing data patterns. Conclusion:EdgeFL offers a solution for software engineers and companies seeking the benefits of FL, while effectively overcoming the challenges and privacy concerns associated with traditional FL frameworks. Its decentralized approach, simplified implementation, combined with enhanced customization and fault tolerance, make it suitable for diverse applications and industries.

Robust filter pruning guided by deep frequency-features for edge intelligence

With the popularity of mobile applications, smart terminals require the deployment of lightweight deep neural networks (DNNs). However, those lightweight DNNs deployed on resource-constrained edge devices, have serious security vulnerabilities to adversarial examples. Pruning methods have been widely used to obtain lightweight DNNs, which mainly focus on improving classification accuracy, but ignore robustness. To this end, we present a Fourier space-based robust pruning algorithm (FSRP). We find that a robust network pays more attention to the low-frequency region of the adversarial examples’ feature maps, but ignores the high-frequency region. According to this finding, we design a filter robust indicator, a ratio of low-frequency components to high-frequency components of the feature map, to guide the pruning process. With this new robust pruning criterion, we adopt a strategy of local pruning that removes the filters with low robustness layer by layer in the model. Extensive experiments on CIFAR10/CIFAR100 and ImageNet show that the robust pruning accuracy is significantly improved under the FSRP robust pruning criterion. On CIFAR10, a pruning ratio of 90% on the VGG16 network still shows a 14.1% improvement in robust accuracy under auto attacks (AA).

Reinforcement Learning for Real-Time Federated Learning for Resource-Constrained Edge Cluster

Reinforcement Learning for Real-Time Federated Learning for Resource-Constrained Edge Cluster

A Comprehensive IoT edge based smart irrigation system for tomato cultivation

Agriculture industry is the primary engine for a country's economic development. Growing crops using minimum irrigation water is a major challenge for farmers. In conventional farming, crops may be affected by various diseases due to inadequate irrigation scheduling. Recent proposals have suggested using Edge-IoT, AI, and distributed computing to accelerate the inference procedure utilized in smart irrigation applications. The use of resource-constrained edge servers and edge devices used to deliver smart agriculture applications can cause latency-sensitive workloads to interfere with one another. To address this issue, we design a long-range (LoRa) edge IoT computing-based sustainable and customized smart irrigation framework to capture the real-time data of tomato plants. This helps in automatic underground drip irrigation scheduling. This also predicts total water demand and usage, and measure plant growth status. The edge-IoT cloud data transmission control and optimization has been enforced using Smart irrigation data optimization and robust transmission (SIDORT) Message Queuing Telemetry Transport (MQTT) system. We develop a hybrid algorithm named Linked least traversal (LLT) for machine-to-machine communication (M2M). Also, a Reinforcement learning (RL) based Optimal Soil Wetness Closeness Policy (OSWCP) for irrigation scheduling has been proposed. The performance of the proposed smart irrigation models has been validated through extensive experiments using real-time data in which OSWCP performance has been measured at a 97.88 % accuracy rate. Additionally, a comparison of our proposed architecture has been accomplished by resolving the existing smart irrigation system challenges.

EvolutionViT: Multi-objective evolutionary vision transformer pruning under resource constraints

Vision Transformer (ViT) has emerged as a pivotal model for a variety of visual tasks, surpassing convolutional neural networks by a substantial margin. However, the performance of ViT is seriously impaired by intensive computational and storage costs requirements, posing significant barriers for real-world applications or deployment on resource-constrained edge devices. To address this limitation, compressing the ViT to accelerate its inference at no appreciable degradation of vision performance has attracted widespread attention. Although there are some studies on accelerating ViT, they seldom consider resource constraints and multi-criteria decision making in the process. This article formulates ViT pruning as a large-scale constrained multi-objective optimization problem, and proposes a patch pruning framework for accelerating ViT, called EvolutionViT, based on the developed multi-objective optimization model. EvolutionViT can effectively tradeoff between computational cost and performance under resource constraints, automatically searching for solutions while optimizing two conflicting objectives. In particular, exploiting the knee solution and boundary solutions to directly guide the entire evolutionary process, EvolutionViT can efficiently identify a knee solution that satisfies the resource constraints, which in turn avoids the manual search for a good trade-off. To verify and evaluate our proposed method, we compare EvolutionViT with a number of representative ViT models on the ImageNet dataset. The comprehensive simulation results show that the proposed EvolutionViT demonstrates a competitive advantage compared to peers, with significantly reduced computational expense at the cost of slightly degraded performance.

Revealing CNN Architectures via Side-Channel Analysis in Dataflow-based Inference Accelerators

Convolutional Neural Networks (CNNs) are widely used in various domains, including image recognition, medical diagnosis and autonomous driving. Recent advances in dataflow-based CNN accelerators have enabled CNN inference in resource-constrained edge devices. These dataflow accelerators utilize inherent data reuse of convolution layers to process CNN models efficiently. Concealing the architecture of CNN models is critical for privacy and security. This paper evaluates memory-based side-channel information to recover CNN architectures from dataflow-based CNN inference accelerators. The proposed attack exploits spatial and temporal data reuse of the dataflow mapping on CNN accelerators and architectural hints to recover the structure of CNN models. Experimental results demonstrate that our proposed side-channel attack can recover the structures of popular CNN models, namely Lenet, Alexnet, VGGnet16, and YOLOv2.

MLino bench: A comprehensive benchmarking tool for evaluating ML models on edge devices

In today’s rapidly evolving technological landscape, Machine Learning (ML) has become an integral part of our daily lives, ranging from recommendation systems to advanced medical diagnostics and autonomous vehicles. As ML continues to advance, its applications extend beyond conventional boundaries. With the continuous refinement of the models and frameworks, the possibilities for leveraging these technologies into devices that traditionally lacked any form of computational autonomy are ever-expanding. This shift towards embedding ML capabilities directly into edge devices brings new challenges due to the stringent limitations these devices have in terms of memory, power consumption, and cost. The ML models implemented on such devices must find an equilibrium between memory footprint and performance, attaining a classification time that fulfills real-time demands and maintains a similar level of accuracy as the desktop version. Without automated assistance in managing these considerations, the complexity of evaluating multiple models can lead to suboptimal decisions.In this paper, we introduce MLino Bench, an open-source benchmarking tool tailored for assessing lightweight ML models on edge devices with limited resources and capabilities. The tool accommodates various models, frameworks, and platforms, presenting a meticulous design that enables a comprehensive evaluation directly on the target device. It encompasses crucial metrics such as on-target accuracy, classification time, and model size, providing a versatile framework that assists practitioners in decision-making when deploying models to such devices.The tool employs a fully streamlined benchmark flow involving training the ML model in a high-level interpreted language, porting, compiling, flashing, and finally benchmarking on the actual target. Our experimental evaluation of the tool highlights its flexibility in assessing multiple ML models across different model hyperparameters, frameworks, datasets, and embedded platforms. Furthermore, a distinctive advantage compared to state-of-the-art ML benchmarking tools is the inclusion of classical ML models, including Random Forests, Decision Trees, Support Vector Machines, Naive Bayes, and more. This sets our tool apart from others that predominantly emphasize only neural network models. Due to this inclusive approach, our tool facilitates the evaluation of ML models across a broad spectrum of devices, ranging from resource-constrained edge devices to those with medium and advanced computational capabilities.

A hybrid network TEdgeNeXt for data-limited and resource-constrained fault diagnosis

In the field of intelligent machinery fault diagnosis, overcoming challenges arising from scarce labeled data and the demand for deployment on resource-constrained edge devices is imperative. To address these hurdles, this work aims to improve the ability of deep learning models to learn strong feature representations from limited data, while also reducing the model complexity. We presenting a novel network named TEdgeNeXt, the approach begins with a new signal-to-image conversion method, which is proved to be able to acquire less training data quantity. Structurally, the Convolutional (Conv.) Encoder initially is employed with depth-wise separable convolution to control the size of model rather than the traditional convolution, and the Split Depth-wise Transpose Attention (SDTA) encoder is consequently utilized by leveraging a multidimensional processing approach and the Multi-head Self-Attention which is across the channel dimensions instead of the spatial channel. By doing so, it effectively handles challenges such as high multiply-additions (MAdds) and increased latency through Flops and params. On the other hand, the fine-tune-based transfer learning technique is able to be extended in our approach for improving the capacity of generalizing. Ultimately, it indicates the noticeable improvements in Top-1 Accuracy (T1A), Mean Precision (MP), Mean Recall (MR), and Mean F1 score (MF1) across three distinct datasets.

DIDS: A distributed inference framework with dynamic scheduling capability

Distributed inference is a promising solution for deploying Deep Neural Network (DNN) applications in resource-constrained edge environments. However, due to the complexity and variability of edge scenarios, efficiently completing DNN inference remains challenging. While previous works have made significant progress in various aspects such as partition strategy, device diversity, and memory overhead, few consider the impact of environmental dynamics, particularly environmental workloads. This oversight renders existing solutions impractical. In this study, we propose DIDS, a distributed inference framework with dynamic scheduling capability that considers environmental dynamics. We first conduct a formal analysis of distributed inference and design two basic scheduling strategies: Re-Partition and Complete-Push. Then we introduce Dynamic Push, a runtime scheduler based on Complete-Push that aims to mitigate the impact of environmental workloads through runtime scheduling. The evaluation results show that DIDS can achieve up to 3.7x speedup compared to static distributed inference, in addition, DIDS is also able to achieve significant improvement compared to the state-of-the-art method.

Optimization of Memristor Crossbar’s Mapping Using Lagrange Multiplier Method and Genetic Algorithm for Reducing Crossbar’s Area and Delay Time

Memristor crossbars offer promising low-power and parallel processing capabilities, making them efficient for implementing convolutional neural networks (CNNs) in terms of delay time, area, etc. However, mapping large CNN models like ResNet-18, ResNet-34, VGG-Net, etc., onto memristor crossbars is challenging due to the line resistance problem limiting crossbar size. This necessitates partitioning full-image convolution into sub-image convolution. To do so, an optimized mapping of memristor crossbars should be considered to divide full-image convolution into multiple crossbars. With limited crossbar resources, especially in edge devices, it is crucial to optimize the crossbar allocation per layer to minimize the hardware resource in term of crossbar area, delay time, and area–delay product. This paper explores three optimization scenarios: (1) optimizing total delay time under a crossbar’s area constraint, (2) optimizing total crossbar area with a crossbar’s delay time constraint, and (3) optimizing a crossbar’s area–delay-time product without constraints. The Lagrange multiplier method is employed for the constrained cases 1 and 2. For the unconstrained case 3, a genetic algorithm (GA) is used to optimize the area–delay-time product. Simulation results demonstrate that the optimization can have significant improvements over the unoptimized results. When VGG-Net is simulated, the optimization can show about 20% reduction in delay time for case 1 and 22% area reduction for case 2. Case 3 highlights the benefits of optimizing the crossbar utilization ratio for minimizing the area–delay-time product. The proposed optimization strategies can substantially enhance the neural network’s performance of memristor crossbar-based processing-in-memory architectures, especially for resource-constrained edge computing platforms.

EDGE AI BASED OBJECT DETECTION SYSTEM USING TFLITE

Edge computing has gained prominence in recent years due to its ability to process data closer to the source, reducing latency and bandwidth requirements. In this paper, we propose an Edge AI Based Object Detection System using TensorFlow Lite (TFLite), designed to perform real-time object detection on resource-constrained edge devices. The system leverages the efficiency and portability of TFLite, a lightweight framework for deploying machine learning models on edge devices, to enable efficient inference without relying on cloud connectivity. Our proposed system integrates state-of-the-art object detection models, such as SSD (Single Shot Multibox Detector) and YOLO (You Only Look Once), into the TFLite runtime environment. Through optimization techniques such as model quantization, pruning, and architecture modifications, we tailor these models to meet the computational and memory constraints of edge devices while maintaining high detection accuracy. Furthermore, we explore hardware acceleration options, including GPU and DSP (Digital Signal Processor), to further enhance inference speed and energy efficiency. We evaluate the performance of the Edge AI Based Object Detection System on various edge devices, including smartphones, IoT (Internet of Things) devices, and embedded systems. Real-world deployment scenarios are considered, encompassing applications such as smart surveillance, industrial automation, and autonomous vehicles. The results demonstrate the system's ability to achieve real-time object detection with low latency and minimal resource consumption, making it well-suited for edge computing environments where real-time responsiveness and privacy are paramount concerns. Keywords: object Detection, Machine Learning, Tensor Flow lite, deep learning.

A High-Performance and Ultra-Low-Power Accelerator Design for Advanced Deep Learning Algorithms on an FPGA

This article addresses the growing need in resource-constrained edge computing scenarios for energy-efficient convolutional neural network (CNN) accelerators on mobile Field-Programmable Gate Array (FPGA) systems. In particular, we concentrate on register transfer level (RTL) design flow optimization to improve programming speed and power efficiency. We present a re-configurable accelerator design optimized for CNN-based object-detection applications, especially suitable for mobile FPGA platforms like the Xilinx PYNQ-Z2. By not only optimizing the MAC module using Enhanced clock gating (ECG), the accelerator can also use low-power techniques such as Local explicit clock gating (LECG) and Local explicit clock enable (LECE) in memory modules to efficiently minimize data access and memory utilization. The evaluation using ResNet-20 trained on the CIFAR-10 dataset demonstrated significant improvements in power efficiency consumption (up to 22%) and performance. The findings highlight the importance of using different optimization techniques across multiple hardware modules to achieve better results in real-world applications.

RegionFilter: Region-aware video filtering mechanism on resource-constrained edge nodes

The video analytics pipeline consumes substantial network and computing resources, and in many scenarios videos are sent from cameras to servers. Due to limited camera-side resources, most existing systems are frame-level filtering approaches that filter out frames that are unchanged or irrelevant to applications before transmitting data. However, video applications usually care about the regions of interest (RoIs) of frames rather than background regions, and frame-level filtering fails to consider the spatial redundancy of intra-frame regions. We argue that video analytics should adopt a region-level filtering approach. Unfortunately, how to perform region-aware filtering on cameras without GPU is a significant challenge. To overcome this, we built RegionFilter, which achieves region awareness on resource-constrained edge nodes through feedback results from server-side DNN models. RegionFilter splits each video segment into multiple sub-segments in spatial distribution through region awareness. Then different configuration parameters (e.g., resolution, quantization parameter (QP)) are selected according to the video application, and the sub-segments are encoded into sub-streamings with different qualities that are uploaded to servers for further processing. Experiments on various video tasks and datasets show that RegionFilter achieves considerable filtering benefits (bandwidth savings of 11.7–41.2% when transmitting HD video) while always meeting the desired accuracy.