Edge Devices Research Articles

Federated Learning for Edge Computing in REST APIs: Investigate the Feasibility of Employing Federated Learning Techniques to Train Machine Learning Models Directly on Edge Devices Interacting with REST APIs, Minimizing Data Transfer and Enhancing Privacy

This research explores the feasibility of using federated learning techniques to train machine learning models directly on edge devices interacting through REST APIs. This article designs an architecture that facilitates federated learning using REST APIs for communication between a central server and edge nodes. The architecture ensures efficient model distribution, update aggregation, and iterative global model refinement. This research implements this system in a simulated environment to evaluate its performance. Initial results show that federated learning significantly reduces data transfer and enhances privacy without compromising model accuracy. The REST API-based communication model effectively coordinates updates between the central server and edge devices, and privacy-preserving techniques maintain data security throughout the process. This study confirms that federated learning is a viable solution for edge computing, addressing key challenges in data privacy and communication efficiency, and opens pathways for scalable and effective distributed learning systems in real-world IoT applications. Federated learning offers a decentralized alternative, enabling local model training on edge devices with only model updates shared with a central server. This approach minimizes data transfer and enhances privacy.

Light4Mars: A lightweight transformer model for semantic segmentation on unstructured environment like Mars

Auto-semantic segmentation is important for robots on unstructured and dynamic environments like planets where ambient conditions cannot be controlled and the scale is larger than that found indoors. Current learning-based methods have achieved breathtaking improvements on this topic. For onboard applications, however, all those methods still suffer from huge computational costs and are difficult to deploy on edge devices. In this paper, unlike previous transformer-based SOTA approaches that heavily relied on complex design, we proposed Light4Mars, a lightweight model with minimal computational complexity while maintaining high segmenting accuracy. We designed a lightweight squeeze window transformer module that focuses on window-scale feature extraction and is more effective in learning global and local contextual information. The aggregated local attention decoder is utilized to fuse semantic information at different scales, especially for unstructured scenes. Since there are few all-terrain datasets for semantic segmentation of unstructured scenes like Mars, we built a synthetic dataset SynMars-TW, referencing images collected by the ZhuRong rover on the Tianwen-1 mission and the Curiosity rover. Extensive experiments on SynMars-TW and the real Mars dataset, MarsScapes show that our approach achieves state-of-the-art performance with favorable computational simplicity. To the best of our knowledge, the proposed Light4Mars-T network is the first segmentation model for Mars image segmentation with parameters lower than 0.1M. Code and datasets are available at https://github.com/CVIR-Lab/Light4Mars.

A Novel Method for Rolling Bearing Fault Diagnosis Based on Gramian Angular Field and CNN-ViT.

Fault diagnosis is one of the important applications of edge computing in the Industrial Internet of Things (IIoT). To address the issue that traditional fault diagnosis methods often struggle to effectively extract fault features, this paper proposes a novel rolling bearing fault diagnosis method that integrates Gramian Angular Field (GAF), Convolutional Neural Network (CNN), and Vision Transformer (ViT). First, GAF is used to convert one-dimensional vibration signals from sensors into two-dimensional images, effectively retaining the fault features of the vibration signal. Then, the CNN branch is used to extract the local features of the image, which are combined with the global features extracted by the ViT branch to diagnose the bearing fault. The effectiveness of this method is validated with two datasets. Experimental results show that the proposed method achieves average accuracies of 99.79% and 99.63% on the CWRU and XJTU-SY rolling bearing fault datasets, respectively. Compared with several widely used fault diagnosis methods, the proposed method achieves higher accuracy for different fault classifications, providing reliable technical support for performing complex fault diagnosis on edge devices.

Hybrid metaheuristics for selective inference task offloading under time and energy constraints for real-time IoT sensing systems

The recent widespread of AI-powered real-time applications necessitates the use of edge computing for inference task offloading. Power constrained edge devices are required to balance between processing inference tasks locally or offload to edge servers. This decision is determined according to the time constraint demanded by the real-time nature of applications, and the energy constraint dictated by the device’s power budget. This problem is further exacerbated in the case of systems leveraging multiple local inference models varying in size and accuracy. In this work, we tackle the problem of assigning inference models to inference tasks either using local inference models or by offloading to edge servers under time and energy constraints while maximizing the overall accuracy of the system. This problem is shown to be strongly NP-hard and therefore, we propose a hybrid genetic algorithm (HGSTO) to solve this problem. We leverage the speed of simulated annealing (SA) with the accuracy of genetic algorithms (GA) to develop a hybrid, fast and accurate algorithm compared with classic GA, SA and Particle Swarm Optimization (PSO). Experiment results show that HGSTO achieved on-par or higher accuracy than GA while resulting in significantly lower scheduling times compared to other schemes.

A generic deep learning architecture optimization method for edge device based on start-up latency reduction

In the promising Artificial Intelligence of Things technology, deep learning algorithms are implemented on edge devices to process data locally. However, high-performance deep learning algorithms are accompanied by increased computation and parameter storage costs, leading to difficulties in implementing huge deep learning algorithms on memory and power constrained edge devices, such as smartphones and drones. Thus various compression methods are proposed, such as channel pruning. According to the analysis of low-level operations on edge devices, existing channel pruning methods have limited effect on latency optimization. Due to data processing operations, the pruned residual blocks still result in significant latency, which hinders real-time processing of CNNs on edge devices. Hence, we propose a generic deep learning architecture optimization method to achieve further acceleration on edge devices. The network is optimized in two stages, Global Constraint and Start-up Latency Reduction, and pruning of both channels and residual blocks is achieved. Optimized networks are evaluated on desktop CPU, FPGA, ARM CPU, and PULP platforms. The experimental results show that the latency is reduced by up to 70.40%, which is 13.63% higher than only applying channel pruning and achieving real-time processing in the edge device.

Deep Learning Model Compression Techniques Performance on Edge Devices

Artificial intelligence at the edge can help solve complex tasks faced by various sectors such as automotive, healthcare and surveillance. However, challenged by the lack of computational power from the edge devices, artificial intelligence models are forced to adapt. Many have developed and quantified model compres-sion approaches over the years to tackle this problem. However, not many have considered the overhead of on-device model compression, even though model compression can take a considerable amount of time. With the added metric, we provide a more complete view on the efficiency of model compression on the edge. The objective of this research is identifying the benefit of compression methods and it’s tradeoff between size and latency reduction versus the accuracy loss as well as compression time in edge devices. In this work, quantitative method is used to analyze and rank three common ways of model compression: post-training quantization, unstructured pruning and knowledge distillation on the basis of accuracy, latency, model size and time to compress overhead. We concluded that knowledge distillation is the best, with potential of up to 11.4x model size reduction, and 78.67% latency speed up, with moderate loss of accura-cy and compression time.

Federated Momentum Contrastive Clustering

Self-supervised representation learning and deep clustering are mutually beneficial to learn high-quality representations and cluster data simultaneously in centralized settings. However, it is not always feasible to gather large amounts of data at a central entity, considering data privacy requirements and computational resources. Federated Learning (FL) has been developed successfully to aggregate a global model while training on distributed local data, respecting the data privacy of edge devices. However, most FL research effort focuses on supervised learning algorithms. A fully unsupervised federated clustering scheme has not been considered in the existing literature. We present federated momentum contrastive clustering (FedMCC), a generic federated clustering framework that can not only cluster data automatically but also extract discriminative representations training from distributed local data over multiple users. In FedMCC, we demonstrate a two-stage federated learning paradigm where the first stage aims to learn differentiable instance embeddings and the second stage accounts for clustering data automatically. The experimental results show that FedMCC not only achieves superior clustering performance but also outperforms several existing federated self-supervised methods for linear evaluation and semi-supervised learning tasks. Additionally, FedMCC can easily be adapted to ordinary centralized clustering through what we call momentum contrastive clustering (MCC). We show that MCC achieves state-of-the-art clustering accuracy results in certain datasets such as STL-10 and ImageNet-10. We also present a method to reduce the memory footprint of our clustering schemes.

Low‐Power Charge Trap Flash Memory with MoS2 Channel for High‐Density In‐Memory Computing

AbstractWith the rise of on‐device artificial intelligence (AI) technology, the demand for in‐memory comptuing has surged for data‐intensive tasks on edge devices. However, on‐device AI requires high‐density, low‐power memory‐based computing to efficiently handle large data volumes. Here, this study proposes a reliable multilevel, high gate‐coupling ratio memory device with MoS2 channel tailored for high‐density 3D NAND Flash‐based in‐memory computing. The MoS2 channel, featured by its small bandgap and high‐mobility, facilitates reliable memory window of approximately 8 V thanks to erase operation through hole injection. This not only suppresses vertical charge loss but also alleviates the burden on voltage generator circuits, indicating the suitability of MoS2 as channel material for 3D NAND Flash architecture. Additionally, a low‐k (≈2.2) tunneling layer deposited via initiated chemical vapor deposition increases the gate‐coupling ratio, thereby reducing the operating voltage. Utilizing Au nanoparticles as the charge storage layer, MoS2 memory devices show synaptic plasticity with 6‐bit, endurance (104 cycles), read disturbance (105 cycles), and retention times (105 s). Furthermore, device‐to‐system simulations for neural networks based on MoS2‐memory devices have successfully achieved a fingerprint recognition of 95.8%. These results provide the foundation to develop multi‐bit MoS2‐memory devices for AI accelerators and 3D NAND Flash memory.

Investigation on a lightweight defect detection model for photovoltaic panel

The detection of defect types of photovoltaic (PV) panel is a crucial task in PV system. Existing detection models face challenges in effectively balancing the trade-off between detection accuracy and resource consumption. To address this issue, this paper proposes a new defect detection method for PV panel based on the improved YOLOv8 model, which realizes both the high detection accuracy and the lightweight. Firstly, Reversible Column Networks (RevCol) is used as the Backbone of YOLOv8, which makes sure to preserve the feature information in the process of network transmission and also reduces the number of parameters and Giga floating-point operations per second (GFLOPs). Subsequently, a new lightweight Bottleneck fused with Efficient Multi-Scale Attention (EMA) is designed to optimize the CSPDarknet53 to 2-Stage FPN (C2f) module of Neck in YOLOv8 to enhance the robustness and further decrease network parameters. Finally, Squeeze-and-Excitation (SE) Attention is integrated into the Head of YOLOv8 to prioritize the important channel features and thus enhance the detection performance. The experimental results on the PVEL-AD dataset demonstrate that parameters and GFLOPs of the proposed model are declined by 38.46% and 34.39% respectively, and mAP0.5:0.95 is increased by 2.6% compared with the baseline model. The lightweight improved YOLOv8 model facilitates the deployment of deep learning model on edge devices and provides a novel approach for the online detection of PV panel defects.

Real-Time Object Detection from UAV Inspection Videos by Combining YOLOv5s and DeepStream.

The high-altitude real-time inspection of unmanned aerial vehicles (UAVs) has always been a very challenging task. Because high-altitude inspections are susceptible to interference from different weather conditions, interference from communication signals and a larger field of view result in a smaller object area to be identified. We adopted a method that combines a UAV system scheduling platform with artificial intelligence object detection to implement the UAV automatic inspection technology. We trained the YOLOv5s model on five different categories of vehicle data sets, in which mAP50 and mAP50-95 reached 93.2% and 71.7%, respectively. The YOLOv5s model size is only 13.76 MB, and the detection speed of a single inspection photo reaches 11.26 ms. It is a relatively lightweight model and is suitable for deployment on edge devices for real-time detection. In the original DeepStream framework, we set up the http communication protocol to start quickly to enable different users to call and use it at the same time. In addition, asynchronous sending of alarm frame interception function was added and the auxiliary services were set up to quickly resume video streaming after interruption. We deployed the trained YOLOv5s model on the improved DeepStream framework to implement automatic UAV inspection.

TinyML for smart agriculture: Comparative analysis of TinyML platforms and practical deployment for maize leaf disease identification

Maize, a vital global food crop, is susceptible to various diseases impacting agricultural yield and quality, necessitating effective monitoring strategies. This study presents a comprehensive approach utilizing advanced Convolutional Neural Network (CNN) architecture for efficient maize leaf disease detection and classification. Leveraging datasets from Kaggle, Harvard Dataverse, and Mendeley, the proposed CNN maize disease detection (MDD) system integrates optimization techniques, data augmentation, batch size adjustments, and fine-tuning strategies. Evaluation through data splitting reveals significant correlations among maize leaf classifications and the influence of data augmentation on image quality. Following training, the CNN-MDD model is deployed on a Microcontroller Unit (MCU) simulation, achieving promising optimization parameters: latency of 7.60 ms, RAM size of 726.60 KB, classifier flash size of 344.70 KB, and an accuracy of 94.60 %. These optimized parameters align well with MCU specifications, highlighting the potential for practical deployment on low-cost edge devices like Arduino Ble 33 Sense. Additionally, this research contributes to the creation of a Ghanaian maize crop pest and disease dataset and the development of an optimal and cost-effective CNN-MDD MCU edge device, aiming to provide local farmers in Ghana and Sub-Saharan Africa with a swift, offline solution for maize disease detection, thus fostering sustainable agricultural practices. We conduct a comparative analysis of Edge Impulse and TensorFlow, two prominent machine-learning platforms for TinyML model development and deployment in various applications, including smart crop disease detection. Utilizing a dataset comprising 12,344 raw images and 7,454 augmented images of healthy and diseased maize leaves, we evaluate the workflow, features, performance, and usability of each platform. Our findings indicate that Edge Impulse holds a slight advantage over TensorFlow in terms of ease of use, reliability, and memory footprint, whereas TensorFlow offers greater flexibility, customization options, and higher accuracy. We conclude that while both platforms are suitable for crop disease detection, a combined approach allows for leveraging TensorFlow's robust training capabilities for complex neural networks alongside the simplicity of Edge Impulse for deploying models on low-powered edge devices. This synergistic approach enhances the effectiveness and efficiency of crop disease detection systems, ultimately benefiting agricultural practices and food security.

Parallel photonic chip for nanosecond end-to-end image processing, transmission, and reconstruction

Image processing, transmission, and reconstruction constitute a major proportion of information technology. The rapid expansion of ubiquitous edge devices and data centers has led to substantial demands on the bandwidth and efficiency of image processing, transmission, and reconstruction. The frequent conversion of serial signals between the optical and electrical domains, coupled with the gradual saturation of electronic processors, has become the bottleneck of end-to-end machine vision. Here, we present an optical parallel computational array chip (OPCA chip) for end-to-end processing, transmission, and reconstruction of optical intensity images. By proposing constructive and destructive computing modes on the large-bandwidth resonant optical channels, a parallel computational model is constructed to implement end-to-end optical neural network computing. The OPCA chip features a measured response time of 6 ns and an optical bandwidth of at least 160 nm. Optical image processing can be efficiently executed with minimal energy consumption and latency, liberated from the need for frequent optical–electronic and analog–digital conversions. The proposed optical computational sensor opens the door to extremely high-speed processing, transmission, and reconstruction of visible contents with nanoseconds response time and terahertz bandwidth.

ResPrune: An energy-efficient restorative filter pruning method using stochastic optimization for accelerating CNN

Convolutional Neural Networks (CNNs) are frequently employed for image pattern recognition and other computer vision tasks. When over-parameterized deep learning models are used for inference, resource-constrained edge devices may struggle. As a result, model compression, particularly filter pruning, has become critical. A reduction in model size might result in less calculation, resulting in faster hardware execution and lower energy consumption. One of the drawbacks of current pruning strategies is that once the filters are pruned, their weights are permanently lost. To address this constraint, we propose a unique two-phase pruning technique in which the filters to be pruned are selected using two criteria: l2-norm and redundancy. Second, rather than omitting the selected filters for all future epochs, we restore them to their original value with some stochasticity. Retaining the most optimal filter weights in earlier epochs enables the survival of the fittest filters, resulting in higher model convergence. Experiments on three benchmark datasets, CIFAR-10, CIFAR-100, and ILSVRC-2012, reveal that our strategy outperforms other state-of-the-art pruning methods by a minimum reduction of 57% FLOPs with an accuracy loss as minimal as 0.08 %.

A more efficient method for wire rope surface defect detection based on fusing cross-stage features and lightweight convolution modules

Abstract Detecting surface defects in industrial production presents challenges, including deploying lightweight algorithms on edge devices and balance between detection speed and accuracy. This paper introduces a novel lightweight method YOLO-FP for real-time detection of wire rope defects in industrial environments. Utilizing the YOLOv7tiny framework, we developed a lightweight cross-stage feature fusion module to enhance processing feature information. This mitigates the impact of redundant information from traditional convolution, reducing network size and improving detection speed. Additionally, the YOLO-FP network integrates lightweight convolution modules and an attention mechanism. Trained and tested on data from Changan, Great Wall, and Guangzhou Automobile models, our method achieved a 96.06% mean average precision, surpassing the original YOLOv7tiny model. Furthermore, it reduces the model size by 41.09% and enhances detection speed by 18.53%, making it promising for real-time edge device applications in wire rope production.

AMED: Automatic Mixed-Precision Quantization for Edge Devices

Quantized neural networks are well known for reducing the latency, power consumption, and model size without significant harm to the performance. This makes them highly appropriate for systems with limited resources and low power capacity. Mixed-precision quantization offers better utilization of customized hardware that supports arithmetic operations at different bitwidths. Quantization methods either aim to minimize the compression loss given a desired reduction or optimize a dependent variable for a specified property of the model (such as FLOPs or model size); both make the performance inefficient when deployed on specific hardware, but more importantly, quantization methods assume that the loss manifold holds a global minimum for a quantized model that copes with the global minimum of the full precision counterpart. Challenging this assumption, we argue that the optimal minimum changes as the precision changes, and thus, it is better to look at quantization as a random process, placing the foundation for a different approach to quantize neural networks, which, during the training procedure, quantizes the model to a different precision, looks at the bit allocation as a Markov Decision Process, and then, finds an optimal bitwidth allocation for measuring specified behaviors on a specific device via direct signals from the particular hardware architecture. By doing so, we avoid the basic assumption that the loss behaves the same way for a quantized model. Automatic Mixed-Precision Quantization for Edge Devices (dubbed AMED) demonstrates its superiority over current state-of-the-art schemes in terms of the trade-off between neural network accuracy and hardware efficiency, backed by a comprehensive evaluation.

A novel adaptive sampling algorithm for cyber-physical systems

AbstractSensors are the main components in Cyber-Physical Systems (CPS), which transmit large amounts of physical values and big data to computing platforms for processing. On the other hand, the embedded processors (as edge devices in fog computing) spend most of their time reading the sensor signals as compared with computing time. The impact of sensors on the performance of fog computing is very great, thus, the enhancement of the reading time of sensors will positively affect the performance of fog computing, and solves the CPS challenges such as delay, timed precision, temporal behavior, energy, and cost. In this paper, we propose an algorithm based on the 1st derivative of the sensor signal to generate an adaptive sampling frequency. The proposed algorithm uses an adaptive frequency to capture the sudden and rapid change in sensor signal in the steady state. Finally, we realize and tested it using the Ptolemy II Modeling Environment.

Identification of a Person in a Trajectory Based on Wearable Sensor Data Analysis.

Human trajectories can be tracked by the internal processing of a camera as an edge device. This work aims to match peoples' trajectories obtained from cameras to sensor data such as acceleration and angular velocity, obtained from wearable devices. Since human trajectory and sensor data differ in modality, the matching method is not straightforward. Furthermore, complete trajectory information is unavailable; it is difficult to determine which fragments belong to whom. To solve this problem, we newly proposed the SyncScore model to find the similarity between a unit period trajectory and the corresponding sensor data. We also propose a Likelihood Fusion algorithm that systematically updates the similarity data and integrates it over time while keeping other trajectories in mind. We confirmed that the proposed method can match human trajectories and sensor data with an accuracy, a sensitivity, and an F1 of 0.725. Our models achieved decent results on the UEA dataset.

A strong anti-noise and easily deployable bearing fault diagnosis model based on time–frequency dual-channel Transformer

Deep learning is widely used in Bearing Fault Diagnosis (BFD). Nonetheless, practical industrial production often generates a large amount of industrial noise. These noises exhibit randomness and complexity, which puts forward higher requirements for diagnosis algorithms. Certain studies have tackled the issue of anti-interference in high-noise environments (SNR≤0dB) by increasing the complexity of the model. However, due to the excessive number of parameters and computation, such models cannot be deployed on low-end edge devices. Balancing resource consumption and accuracy has become a major challenge in BFD modeling research. To solve the above problems, this paper proposes a new Transformer architecture model called LTFAFormer. The LTFAFormer is capable of achieving high-precision diagnostics on low-end edge devices and shows greater noise resistance. In terms of processing sequence information, Transformer has proven to be superior to other solutions. However, when dealing with longer sensor signal data containing complex noise, the traditional self-attention mechanism not only cannot effectively extract fault features, but also generates more computational complexity than CNN. To address this issue, we propose a novel time–frequency dual-channel parallel attention mechanism. Our approach enhances the feature extraction capability of the model by expanding the attention computation scale and reduces the computational resource consumption of the model by optimizing the model structure. To validate the effectiveness of LTFAFormer, we present two cases to demonstrate that LTFAFormer has higher prediction accuracy while satisfying lightweight. Especially in high-noise environments, LTFAFormer has stronger robustness. In this paper provides a new set of feasible strategies for the practical deployment of BFD models in practical industrial environments.

An Improved YOLOv5s Model for Building Detection

With the continuous advancement of autonomous vehicle technology, the recognition of buildings becomes increasingly crucial. It enables autonomous vehicles to better comprehend their surrounding environment, facilitating safer navigation and decision-making processes. Therefore, it is significant to improve detection efficiency on edge devices. However, building recognition faces problems such as severe occlusion and large size of detection models that cannot be deployed on edge devices. To solve these problems, a lightweight building recognition model based on YOLOv5s is proposed in this study. We first collected a building dataset from real scenes and the internet, and applied an improved GridMask data augmentation method to expand the dataset and reduce the impact of occlusion. To make the model lightweight, we pruned the model by the channel pruning method, which decreases the computational costs of the model. Furthermore, we used Mish as the activation function to help the model converge better in sparse training. Finally, comparing it to YOLOv5s (baseline), the experiments show that the improved model reduces the model size by 9.595 MB, and the mAP@0.5 reaches 82.3%. This study will offer insights into lightweight building detection, demonstrating its significance in environmental perception, monitoring, and detection, particularly in the field of autonomous driving.

Federated learning inspired Antlion based orchestration for Edge computing environment.

Edge computing is a scalable, modern, and distributed computing architecture that brings computational workloads closer to smart gateways or Edge devices. This computing model delivers IoT (Internet of Things) computations and processes the IoT requests from the Edge of the network. In a diverse and independent environment like Fog-Edge, resource management is a critical issue. Hence, scheduling is a vital process to enhance efficiency and allocation of resources properly to the tasks. The manuscript proposes an Artificial Neural Network (ANN) inspired Antlion algorithm for task orchestration Edge environments. Its aim is to enhance resource utilization and reduce energy consumption. Comparative analysis with different algorithms shows that the proposed algorithm balances the load on the Edge layer, which results in lower load on the cloud, improves power consumption, CPU utilization, network utilization, and reduces average waiting time for requests. The proposed model is tested for healthcare application in Edge computing environment. The evaluation shows that the proposed algorithm outperforms existing fuzzy logic algorithms. The performance of the ANN inspired Antlion based orchestration approach is evaluated using performance metrics, power consumption, CPU utilization, network utilization, and average waiting time for requests respectively. It outperforms the existing fuzzy logic, round robin algorithm. The proposed technique achieves an average cloud energy consumption improvement of 95.94%, and average Edge energy consumption improvement of 16.79%, 19.85% in average CPU utilization in Edge computing environment, 10.64% in average CPU utilization in cloud environment, and 23.33% in average network utilization, and the average waiting time decreases by 96% compared to fuzzy logic and 1.4% compared to round-robin respectively.