Edge Devices Research Articles

Feasibility Study of Edge Computing Empowered by Artificial Intelligence—A Quantitative Analysis Based on Large Models

The advancement of artificial intelligence (AI) demands significant data and computational resources that have an adverse impact on the environment. To address this issue, a novel computing architecture that is both energy efficient and eco-friendly is urgently required. Edge computing has emerged as an increasingly popular solution to this problem. In this study, we explore the development history of edge computing and AI and analyze the potential of model quantization to link AI and edge computing. Our comparative analysis demonstrates that the quantization approach can effectively reduce the model’s size and accelerate model inference while maintaining its functionality, thereby enabling its deployment on edge devices. This research serves as a valuable guide and reference for future advancements in edge AI.

Melon ripeness detection by an improved object detection algorithm for resource constrained environments

BackgroundRipeness is a phenotype that significantly impacts the quality of fruits, constituting a crucial factor in the cultivation and harvesting processes. Manual detection methods and experimental analysis, however, are inefficient and costly.ResultsIn this study, we propose a lightweight and efficient melon ripeness detection method, MRD-YOLO, based on an improved object detection algorithm. The method combines a lightweight backbone network, MobileNetV3, a design paradigm Slim-neck, and a Coordinate Attention mechanism. Additionally, we have created a large-scale melon dataset sourced from a greenhouse based on ripeness. This dataset contains common complexities encountered in the field environment, such as occlusions, overlapping, and varying light intensities. MRD-YOLO achieves a mean Average Precision of 97.4% on this dataset, achieving accurate and reliable melon ripeness detection. Moreover, the method demands only 4.8 G FLOPs and 2.06 M parameters, representing 58.5% and 68.4% of the baseline YOLOv8n model, respectively. It comprehensively outperforms existing methods in terms of balanced accuracy and computational efficiency. Furthermore, it maintains real-time inference capability in GPU environments and demonstrates exceptional inference speed in CPU environments. The lightweight design of MRD-YOLO is anticipated to be deployed in various resource constrained mobile and edge devices, such as picking robots. Particularly noteworthy is its performance when tested on two melon datasets obtained from the Roboflow platform, achieving a mean Average Precision of 85.9%. This underscores its excellent generalization ability on untrained data.ConclusionsThis study presents an efficient method for melon ripeness detection, and the dataset utilized in this study, alongside the detection method, will provide a valuable reference for ripeness detection across various types of fruits.

LUD-YOLO: A novel lightweight object detection network for unmanned aerial vehicle

Autonomous execution of tasks by unmanned aerial vehicles (UAVs) relies heavily on object detection. However, object detection in most images presents challenges such as complex backgrounds, small targets, and obstructions. Additionally, the limited computing speed and memory of the UAV processor affects the accuracy of conventional object detection algorithms. This paper proposes LUD-You Only Look Once (YOLO), a small and lightweight object detection algorithm for UAVs based on YOLOv8. The proposed algorithm introduces a new multiscale feature fusion mode that solves the degradation in feature propagation and interaction through the introduction of upsampling in the feature pyramid network and the progressive feature pyramid network. The application of the dynamic sparse attention mechanism in the Cf2 module achieves flexible computing allocation and content awareness. Furthermore, the proposed model is optimized to be sparse and lightweight, making it possible to deploy on UAV edge devices. Finally, the effectiveness and superiority of LUD-YOLO were verified on the VisDrone2019 and UAVDT datasets. The results of ablation and comparison experiments show that compared with the original algorithm, LUDY-N and LUDY-S have shown excellent performance in various evaluation indexes, indicating that the proposed improvement strategies make the model have better robustness and generalization. Moreover, compared with multiple other popular competitors, the proposed improvement strategies enable LUD-YOLO to have the best overall performance, providing an effective solution for UAVs object detection while balancing model size and detection accuracy.

Real-time tilapia fillet defect segmentation on edge device for robotic trimming

The implementation of robotic tilapia fillet trimming instead of manual labor is a pivotal advancement in intelligent fish processing, offering substantial superiority in operational efficiency and product quality. The study presents an improved model called TFDS-YOLOv8n for tilapia fillet defects segmentation. The model incorporates Coordinate Attention (CA) into the feature extraction layer, thereby enhancing its ability to capture characteristics at various levels of the input. Additionally, the feature fusion layer is reconstructed as Slim-Neck to reduce the number of parameters without compromising prediction accuracy. Furthermore, the bounding box loss function is modified by MPDIoU to expedite the model convergence. The proposed model was tested employing the dataset collected from the practical tilapia processing plant. Ablation experiments demonstrate that TFDS-YOLOv8n achieves a reduction of 0.29 MB in parameters and 1 G in Floating Point Operations (FLOPs) while increasing bbox_mAP and mask_mAP by 2.8 % and 2.5 %, respectively. Eventually, the model is further accelerated by TensorRT and deployed on the edge device. Experimental results show that the quantized model possesses a faster inference speed than the untransformed model, which realizes real-time detection of tilapia fillet defects and facilitates robotization in fish processing.

Improved lightweight infrared road target detection method based on YOLOv8

Infrared-based road scene object detection algorithms often face issues with excessive parameters and computational demands, making them incompatible with edge devices having constrained computational capabilities. This paper introduces an enhanced lightweight infrared-based road object detection algorithm based on YOLOv8n. Firstly, a streamlined network architecture is devised by merging YOLOv8n’s C2f module with PConv, creating a lighter module and reducing the neural network’s downsampling rate of infrared images. This strategy reduces redundant computations and memory access, preventing the loss of fine details in infrared images caused by deep convolutional neural networks. Additionally, the model’s accuracy in detecting infrared targets is significantly enhanced through the integration of the coordinate attention mechanism. Finally, replacing CIoU with Wise-IoU for bounding box regression in YOLOv8n accelerates the model’s convergence. Empirical findings indicate that in contrast to the YOLOv8n algorithm, the optimized model showcases a 34.17 % reduction in model size, a 40.35 % decrease in parameters, and a 4.8 % increase in average detection accuracy. This enhanced algorithm not only achieves a lightweight profile but also delivers superior performance on embedded edge devices.

Reliability-assured service function chain migration strategy in edge networks using deep reinforcement learning

With the widespread adoption of edge computing and the rollout of 5G technology, the edge network is experiencing rapid growth. Edge computing enables the execution of certain computational tasks on edge devices, fostering more efficient resource utilization. However, the reliability of the edge network is constrained by its network connections. Network instability can significantly compromise service quality. An effective service function chain (SFC) migration algorithm is essential to optimize resource utilization, enhance service quality. This paper begins by analyzing the current research landscape of edge networks and SFC migration algorithms. Subsequently, the challenges associated with edge network and SFC migration are formally articulated, leading to the proposal of a SFC migration algorithm based on deep reinforcement learning (DRL) with a focus on reliability assurance (RA-SFCM). The algorithm leverages multi-agent deep reinforcement learning to dynamically perceive changes in the edge network environment. It introduces an advantage function to evaluate the performance of each agent relative to the average level and incorporates a central attention mechanism with multiple attention heads to better capture the interdependencies and relationships among different agents. Additionally, this paper innovatively defines and quantifies the reliability of the migration process. By introducing a reliability penalty mechanism based on the migration target nodes and link capacity, it enhances the reliability of the migration schemes. The experimental results conclusively demonstrate the remarkable advantages of the RA-SFCM algorithm in terms of real-time performance, resource utilization efficiency, and reliability. Compared to algorithms such as Sa-VNFM, ROVM, and DLTSAC, RA-SFCM exhibits superior performance. For RA-SFCM, the optimized deployment migration strategy enhances real-time performance, precise resource management improves utilization efficiency, and advanced fault tolerance mechanisms strengthen reliability.

Implementation of Application Specific Instruction set Processor for Approximate Computing

The performance, energy efficiency, power dissipation, and cost of the processors used in Internet of Things edge devices can be improved by introducing approximate computing. In this paper, the design and synthesis of an application-specific instruction set processor for approximate computing is proposed. Approximate computing can also be defined by using Artificial Intelligence-based technique like the Fractal theory and Inconsistent Information System. Application Specific Instruction Set Processor is a new processor design area and is used to design a new instruction set using existing processor configuration available on the EDA tool. From the results, it is observed that significant improvement is achieved in processor performance using the TIE application as compared to without the TIE application which is applied to user-designed instruction.

Fault detection and diagnosis of grid-connected photovoltaic systems using energy valley optimizer based lightweight CNN and wavelet transform

Early fault detection and diagnosis of grid-connected photovoltaic systems (GCPS) is imperative to improve their performance and reliability. Low-cost edge devices have emerged as innovative solutions for real-time monitoring, reducing latency, and improving response times. In this work, a lightweight Convolutional Neural Network (CNN) is designed and fine-tuned using Energy Valley Optimizer (EVO) for fault diagnosis. The CNN input consists of two-dimensional scalograms generated using Continuous Wavelet Transform (CWT). The proposed diagnosis technique demonstrated superior performance compared to benchmark architectures, namely MobileNet, NASNetMobile, and InceptionV3, achieving higher test accuracies and lower losses on binary and multi-fault classification tasks on balanced, unbalanced, and noisy datasets. Further, a quantitative comparison is conducted with similar recent studies. The obtained results indicate good performance and high reliability of the proposed fault diagnosis method.

A smart surveillance system utilizing modified federated machine learning: Gossip‐verifiable and quantum‐safe approach

SummaryEdge computing has the capability to process data closer to its point of origin, leading to the development of critical autonomous infrastructures with frequently communicating peers. The proposed work aims to evaluate the effectiveness of security and privacy mechanisms tailored for distributed systems, particularly focusing on scenarios where the nodes are closed‐circuit television (CCTV) systems. Ensuring public safety, object tracking in surveillance systems is a vital responsibility. The workflow has been specifically crafted and simulated for the purpose of weapon detection within public CCTV systems, utilizing sample edge devices. The system's primary objective is to detect any unauthorized use of weapons in public spaces while concurrently ensuring the integrity of video footage for use in criminal investigations. The outcomes of prior research on distributed machine learning (DML) techniques are compared with modified federated machine learning (FML) techniques, specifically designed for being Gossip verifiable and Quantum Safe. The conventional federated averaging algorithm is modified by incorporating the secret sharing principle, coupled with code‐based McEliece cryptosystem. This adaptation is designed to fortify the system against quantum threats. The Gossip data dissemination protocol, executed via custom blockchain atop the distributed network, serves to authenticate and validate the learning model propagated among the peers in the network. It provides additional layer of integrity to the system. Potential threats to the proposed model are analyzed and the efficiency of the work is assessed using formal proofs. The outcomes of the proposed work demonstrate that the trustworthiness and consistency are meticulously preserved for both the model and data within the DML framework on the Edge computing platform.

Improving the sustainability of WiFi-enabled indoor localization systems through meta-heuristic based instance selection approach

Due to rapid urban development, many indoor and semi-indoor structures, such as underground malls, metro stations, and bus-stops, have emerged. Traditional GNSS outdoor navigation fails in these locations. To meet urban citizens’ needs, we require sustainable ubiquitous localization solutions, leveraging existing WiFi infrastructure. In such applications, processing data at the edge within constrained environments is imperative. Many prior studies neglect the practical challenge of implementing machine learning-based edge device localization, as they assume reliance on cloud servers. Selecting well-distributed data instances that preserve location class boundaries is essential for effective training in such conditions. In this work, we propose an instance selection approach that is modeled by applying a modified Binary Particle Swarm Optimization (BPSO) technique. The k-differentiating neighbor-based measure is incorporated to determine instance hardness while exploring the solution space through BPSO. The work is implemented on three publicly available benchmark datasets that are collected from two university campuses and one shopping mall. The proposed work is found to have achieved an instance reduction of 35% with only 1%–2% decrease in the accuracy measurement and an appreciable error deviation metric of 2.78 m.

Revealing CNN Architectures via Side-Channel Analysis in Dataflow-based Inference Accelerators

Convolutional Neural Networks (CNNs) are widely used in various domains, including image recognition, medical diagnosis and autonomous driving. Recent advances in dataflow-based CNN accelerators have enabled CNN inference in resource-constrained edge devices. These dataflow accelerators utilize inherent data reuse of convolution layers to process CNN models efficiently. Concealing the architecture of CNN models is critical for privacy and security. This paper evaluates memory-based side-channel information to recover CNN architectures from dataflow-based CNN inference accelerators. The proposed attack exploits spatial and temporal data reuse of the dataflow mapping on CNN accelerators and architectural hints to recover the structure of CNN models. Experimental results demonstrate that our proposed side-channel attack can recover the structures of popular CNN models, namely Lenet, Alexnet, VGGnet16, and YOLOv2.

MLino bench: A comprehensive benchmarking tool for evaluating ML models on edge devices

In today’s rapidly evolving technological landscape, Machine Learning (ML) has become an integral part of our daily lives, ranging from recommendation systems to advanced medical diagnostics and autonomous vehicles. As ML continues to advance, its applications extend beyond conventional boundaries. With the continuous refinement of the models and frameworks, the possibilities for leveraging these technologies into devices that traditionally lacked any form of computational autonomy are ever-expanding. This shift towards embedding ML capabilities directly into edge devices brings new challenges due to the stringent limitations these devices have in terms of memory, power consumption, and cost. The ML models implemented on such devices must find an equilibrium between memory footprint and performance, attaining a classification time that fulfills real-time demands and maintains a similar level of accuracy as the desktop version. Without automated assistance in managing these considerations, the complexity of evaluating multiple models can lead to suboptimal decisions.In this paper, we introduce MLino Bench, an open-source benchmarking tool tailored for assessing lightweight ML models on edge devices with limited resources and capabilities. The tool accommodates various models, frameworks, and platforms, presenting a meticulous design that enables a comprehensive evaluation directly on the target device. It encompasses crucial metrics such as on-target accuracy, classification time, and model size, providing a versatile framework that assists practitioners in decision-making when deploying models to such devices.The tool employs a fully streamlined benchmark flow involving training the ML model in a high-level interpreted language, porting, compiling, flashing, and finally benchmarking on the actual target. Our experimental evaluation of the tool highlights its flexibility in assessing multiple ML models across different model hyperparameters, frameworks, datasets, and embedded platforms. Furthermore, a distinctive advantage compared to state-of-the-art ML benchmarking tools is the inclusion of classical ML models, including Random Forests, Decision Trees, Support Vector Machines, Naive Bayes, and more. This sets our tool apart from others that predominantly emphasize only neural network models. Due to this inclusive approach, our tool facilitates the evaluation of ML models across a broad spectrum of devices, ranging from resource-constrained edge devices to those with medium and advanced computational capabilities.

A comprehensive survey of deep learning-based lightweight object detection models for edge devices

This study concentrates on deep learning-based lightweight object detection models on edge devices. Designing such lightweight object recognition models is more difficult than ever due to the growing demand for accurate, quick, and low-latency models for various edge devices. The most recent deep learning-based lightweight object detection methods are comprehensively described in this work. Information on the lightweight backbone architectures used by these object detectors has been listed. The training and inference processes concerning to deep learning applications on edge devices is being discussed. To raise readers’ awareness of this developing domain, a variety of applications for deep learning-based lightweight object detectors and related utilities have been offered. Designing potent, lightweight object detectors based on deep learning has been suggested as a counter to such problems. On well-known datasets such as MS-COCO and PASCAL-VOC, we thoroughly examine the performance of certain conventional deep learning-based lightweight object detectors.

Towards intelligent environments: human sensing through 3D point cloud

To realize the concept of “Society 5.0”, which seeks to enhance well-being by integrating the Internet of Things to connect people, objects, and environments, intelligent environments are essential. These environments focus on creating sensor-enhanced spaces capable of recognizing and interpreting human behaviors and needs. This paper explores how we achieve intelligent environment by highlighting current technologies and applications of human sensing and tracking that enables such intelligent environments. Specifically, it introduces the development and application of LiDAR-based human sensing and tracking technologies, including the Hitonavi and Hitonavi-μ\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\mu $$\\end{document} systems, designed for multi-person tracking in extensive areas and for enabling person detection and activity recognition on edge devices, respectively. Moreover, the paper explains the services and applications of these methodologies to smart buildings, shopping malls, and elderly care, showcasing their potential to enhance safety, convenience, and understanding of human behavior within intelligent environments. Throughout this paper, we present our vision for creating reliable intelligent environments, which are fundamentally supported by sophisticated human sensing technologies.

A Reconfigurable Ferroelectric Transistor as An Ultra‐Scaled Cell for Low‐Power In‐Memory Data Processing

AbstractCompact in‐memory computing architectures are desirable to embed artificial intelligence (AI) in resource‐restricted edge devices. However, current technologies face limitations in both the area and energy efficiency. Here, a reconfigurable ferroelectric tunnel field‐effect transistor (ferro‐TFET) is presented that can be used as an ultra‐scaled cell for low‐power in‐memory data processing. A gate‐all‐around ferroelectric film is integrated on a vertical nanowire TFET with a gate/source overlapped channel, enabling non‐volatilely reconfigurable anti‐ambipolarity by programming the ferroelectric polarization state. By considering the stored polarization state and reading voltage as inputs, an XNOR operation is achieved in a single‐gate ferro‐TFET. It is shown that the ferro‐TFETs can be implemented in a crossbar array for convolutional frequency filtering whose performance can be evaluated by an impulse‐response method considering the effect of device‐to‐device variation based on statistics. Benefiting from the miniaturized footprint, non‐volatility, and low‐power operation, ferro‐TFETs show promises as a one‐transistor in‐memory computing cell for area‐ and energy‐efficient edge AI applications.

Adaptive heuristic edge assisted fog computing design for healthcare data optimization

Patient care, research, and decision-making are all aided by real-time medical data analysis in today’s rapidly developing healthcare system. The significance of this research comes in the fact that it has the ability to completely change the healthcare system by relocating computing resources closer to the data source, hence facilitating more rapid and accurate analysis of medical data. Latency, privacy concerns, and inability to scale are common in traditional cloud-centric techniques. With their ability to process data close to where it is created, edge and fog computing have the potential to revolutionize medical analysis. The healthcare industry has unique opportunities and problems for the application of edge and fog computing. There must be an emphasis on data security and privacy, workload flexibility, interoperability, resource optimization, and data integration without any interruptions. In this research, it is suggested the Adaptive Heuristic Edge assisted Fog Computing design (AHE-FCD) to solve these issues using a novel architecture meant to improve medical analysis. Together, edge devices and fog nodes may perform distributed data processing and analytics with the help of AHE-FCD. Heuristic algorithms are often employed for optimization issues that establishing an optimum solution using standard approaches is difficult and impossible. Heuristic algorithms utilize search algorithms to explore the search space and identify a result. Improved patient care, medical research, and healthcare process efficiency are all possible to AHE-FCD real-time, low-latency analysis at the edge and fog layers. Improved medical analysis with minimal latency, high reliability, and data privacy are all likely to emerge from the study’s findings. As a result, rather from being centralized, operations in a sophisticated distributed system occur at several end points. That helps the situation quicker to detect possible dangers prior to propagate across the network. The AHE-FCD is a promising breakthrough that moves us closer to the realization of advanced medical analysis systems, where prompt and well-informed decision-making is essential to providing excellent healthcare.

Microservices-Based Resource Provisioning for Multi-User Cloud VR in Edge Networks

Cloud virtual reality (VR) is attracting attention in terms of its lightweight head-mounted display (HMD), providing telepresence and mobility. However, it is still in the research stages due to motion-to-photon (MTP) latency, the need for high-speed network infrastructure, and large-scale traffic processing problems. These problems are expected to be partially solved through edge computing, but the limited computing resource capacity of the infrastructure presents new challenges. In particular, in order to efficiently provide multi-user content such as remote meetings on edge devices, resource provisioning is needed that considers the application’s traffic patterns and computing resource requirements at the same time. In this study, we present a microservice architecture (MSA)-based application to provide multi-user cloud VR in edge computing and propose a scheme for planning an efficient service deployment considering the characteristics of each service. The proposed scheme not only guarantees the MTP latency threshold for all users but also aims to reduce networking and computing resource waste. The proposed scheme was evaluated by simulating various scenarios, and the results were compared to several studies. It was confirmed that the proposed scheme represents better performance metrics than the comparison schemes in most cases from the perspectives of networking, computing, and MTP latency.

Benchmarking analysis of computer vision algorithms on edge devices for the real-time detection of digital dermatitis in dairy cows

Digital dermatitis (DD) is a bovine claw disease responsible for ulcerative lesions on the coronary band of the foot. It causes significant animal welfare and economic losses to the cattle industry. Early detection of DD can lead to prompt treatment and decrease lameness. Current detection and staging methods require a trained individual to evaluate the interdigital space on each foot for clinical signs of DD. Computer vision (CV), a type of artificial intelligence for image analysis, has demonstrated promising results on object detection tasks. However, farms require robust solutions that can be deployed in harsh conditions including dust, debris, humidity, precipitation, other equipment issues. The study aims to train, deploy, and benchmark DD detection models on edge devices. Images were collected from commercial dairy farms with the camera facing the interdigital space on the plantar surface of the foot. Images were scored for M-stages of DD by a trained investigator using the M-stage DD classification system. Models were trained to detect and score DD lesions and embedded on an edge device. The Tiny YOLOv4 model deployed on a CV specific integrated camera module connected to a single board computer achieved a mean average precision (mAP) of 0.895, an overall prediction accuracy of 0.873, and a Cohen’s kappa of 0.830 for agreement between the computer vision model and the trained investigator. The model reached a final inference speed of 40 frames per second (FPS) and ran stably without any interruptions. The CV model was able to detect DD lesions on an edge device with high performance and speed. The CV tool can be used for early detection and prompt treatment of DD in dairy cows. Real-time detection of DD on edge device will improve health outcomes, while simultaneously decreasing labor costs. We demonstrate that the deployed model can be a low-power and portable solution for real-time detection of DD on dairy farms. This result is a step towards applying CV algorithms to veterinary medicine and implementing real-time detection of health outcomes in precision farming.

Federated dual averaging learning algorithm with delayed gradients for composite optimization

Federated learning offers a promising paradigm for training machine learning models in distributed edge networks. This paper focuses on a critical aspect of federated learning, specifically examining the influence of communication delays between the server and edge devices. We study the problem of federated composite optimization (FCO) in the scenario where communication delays are introduced due to client-sides. To tackle this issue, we propose a federated dual averaging learning algorithm that incorporates delayed gradients. We conduct a thorough convergence analysis of our algorithm taking into account the impact of the delayed gradient information. Our analysis demonstrates that under the assumption of bounded expected random delays, the algorithm exhibits the asymptotic convergence towards the optimal value. Furthermore, we present explicit convergence rates of the proposed algorithm for fixed and random delays. Our results show that the impact of delays on FCO problems with smooth loss functions gradually becomes negligible. Finally, to demonstrate the efficacy of the proposed algorithm, we conduct numerical simulations on the FEMNIST dataset utilizing ℓ1-regularized logistic regression. These experimental results confirm our findings that the algorithm converges effectively even in the presence of delays.

A lightweight and continuous dimensional emotion analysis system of facial expression recognition under complex background

Facial expression recognition technology has a brilliant prospect in applying the Internet of Things systems. Concerning the limited hardware computing capability and high real-time processing requirements, this paper proposes a lightweight emotion analysis system based on edge computing, which could be deployed on edge devices. To further improve the accuracy of dimensional emotion analysis, we propose a modified network structure of MobileNetV3 to measure the intensity of dimensional emotion. The optimization scheme includes introducing the improved efficient channel attention mechanism and the feature pyramid network, adjusting the structure of the model, and optimizing the loss function. Furthermore, the system uses Intel’s OpenVINO toolkit to make the model more suitable for stable operation and provides an operable human–computer interaction interface. The experimental results show that the system has the advantages of few parameters, high recognition accuracy, and low latency. The optimized network size is reduced by 67% compared with the original, the root mean square error of potency and activation is 0.413 and 0.389, and the latency is up to 9ms in Myriad. This work meets the requirements of practical applications and has essential significance with the demand for continuous dimensional emotion analysis. The source code is available at https://github.com/SCNU-RISLAB/Lightweight-and-Continuous-Dimensional-Emotion-Analysis-System-of-Facial-Expression-Recognition/.