On-device Machine Learning Research Articles

An empirical study of AI techniques in mobile applications

The integration of artificial intelligence (AI) into mobile applications has significantly transformed various domains, enhancing user experiences and providing personalized services through advanced machine learning (ML) and deep learning (DL) technologies. AI-driven mobile apps typically refer to applications that leverage ML/DL technologies to perform key tasks such as image recognition and natural language processing. Despite existing research exploring how mobile apps exploit AI techniques, they have the following main limitations: (1) Most existing studies focus on DL-based apps, with limited research on ML-based apps. (2) Existing research typically focuses on investigating the apps and the technologies utilized in the apps, lacking user-level analysis. (3) The number of apps studied is limited, with only 1,000 to 2,000 ML/DL apps identified after filtering. To fill the gap, in this paper, we conducted the most extensive empirical study on AI applications, exploring on-device ML apps, on-device DL apps, and AI service-supported (cloud-based) apps. Our study encompasses 56,682 real-world AI applications, focusing on three crucial perspectives: (1) Application analysis, where we analyze the popularity of AI apps and investigate the update states of AI apps; (2) Framework and model analysis, where we analyze AI framework usage and AI model protection; (3) User analysis, where we examine user privacy protection and user review attitudes. Our study has strong implications for AI app developers, users, and AI R&D. On one hand, our findings highlight the growing trend of AI integration in mobile applications, demonstrating the widespread adoption of various AI frameworks and models. On the other hand, our findings emphasize the need for robust model protection to enhance app security. Additionally, our study highlights the importance of user privacy and presents user attitudes towards the AI technologies utilized in current AI apps. We provide our AI app dataset (currently the most extensive AI app dataset) as an open-source resource for future research on AI technologies utilized in mobile applications.

On-device Online Learning and Semantic Management of TinyML Systems

Recent advances in Tiny Machine Learning (TinyML) empower low-footprint embedded devices for real-time on-device Machine Learning (ML). While many acknowledge the potential benefits of TinyML, its practical implementation presents unique challenges. This study aims to bridge the gap between prototyping single TinyML models and developing reliable TinyML systems in production: (1) Embedded devices operate in dynamically changing conditions. Existing TinyML solutions primarily focus on inference, with models trained offline on powerful machines and deployed as static objects. However, static models may underperform in the real world due to evolving input data distributions. We propose online learning to enable training on constrained devices, adapting local models toward the latest field conditions. (2) Nevertheless, current on-device learning methods struggle with heterogeneous deployment conditions and the scarcity of labeled data when applied across numerous devices. We introduce federated meta-learning incorporating online learning to enhance model generalization, facilitating rapid learning. This approach ensures optimal performance among distributed devices by knowledge sharing. (3) Moreover, TinyML’s pivotal advantage is widespread adoption. Embedded devices and TinyML models prioritize extreme efficiency, leading to diverse characteristics ranging from memory and sensors to model architectures. Given their diversity and non-standardized representations, managing these resources becomes challenging as TinyML systems scale up. We present semantic management for the joint management of models and devices at scale. We demonstrate our methods through a basic regression example and then assess them in three real-world TinyML applications: handwritten character image classification, keyword audio classification, and smart building presence detection. The results confirm the effectiveness of our approaches from various perspectives, such as accuracy improvement, resource savings, and engineering effort reduction.

Human motion activity recognition and pattern analysis using compressed deep neural networks

ABSTRACT This work presents an on-device machine learning model with the ability to identify different mobility gestures called human activity recognition (HAR), which includes running, walking, squatting, jumping, and others. The data is collected through an Arduino Nano 33 BLE Sense board with a sampling rate of 119 Hz, which is embedded with an Inertial Measurement Unit (IMU) sensor. The same board is used as a microcontroller to identify human gestures by developing an end-to-end edge computing application. A deep neural network model is trained and then compressed for deployment on the board to create a self-contained, embedded device capable of identifying the type of gesture performed. Three deep learning models, namely Multi-Layer Perceptron (MLP), Convolutional Neural Network – Long Short Term Memory (CNN-LSTM) & CNN-Gated Recurrent Unit (CNN-GRU), are evaluated in the identification of the mobility gestures. The observed accuracy of the models is 96%, 97.1% and 97.8%, respectively, MLP, CNN-LSTM & CNN-GRU across the different gesture categories. The study shows the utility of embedded devices with deep neural network-based models, which can provide low cost, minimal power usage, and meet data privacy requirements in HAR.

Real-time Accessibility Optimization: Enhancing Web Accessibility Through on-device Machine Learning and Automated Adaptation

Real-time Accessibility Optimization: Enhancing Web Accessibility Through on-device Machine Learning and Automated Adaptation

Edge AI and On-Device Machine Learning For Real Time Processing

Edge Artificial Intelligence (Edge AI) and on-device machine learning represent significant advancements in computing paradigms, enabling real-time data processing directly at the edge of the network. This approach minimizes the need for centralized cloud-based processing, thereby reducing latency, enhancing privacy, and improving operational efficiency. This paper provides a comprehensive review of Edge AI and on-device machine learning, focusing on their applications, challenges, and future directions. Key applications include smart home devices, healthcare monitoring, autonomous vehicles, and smart city infrastructure, where local data processing facilitates immediate responses and decision-making. However, deploying AI models on edge devices introduces challenges related to model compression, limited computational resources, and power constraints. Security and privacy concerns also arise, necessitating robust data protection and privacy-preserving techniques. Furthermore, the paper explores emerging technologies such as 5G integration and advances in AI hardware, which are expected to drive future developments in Edge AI. Research directions are highlighted, emphasizing the need for new algorithms tailored to edge environments and enhancements in system robustness and reliability. By addressing these challenges and leveraging emerging technologies, Edge AI is poised to transform a wide range of sectors, offering enhanced real-time processing capabilities and paving the way for innovative applications.

A review of on-device machine learning for IoT: An energy perspective

Recently, there has been a substantial interest in on-device Machine Learning (ML) models to provide intelligence for the Internet of Things (IoT) applications such as image classification, human activity recognition, and anomaly detection. Traditionally, ML models are deployed in the cloud or centralized servers to take advantage of their abundant computational resources. However, sharing data with the cloud and third parties degrades privacy and may cause propagation delay in the network due to a large amount of transmitted data impacting the performance of real-time applications. To this end, deploying ML models on-device (i.e., on IoT devices), in which data does not need to be transmitted, becomes imperative. However, deploying and running ML models on already resource-constrained IoT devices is challenging and requires intense energy consumption. Numerous works have been proposed in the literature to address this issue. Although there are considerable works that discuss energy-aware ML approaches for on-device implementation, there remains a gap in the literature on a comprehensive review of this subject. In this paper, we provide a review of existing studies focusing on-device ML models for IoT applications in terms of energy consumption. One of the key contributions of this study is to introduce a taxonomy to define approaches for employing energy-aware on-device ML models on IoT devices in the literature. Based on our review in this paper, our key findings are provided and the open issues that can be investigated further by other researchers are discussed. We believe that this study will be a reference for practitioners and researchers who want to employ energy-aware on-device ML models for IoT applications.

Privacy-Preserving Federal Learning Chain for Internet of Things

The expansion of Internet of Things (IoT) spawns large on-device machine learning demands, while the machine learning can be a hard task for resource constrained IoT terminals with fragmented dataset. Federal learning (FL), which aims to build a joint model across multiple devices, IoT-FL has now become a promising path for learning on terminals. In broad FL fields, current server-client pattern cannot jump out of the third party self-trustless problem, and recent researches suggest that even sharing training results may also reveal the raw data sets. Homomorphic encryption (HE) is a powerful method in privacy preserving, while so far it is hard to applied HE into multi-party computing (MPC) scenarios including FL. Combining with the existing IoT architecture, in this paper, we customize a scheme (FL chain) dedicated for the privacy and trustiness issues in IoT-FL scenarios, which integrates blockchain smart contract and HE. Differ from traditional schemes, our FL chain is highly adaptive with current IoT architecture and it is the first scheme that applied HE into IoT-FL privacy preserving. Theoretical analysis and experimental results prove the feasibility of FL chain.

Tiny, Always-on, and Fragile: Bias Propagation through Design Choices in On-device Machine Learning Workflows

Billions of distributed, heterogeneous, and resource constrained IoT devices deploy on-device machine learning (ML) for private, fast, and offline inference on personal data. On-device ML is highly context dependent and sensitive to user, usage, hardware, and environment attributes. This sensitivity and the propensity toward bias in ML makes it important to study bias in on-device settings. Our study is one of the first investigations of bias in this emerging domain and lays important foundations for building fairer on-device ML. We apply a software engineering lens, investigating the propagation of bias through design choices in on-device ML workflows. We first identify reliability bias as a source of unfairness and propose a measure to quantify it. We then conduct empirical experiments for a keyword spotting task to show how complex and interacting technical design choices amplify and propagate reliability bias . Our results validate that design choices made during model training, like the sample rate and input feature type, and choices made to optimize models, like light-weight architectures, the pruning learning rate, and pruning sparsity, can result in disparate predictive performance across male and female groups. Based on our findings, we suggest low effort strategies for engineers to mitigate bias in on-device ML.

An on-device machine learning assisted system for unobtrusive cardiac auscultation

This paper uses on-device machine learning to present a microcontroller-based cardiac auscultation system for heart sound classification. Real-time analysis of heart sounds for the identification of abnormalities can prove important in early diagnosis of heart disease. However, this process of cardiac auscultation faces two important challenges, one of which is the form factor of the device that is being used for the auscultation/analysis and the other being poor classification performance due to the presence of environment noise in the recorded heart sound. This paper, therefore, develops a signal processing pipeline to extract three distinct two-dimensional features from a recorded heart sound and numerically analyses the effect of environment noise on them. Henceforth, a Convolutional Neural Network (CNN) based classification layer is proposed to perform real-time analysis on heart sounds. The classifier is seen to achieve a peak accuracy of 91.2%, a precision of 91.2%, an F-1 score of 90.2%, with Youden’s index of 0.847 and discriminant power of 2.980. This classification layer then forms the basis of development for a portable device that can perform real-time heart sound classification while consuming around 68.34 mW of power and is also impervious to environmental distortions.

A novel image model for vehicle classification in restricted areas using on-device machine learning

A novel image model for vehicle classification in restricted areas using on-device machine learning

Indolizine-based fluorescent compounds array for noninvasive monitoring of glucose in bio-fluids using on-device machine learning

Monitoring blood sugar level is highly important for managing diabetes and preventing complications. Herein, we describe a paper-based portable assay system for highly accurate monitoring of glucose levels in tears. The system combines enzyme reactions, the perturbation of fluorescence properties, automated image analysis, and pattern recognition powered by a machine-learning algorithms in a single assay without extra manual steps. A small size (20 × 20 mm) assay system was prepared simply by printing black wax crosswise on cellulose paper, melting the wax with heat, and depositing four different fluorescent compounds and glucose oxidase at the test zones. Applying a small drop of sample in the center zone of the system evenly distributes the samples to the test zone via the paper capillary process and eventually induces a generation of unique fluorescence pattern that is specific to glucose. Two models are used to analyze image data collected from the fluorescent compounds array to make accurate estimations on glucose concentration: a random forest model that discretely classifies the concentration with respect to the training data granularity and an SVM regression model that estimates glucose concentration at interpolated finer granularity. These models are combined with feature engineering techniques for efficient sensor data processing on resource limited mobile platforms. As a result, using the random forest approach, glucose concentration in artificial tears can be accurately classified with 96.7% accuracy (0.2 mM intervals) (MSE of 0.060 mM) and the SVM regression model results in an MSE of 0.026 mM.

A Review of On-Device Machine Learning for Iot: An Energy

A Review of On-Device Machine Learning for Iot: An Energy

Energy consumption of on-device machine learning models for IoT intrusion detection

Recently, Smart Home Systems (SHSs) have gained enormous popularity with the rapid development of the Internet of Things (IoT) technologies. Besides offering many tangible benefits, SHSs are vulnerable to attacks that lead to security and privacy concerns for SHS users. Machine learning (ML)-based Intrusion Detection Systems (IDS) are proposed to address such concerns. Conventionally, ML models are trained and tested on computationally powerful platforms such as cloud services. Nevertheless, the data shared with the cloud is vulnerable to privacy attacks and causes latency, which decreases the performance of real-time applications like intrusion detection systems. Therefore, on-device ML models, in which the user data is kept locally, have emerged as promising solutions to ensure the security and privacy of the data for real-time applications. However, performing ML tasks requires high energy consumption. To the best of our knowledge, no study has been conducted to analyze the energy consumption of ML-based IDS. Therefore, in this paper, we perform a comparative analysis of on-device ML algorithms in terms of energy consumption for IoT intrusion detection applications. For a thorough analysis, we study the training and inference phases separately. For training, we compare the cloud computing-based ML, edge computing-based ML, and IoT device-based ML approaches. For the inference, we evaluate the TinyML approach to run the ML algorithms on tiny IoT devices such as Micro Controller Units (MCUs). Comparative performance evaluations show that deploying the Decision Tree (DT) algorithm on-device gives better results in terms of training time, inference time, and power consumption.

PervasiveFL: Pervasive Federated Learning for Heterogeneous IoT Systems

Federated learning (FL) has been recognized as a promising collaborative on-device machine learning method in the design of Internet of Things (IoT) systems. However, most existing FL methods fail to deal with IoT applications that contain a variety of IoT devices equipped with different types of neural network (NN) models. This is because traditional FL methods assume that local models on devices should have the same architecture as the global model on cloud. To address this problem, we propose a novel framework named PervasiveFL that enables efficient and effective FL among heterogeneous IoT devices. Without modifying original local models, PervasiveFL installs one lightweight NN model named modellet on each device. By using the deep mutual learning (DML) and our entropy-based decision gating (EDG) method, modellets and local models can selectively learn from each other through soft labels using locally captured data. Meanwhile, since modellets are of the same architecture, the learned knowledge by modellets can be shared among devices in a traditional FL manner. In this way, PervasiveFL can be pervasively applied to any heterogeneous IoT system. Comprehensive experimental results on four well-known datasets show that PervasiveFL can not only pervasively enable FL among heterogeneous devices within a large-scale IoT system, but also significantly enhance the inference accuracy of heterogeneous IoT devices with low communication overhead.

SmartAct: Energy Efficient and Real-Time Hand-to-Mouth Gesture Detection Using Wearable RGB-T.

Researchers have been leveraging wearable cameras to both visually confirm and automatically detect individuals' eating habits. However, energy-intensive tasks such as continuously collecting and storing RGB images in memory, or running algorithms in real-time to automate detection of eating, greatly impacts battery life. Since eating moments are spread sparsely throughout the day, battery life can be mitigated by recording and processing data only when there is a high likelihood of eating. We present a framework comprising a golf-ball sized wearable device using a low-powered thermal sensor array and real-time activation algorithm that activates high-energy tasks when a hand-to-mouth gesture is confirmed by the thermal sensor array. The high-energy tasks tested are turning on the RGB camera (Trigger RGB mode) and running inference on an on-device machine learning model (Trigger ML mode). Our experimental setup involved the design of a wearable camera, 6 participants collecting 18 hours of data with and without eating, the implementation of a feeding gesture detection algorithm on-device, and measures of power saving using our activation method. Our activation algorithm demonstrates an average of at-least 31.5% increase in battery life time, with minimal drop of recall (5%) and without impacting the accuracy of detecting eating (a slight 4.1% increase in F1-Score).

Automatic sleep stage classification: A light and efficient deep neural network model based on time, frequency and fractional Fourier transform domain features

This work proposed a novel method for automatic sleep stage classification based on the time, frequency, and fractional Fourier transform (FRFT) domain features extracted from a single-channel electroencephalogram (EEG). Bidirectional long short-term memory was applied to the proposed model to train it to learn the sleep stage transition rules according to the American Academy of Sleep Medicine's manual for automatic sleep stage classification. Results indicated that the features extracted from the fractional Fourier-transformed single-channel EEG may improve the performance of sleep stage classification. For the Fpz-Cz EEG of Sleep-EDF with 30 s epochs, the overall accuracy of the model increased by circa 1% with the help of the FRFT domain features and even reached 81.6%. This work thus made the application of FRFT to automatic sleep stage classification possible. The parameters of the proposed model measured 0.31 MB, which are 5% of those of DeepSleepNet, but its performance is similar to that of DeepSleepNet. Hence, the proposed model is a light and efficient model based on deep neural networks, which also has a prospect for on-device machine learning.

Byzantine-Robust and Efficient Federated Learning for the Internet of Things

The expansion of Internet-of-Things (IoT) devices with a wealth of generated data opens up new possibilities for intelligent IoT applications (i.e., smart home and smart transportation), but the increasing concern about data privacy makes the enabling force of intelligent IoT, machine learning (ML), harder to deploy. Federated learning (FL), an emerging distributed ML paradigm that allows on-device ML model training without sharing private raw data, is becoming a promising solution to achieve collaborative intelligence in IoT. However, the privacy-preserving design of FL makes it vulnerable to Byzantine workers who behave arbitrarily and send poisonous model updates to the central server to corrupt the joint learning process. Moreover, traditional FL suffers from massive communications overhead due to the large number of training rounds to convergence with non-independent identically distributed (non-i.i.d.) data across clients. To address these problems, we propose a novel robust aggregation rule for FL, federated mutual information (FedMI) which leverages the mutual information between clients to build resilience to Byzantine workers and accelerate the convergence speed. We perform experiments over the non-i.i.d FEMNIST dataset under different adversarial settings. Experimental results demonstrate the effectiveness of the proposed FedMI: much faster convergence speed and higher defense capability when compared to the state-of-the-art robust aggregation rules for FL.

TinyML-Based Fall Detection for Connected Personal Mobility Vehicles

A new wave of electric vehicles for personal mobility is currently crowding public spaces. They offer a sustainable and efficient way of getting around in urban environments, however, these devices bring additional safety issues, including serious accidents for riders. Thereby, taking advantage of a connected personal mobility vehicle, we present a novel on-device Machine Learning (ML)-based fall detection system that analyzes data captured from a range of sensors integrated on an on-board unit (OBU) prototype. Given the typical processing limitations of these elements, we exploit the potential of the TinyML paradigm, which enables embedding powerful ML algorithms in constrained units. We have generated and publicly released a large dataset, including real riding measurements and realistically simulated falling events, which has been employed to produce different TinyML models. The attained results show the good operation of the system to detect falls efficiently using embedded OBUs. The considered algorithms have been successfully tested on mass-market low-power units, implying reduced energy consumption, flash footprints and running times, enabling new possibilities for this kind of vehicles.

Paving the Way Towards Collective Intelligence at the IoT Edge

With the scale of data generated and the increasing computing capabilities at the IoT Edge, moving intelligence to end-devices and gateways in IoT networks is within the realms of possibility. There are tremendous hopes on how self-awareness in IoT networks could be helpful to solve a variety of smart city use case problems. Self-awareness implies self-teaching; the network devices autonomously become smarter with AI. Collective intelligence is a subset of self-awareness but not yet formalized and exploited at the Edge of IoT networks. In this paper, current mathematical models for Collective Intelligence are studied and compared. Their suitability for on-device machine learning deployment is also explored. Moreover, it attempts to lay down a framework using graph theory coupled with artificial neural networks to model a collective intelligence system applicable to a specific scenario.

Inference Over Wireless IoT Links With Importance-Filtered Updates

We consider a communication cell comprised of Internet-of-Things (IoT) nodes transmitting to a common Access Point (AP). The nodes in the cell are assumed to generate data samples periodically, which are to be transmitted to the AP. The AP hosts a machine learning model, such as a neural network, which is trained on the received data samples to make accurate inferences. We address the following tradeoff: The more often the IoT nodes transmit, the higher the accuracy of the inference made by the AP, but also the higher the energy expenditure at the IoT nodes. We propose a data filtering scheme employed by the IoT nodes, which we refer to as distributed importance filtering in order to filter out redundant data samples already at the IoT nodes. The IoT nodes do not have large on-device machine learning models and the data filtering scheme operates under periodic instructions from the model placed at the AP. The proposed scheme is evaluated using neural networks on a benchmark machine vision dataset, as well as in two practical scenarios: leakage detection in water distribution networks and air-pollution detection in urban areas. The results show that the proposed scheme offers significant benefits in terms of network longevity as it preserves the devices' resources, whilst maintaining high inference accuracy. Our approach reduces the the computational complexity for training the model and obviates the need for data pre-processing, which makes it highly applicable in practical IoT scenarios.