Resource-constrained Mobile Devices Research Articles

Cross-Attention Enhanced Pyramid Multi-Scale Networks for Sensor-Based Human Activity Recognition.

Human Activity Recognition (HAR) has recently attracted widespread attention, with the effective application of this technology helping people in areas such as healthcare, smart homes, and gait analysis. Deep learning methods have shown remarkable performance in HAR. A pivotal challenge is the trade-off between recognition accuracy and computational efficiency, especially in resource-constrained mobile devices. This challenge necessitates the development of models that enhance feature representation capabilities without imposing additional computational burdens. Addressing this, we introduce a novel HAR model leveraging deep learning, ingeniously designed to navigate the accuracy-efficiency trade-off. The model comprises two innovative modules: 1) Pyramid Multi-scale Convolutional Network (PMCN), which is designed with a symmetric structure and is capable of obtaining a rich receptive field at a finer level through its multiscale representation capability; 2) Cross-Attention Mechanism, which establishes interrelationships among sensor dimensions, temporal dimensions, and channel dimensions, and effectively enhances useful information while suppressing irrelevant data. The proposed model is rigorously evaluated across four diverse datasets: UCI, WISDM, PAMAP2, and OPPORTUNITY. Additional ablation and comparative studies are conducted to comprehensively assess the performance of the model. Experimental results demonstrate that the proposed model achieves superior activity recognition accuracy while maintaining low computational overhead.

CNN-Transformer hybrid network for concrete dam crack patrol inspection

Regular patrol inspection of concrete dams can detect cracks at an early stage. However, conventional crack segmentation models based on deep learning (DL) are difficult to be deployed in resource-constrained mobile devices due to the large number of parameters. This paper describes a lightweight semantic segmentation model, termed as CrackTrNet, for images of concrete dam cracks. CrackTrNet is a hybrid U-shaped model based on convolutional neural network (CNN) and Vision Transformer. The CNN is adopted to extract low-level visual features and the Transformer focuses on learning the global contextual information. The results demonstrate that its segmentation accuracy can reach 97.60%, while the model size is only 34.86 MB, which is 66.12%–87.85% lower than that of current mainstream DL-based models. To make the model more practical, a crack inspection mobile application (APP) is developed using Android Studio. The integration of lightweight CrackTrNet and APP can effectively assist the intelligent inspection of dam cracks to ensure structural safety.

Evolutionary Multitasking for Costly Task Offloading in Mobile-Edge Computing Networks

The offloading of computation-intensive tasks to an edge server near resource-constrained mobile devices can provide improved application performance and user experience. However, with the rapid growth of mobile devices connected to the edge server, it is challenging to directly obtain an optimal task offloading scheme due to increasing computational cost and problem scale. In this study, we model the costly task offloading problem (CTOP) in mobile edge computing networks to achieve efficient joint optimization of energy consumption and processing latency for mobile devices. Inspired by the success of evolutionary multitasking in solving complex optimization problems by leveraging the experience of simple optimization problems, we develop a novel multitasking framework whose effectiveness is demonstrated in solving the CTOP. In this framework, auxiliary tasks are created to optimize the local processing overhead and the edge processing overhead of task offloading. On this basis, we propose an effective multitask evolutionary algorithm that includes segmented knowledge transfer and auxiliary task update. Specifically, source and extended decision variables are considered as different knowledge to be utilized, while the auxiliary tasks are allowed to be updated dynamically. Related knowledge that is learned from cheap and simple auxiliary tasks promotes the evolutionary search for CTOP. Experimental results verify the effectiveness of knowledge transfer. Compared to existing multitasking and single-tasking algorithms, the proposed algorithm shows competitive performance in CTOP instances and achieves better comprehensive performance in terms of energy consumption and processing latency.

Attribute-Based Encryption With Reliable Outsourced Decryption in Cloud Computing Using Smart Contract

Outsourcing the heavy decryption computation to a cloud service provider has been a promising solution for a resource-constrained mobile device to deploy an attribute-based encryption scheme. However, the current attribute based encryption with outsourced decryption schemes only enable the mobile device to verify whether the cloud service provider has returned a correct decryption result, they lack a mechanism to enable the cloud service provider to escape from a mobile device's wrong claim if it has returned a correct decryption result. This paper, for the first time, proposes an attribute based encryption with reliable outsourced decryption scheme using the blockchain smart contract. In the proposed scheme, not only can the mobile device verify whether the cloud service provider has returned a correct decryption result, but also the cloud service provider can escape from a wrong claim if the returned decryption result is correct. Moreover, our system achieves the fairness property, which means the cloud service provider can get the reward from the mobile device if and only if it has returned a correct decryption result. Finally, we conduct an implementation to demonstrate that the proposed scheme is practical and efficient.

Acheiving United Nations' SDG3 Through Empowering Health Artificial Intelligence on Resource-Constrained Mobile Devices Without Connectivity

At least half of the world's population do not have access to essential health services. Worse, large numbers of households are being pushed into poverty because they must pay for health care out of their own pockets.

Empowering Rural Healthcare: MobileNet-Driven Deep Learning for Early Diabetic Retinopathy Detection in Nepal

Diabetic retinopathy (DR) is a pervasive public health challenge, particularly in underserved rural regions like Nepal. Timely detection and management of DR are imperative to prevent vision impairment and blindness among individuals with diabetes. However, the scarcity of advanced medical facilities and trained ophthalmologists in remote areas hampers early diagnosis. This study addresses this critical issue by harnessing deep learning technology, specifically MobileNet, to develop an accessible and cost-effective solution for the early detection of diabetic retinopathy in Nepal's rural communities. The research aims to design and implement a robust DR detection system capable of operating on resource-constrained mobile devices. MobileNet, renowned for its efficiency and suitability for mobile applications, serves as the cornerstone of the solution. Through fine-tuning MobileNet on a comprehensive dataset comprising diverse DR manifestations, the model learns to accurately classify the presence of retinopathy. Recognizing the challenge of limited internet connectivity in rural Nepal, the project explores on-device inference techniques. This enables the model to run directly on mobile devices, minimizing the dependence on continuous internet access and making the diagnostic tool available even in remote regions. The anticipated impact of this project is twofold: firstly, it empowers healthcare workers in rural Nepal to provide early and precise DR screenings, facilitating timely intervention and reducing avoidable blindness. Secondly, it demonstrates the feasibility of deploying deep learning and mobile technology to address healthcare challenges in resource-limited settings beyond diabetic retinopathy.By harnessing MobileNet-based deep learning, this solution has the potential to significantly enhance healthcare accessibility and contribute to the overall well-being of the diabetic population in underserved areas.

Cattle face detection method based on channel pruning YOLOv5 network and mobile deployment

The detection and identification of individual cattle plays an integral role in precision feeding and insurance claims, among others. Most current research is based on high-performance computing devices, which limits the application of deep learning techniques to mobile terminals. To this end, in this paper, we propose a channel-pruned YOLOv5 network-based method for cattle face detection on mobile terminals, referred to as NS-YOLO. First, the original model is sparsely trained and a sparse regularization penalty term is applied to the BN layers, then the corresponding mask values are labeled according to different weight thresholds, and the channels are pruned with global thresholds. Second, the detection precision is recovered by fine-tuning the model. Finally, the NCNN forward inference framework is used to quantize the model and an Android-based cattle face detection application is developed. Experimental results show that the model size, number of parameters and FLOPs are reduced by 86.10%, 88.19% and 63.25%, respectively, and the inference time is reduced by 35.53% compared to the original model, while mAP0.5 is reduced by only 1.6%. In particular, the 16-bit quantized model reduces the model size by 93.97% and the inference time by 34.97% compared to the original model on the mobile side. The results show that the proposed method can be easily deployed in resource-constrained mobile devices and has great advantages in practical engineering applications.

A Lightweight YOLOv5 Real-time Mask-wearing Detection Algorithm for the Post-pandemic Era

Since the outbreak of the COVID-19 pandemic, standardized mask-wearing has become a powerful measure to combat the epidemic. Although the epidemic has been brought under control, vigilance in densely populated areas remains essential. Manual supervision is not only inefficient but also increases the risk of infection among relevant personnel. As a result, this paper proposes a lightweight real-time mask-wearing detection algorithm to monitor mask-wearing in crowds in real time. Built upon the YOLOv5 framework, the proposed algorithm replaces the backbone feature extraction network of the original model with an improved EfficientNetV2, reducing the model's parameter count and enhancing accuracy. The introduction of the ECA module in place of the SE module in the EfficientNetV2 network, coupled with the substitution of DIoU-NMS for the weighted NMS in the original model, further reduces model parameters and improves convergence. Additionally, this approach enhances the detection of occluded objects. Experimental results based on a publicly collected mask dataset demonstrate that the proposed algorithm reduces the model's parameter count by 44.7%, achieves a mAP of 95.3%, and attains an inference speed of 270.3 FPS. The algorithm introduced in this paper effectively identifies whether individuals are wearing masks correctly. Its lightweight nature makes it suitable for deployment on resource-constrained mobile devices, aligning well with post-pandemic epidemic prevention and control efforts in the era to come.

Microservice Application Scheduling in Multi-Tiered Fog-Computing-Enabled IoT.

Fog computing extends mobile cloud computing facilities at the network edge, yielding low-latency application execution. To supplement cloud services, computationally intensive applications can be distributed on resource-constrained mobile devices by leveraging underutilized nearby resources to meet the latency and bandwidth requirements of application execution. Building upon this premise, it is necessary to investigate idle or underutilized resources that are present at the edge of the network. The utilization of a microservice architecture in IoT application development, with its increased granularity in service breakdown, provides opportunities for improved scalability, maintainability, and extensibility. In this research, the proposed schedule tackles the latency requirements of applications by identifying suitable upward migration of microservices within a multi-tiered fog computing infrastructure. This approach enables optimal utilization of network edge resources. Experimental validation is performed using the iFogSim2 simulator and the results are compared with existing baselines. The results demonstrate that compared to the edgewards approach, our proposed technique significantly improves the latency requirements of application execution, network usage, and energy consumption by 66.92%, 69.83%, and 4.16%, respectively.

Performance Analysis of Task Offloading in Mobile Edge Cloud Computing for Brain Tumor Classification Using Deep Learning

The increasing prevalence of brain tumors necessitates accurate and efficient methods for their identification and classification. While deep learning (DL) models have shown promise in this domain, their computational demands pose challenges when deploying them on resource-constrained mobile devices. This paper investigates the potential of Mobile Edge Computing (MEC) and Task Offloading to improve the performance of DL models for brain tumor classification. A comprehensive framework was developed, considering the computational capabilities of mobile devices and edge servers, as well as communication costs associated with task offloading. Various factors influencing task offloading decisions were analyzed, including model size, available resources, and network conditions. Results demonstrate that task offloading effectively reduces the time and energy required to process DL models for brain tumor classification, while maintaining accuracy. The study emphasizes the need to balance computation and communication costs when deciding on task offloading. These findings have significant implications for the development of efficient mobile edge computing systems for medical applications. Leveraging MEC and Task Offloading enables healthcare professionals to utilize DL models for brain tumor classification on resource-constrained mobile devices, ensuring accurate and timely diagnoses. These technological advancements pave the way for more accessible and efficient medical solutions in the future.

SCMA-Enabled Multi-Cell Edge Computing Networks: Design and Optimization

Multi-access edge computing (MEC) is regarded as a promising approach for providing resource-constrained mobile devices with computing resources through task offloading. Sparse code multiple access (SCMA) is a code-domain non-orthogonal multiple access (NOMA) scheme that can meet the demands of multi-cell MEC networks for high data transmission rates and massive connections. In this paper, we propose an optimization framework for SCMA-enabled multi-cell MEC networks. The joint resource allocation and computation offloading problem is formulated to minimize the system cost, which is defined as the weighted energy cost and latency. Due to the nonconvexity of the proposed optimization problem induced by the coupled optimization variables, we first propose an algorithm based on the block coordinate descent (BCD) method to iteratively optimize the transmit power and edge computing resources allocation by deriving closed-form solutions, and further develop an improved low-complexity simulated annealing (SA) algorithm to solve the computation offloading and multi-cell SCMA codebook allocation problem. To solve the problem of partial state observation and timely decision-making in long-term optimization environment, we put forward a multiagent deep deterministic policy gradient (MADDPG) algorithm with centralized training and distributed execution. Furthermore, we extend the framework to the partial offloading case and propose an algorithm based on alternating convex search for solving the task offloading ratio. Numerical results show that the proposed multi-cell SCMA-MEC scheme achieves lower energy consumption and system latency in comparison to the orthogonal frequency division multiple access (OFDMA) and power-domain (PD) NOMA techniques.

Improved YOLOv5-Based Lightweight Object Detection Algorithm for People with Visual Impairment to Detect Buses

Object detection is crucial for individuals with visual impairment, especially when waiting for a bus. In this study, we propose a lightweight and highly accurate bus detection model based on an improved version of the YOLOv5 model. We propose integrating the GhostConv and C3Ghost Modules into the YOLOv5 network to reduce the number of parameters and floating-point operations per second (FLOPs), ensuring detection accuracy while reducing the model parameters. Following that, we added the SimSPPF module to replace the SPPF in the YOLOv5 backbone for increased computational efficiency and accurate object detection capabilities. Finally, we developed a Slim scale detection model by modifying the original YOLOv5 structure in order to make the model more efficient and faster, which is critical for real-time object detection applications. According to the experimental results, the Improved-YOLOv5 outperforms the original YOLOv5 in terms of the precision, recall, and mAP@0.5. Further analysis of the model complexity reveals that the Improved-YOLOv5 is more efficient due to fewer FLOPS, with fewer parameters, less memory usage, and faster inference time capabilities. The proposed model is smaller and more feasible to implement in resource-constrained mobile devices and a promising option for bus detection systems.

CPU Frequency Scaling Optimization in Sustainable Edge Computing

Sustainable edge computing (SEC) is a promising technology that can reduce energy consumption and computing latency for the mobile Internet of things (IoT). By collecting renewable energy such as solar or wind energy from the environment, a sustainable cloudlet outside the electric grid can provide powerful computing capabilities for resource-constrained mobile IoT devices. In the real world, the density of sustainable energy can vary significantly over time. Therefore, the SEC cloudlet needs to dynamically adjust the clock frequency to balance energy consumption and computing latency. In this paper, we consider the limited energy storage of the cloudlet and the dynamic intensity of renewable energy, and then develop offline optimal CPU frequency scaling policies that (a) maximize the computing power of the cloudlet within a certain period of time, and (b) minimize the execution time given tasks offloaded to the cloudlet. An optimal tightest string policy is proposed to solve the optimization problem. In addition, a dynamic programming (DP) based suboptimal solution is introduced to simplify the practical implementation. How to design an online CPU frequency management strategy is also briefly discussed.

Transfer Reinforcement Learning for Adaptive Task Offloading Over Distributed Edge Clouds

In the Big Data era, resource-constrained mobile devices generate an overwhelmingly large amount of data with complex tasks that demand distributed execution. Offloading computation-intensive tasks to nearby edge clouds is promising to solve this problem. However, mobile end devices cannot handle heterogeneous or delay-sensitive tasks. These end devices are also energy constrained with weak adaptability to environment changes. To address and tackle these problems, we present a two-module <i>transfer reinforcement learning</i> (TRL) framework for adaptive task offloading. A domain adaptation module is used to align heterogeneous characteristics of mobile devices. The TRL makes offloading decisions with a <i>deep reinforcement learning</i> (DRL) module. We evaluate the performance of TRL through real-world experiments on edge clouds. Our experiment results show that TRL reduces the task processing time by a factor of 20&#x0025; from using three well known DRL methods. Our method achieved (15.4<inline-formula><tex-math notation="LaTeX">$\sim$</tex-math></inline-formula>40)&#x0025; reduction in task drop rate over these methods. With domain adaptation, the TRL results in (50<inline-formula><tex-math notation="LaTeX">$\sim$</tex-math></inline-formula>80)&#x0025; reduction in model convergence time. These advantages in using the TRL framework make it appealing in real-life edge computing applications.

MR-DRO: A Fast and Efficient Task Offloading Algorithm in Heterogeneous Edge/Cloud Computing Environments

With the rapid development of Internet of Things (IoT) and next-generation communication technologies, resource-constrained mobile devices (MDs) fail to meet the demand of resource-hungry and compute-intensive applications. To cope with this challenge, with the assistance of mobile-edge computing (MEC), offloading complex tasks from MDs to edge cloud servers (CSs) or central CSs can reduce the computational burden of devices and improve the efficiency of task processing. However, it is difficult to obtain optimal offloading decisions by conventional heuristic optimization methods, because the decision-making problem is usually NP-hard. In addition, there are shortcomings in using intelligent decision-making methods, e.g., lack of training samples and poor ability of migration under different MEC environments. To this end, we propose a novel offloading algorithm named meta reinforcement-deep reinforcement learning-based offloading, consisting of a meta-reinforcement learning (meta-RL) model, which improves the migration ability of the whole model, and a deep reinforcement learning (DRL) model, which combines multiple parallel deep neural networks (DNNs) to learn from historical task offloading scenarios. Simulation results demonstrate that our approach can effectively and efficiently generate near-optimal offloading decisions in IoT environments with edge and cloud collaboration, which further improves the computational performance and has strong portability when making offloading decisions.

An efficient mobile model for insect image classification in the field pest management

Accurately recognizing insect pest in their larva phase is significant to take the early treatment on the infected crops, thus helping timely reduce the yield loss in agricultural products. The convolutional neural networks (CNNs)-based classification methods have become the most competitive methods to address many technical challenges related to image recognition in the field. Focusing on accurate and small models carried on mobile devices, this study proposed a novel pest classification method PCNet (Pest Classification Network) based on lightweight CNNs embedded attention mechanism. PCNet was designed with EfficientNet V2 as the backbone, and the coordinate attention mechanism (CA) was incorporated in this architecture to learn the inter-channel pest information and pest positional information of input images. Moreover, combining the feature maps output by mobile inverted bottleneck (MBConv) with the feature maps output by average pooling to develop the feature fusion module, which implements the feature fusion between shallow layers and deep layers to address the loss of insect pest features in the down-sampling procedures. In addition, a stochastic, pipeline-based data augmentation approach was adopted to randomly enhance data diversity and thus avoid model overfitting. The experimental results show that the PCNet model achieved recognition accuracy of 98.4 % on the self-built dataset consisting of 30 classes of larvae, which outperforms three classic CNN models (AlexNet, VGG16, and ResNet101), and four lightweight CNN models (ShuffleNet V2, MobileNet V3, EfficientNet V1 and V2). To further verify the robustness on different datasets, the proposed model was also tested on two other public datasets: IP102 and miniImageNet. The recognition accuracy of PCNet is 73.7 % on the IP102 dataset, outperforming other models and 94.0 % on miniImageNet dataset, which is only lower than that of ResNet101 and MobileNet V3. The number of PCNet parameters is 20.7 M, which is less than those of traditional classic CNN models. The satisfactory accuracy and small size of this model makes it suitable for real-time pest recognition in the field with resource constrained mobile devices. Our code will be available at https://github.com/pby521/PCNet/tree/master.

Efficient On-Device Session-Based Recommendation

On-device session-based recommendation systems have been achieving increasing attention on account of the low energy/resource consumption and privacy protection while providing promising recommendation performance. To fit the powerful neural session-based recommendation models in resource-constrained mobile devices, tensor-train decomposition and its variants have been widely applied to reduce memory footprint by decomposing the embedding table into smaller tensors, showing great potential in compressing recommendation models. However, these model compression techniques significantly increase the local inference time due to the complex process of generating index lists and a series of tensor multiplications to form item embeddings. The resultant on-device recommender fails to provide real-time responses and recommendations. To improve the online recommendation efficiency, we propose to learn compositional encoding-based compact item representations. Specifically, each item is represented by a compositional code that consists of several codewords, and we learn embedding vectors to represent each codeword instead of each item. Then the composition of the codeword embedding vectors from different embedding matrices (i.e., codebooks) forms the item embedding. Since the size of codebooks can be extremely small, the recommender model is thus able to fit in resource-constrained devices and save the codebooks for fast local inference. Besides, to prevent the loss of model capacity caused by compression, we propose a bidirectional self-supervised knowledge distillation framework. Extensive experimental results on two benchmark datasets demonstrate that compared with existing methods, the proposed on-device recommender not only achieves an 8x inference speedup with a large compression ratio but also shows superior recommendation performance. The code is released at https://github.com/xiaxin1998/EODRec.

Hierarchical Model Compression via Shape-Edge Representation of Feature Maps—an Enlightenment From the Primate Visual System

The cumbersome computation of deep neural networks (DNNs) limits the practical deployment of DNNs on resource-constrained mobile multimedia devices. To deploy DNNs on devices with limited power and computing resources, model compression techniques are leveraged to accelerate the networks, where network pruning can improve the inference efficiency for deployed DNNs by removing redundant weights and structures. As one of the important components of DNNs, the feature maps (FMs) can be leveraged to evaluate the importance of network structures for DNN pruning. However, previous methods neglect to fully explore the characteristics of FMs to prune the DNNs. In this paper, we investigate the high capacity and resource efficient analogy-ventral dual-pathway primates visual system to propose a hierarchical pruning framework (dubbed as HPSE). In an efficient primate visual system (PVS), the analog pathway analyzes low-frequency information to facilitate the high-frequency information inference in ventral stream. In HPSE, we extract the low-frequency shape information and high-frequency edge information from FMs to present a novel pruning pipeline that resembles the analysis mechanism of PVS. In particular, we first imitate the analogy pathway to group different FMs in each layer by calculating the shape-feature overlap. Secondly, we leverage the edge information modulated by the grouping results of the first step to prune the network. The effectiveness of HPSE is verified by pruning various DNNs on different benchmarks, and the results have achieved satisfactory performances. For example, for ResNet-56 on CIFAR-10, HPSE reduces 52.9% of FLOPs with a slight accuracy improvement; for ResNet-50 on ImageNet, we achieve 54.3%-FLOPs drop with only 0.49% Top-1 accuracy loss.

Efficient Dropout-Resilient Aggregation for Privacy-Preserving Machine Learning

With the increasing adoption of data-hungry machine learning algorithms, personal data privacy has emerged as one of the key concerns that could hinder the success of digital transformation. As such, Privacy-Preserving Machine Learning (PPML) has received much attention from both academia and industry. However, organizations are faced with the dilemma that, on the one hand, they are encouraged to share data to enhance ML performance, but on the other hand, they could potentially be breaching the relevant data privacy regulations. Practical PPML typically allows multiple participants to individually train their ML models, which are then aggregated to construct a global model in a privacy-preserving manner, e.g., based on multi-party computation or homomorphic encryption. Nevertheless, in most important applications of large-scale PPML, e.g., by aggregating clients' gradients to update a global model for federated learning, such as consumer behavior modeling of mobile application services, some participants are inevitably resource-constrained mobile devices, which may drop out of the PPML system due to their mobility nature. Therefore, the resilience of privacy-preserving aggregation has become an important problem to be tackled. In this paper, we propose a scalable privacy-preserving aggregation scheme that can tolerate dropout by participants at any time, and is secure against both semi-honest and active malicious adversaries by setting proper system parameters. By replacing communication-intensive building blocks with a seed homomorphic pseudo-random generator, and relying on the additive homomorphic property of Shamir secret sharing scheme, our scheme outperforms state-of-the-art schemes by up to 6.37$\times$ in runtime and provides a stronger dropout-resilience. The simplicity of our scheme makes it attractive both for implementation and for further improvements.

OMCBIR: Offline mobile content-based image retrieval with lightweight CNN optimization

Convolutional Neural Networks (CNNs) have achieved great success in computer vision applications. However, due to the high requirements for computation power and memory usage, most state-of-the-art CNNs are difficult to deploy on resource-constrained mobile devices. Although many typical lightweight neural networks have been proposed in the industry, such as MobileNetV2, which reduce the amount of parameters and calculations, they still have a lot of redundancy. Furthermore, few papers consider the use of deep learning models to implement image retrieval on terminals, so we propose a new offline retrieval framework based on lightweight neural network models, called Offline Mobile Content-Based Image Retrieval (OMCBIR). In this framework, we focus on the feature extraction of the model, by introducing pointwise group convolution and channel shuffle into the bottleneck block, reconstructing the network structure, and introducing the convolutional attention module, we propose an extremely lightweight small network-Attention-based Lightweight Network (ALNet). Compared to MobileNetV2, ALNet obtains a higher mAP on each dataset in OMCBIR when the model parameters are reduced by more than 62% and the model size is reduced by more than 63%. Extensive experiments conducted on five public datasets provide a trade-off between retrieval performance and model size of different algorithms, which proves the efficiency of the proposed OMCBIR.