A Review on Federated Learning on Sensor-Based Human Activity Recognition

Federated Learning (FL) has recently attracted great interest in sensor-based human activity recognition (HAR) tasks. However, in real-world environment, sensor data on devices is non-independently and identically distributed (Non-IID), e.g., activity data recorded by most devices is sparse, and sensor data distribution for each client may be inconsistent. As a result, the traditional FL methods in the heterogeneous environment may incur a drifted global model that causes slow convergence and a heavy communication burden. Although some FL methods are gradually being applied to HAR, they are designed for overly ideal scenarios and do not address such Non-IID problem in the real-world setting. It is still a question whether they can be applied to cross-device FL. To tackle this challenge, we propose ProtoHAR, a prototype-guided FL framework for HAR, which aims to decouple the representation and classifier in the heterogeneous FL setting efficiently. It leverages the global prototype to correct the activity feature representation to make the prototype knowledge flow among clients without leaking privacy while solving a better classifier to avoid excessive drift of the local model in personalized training. Extensive experiments are conducted on four publicly available datasets: USC-HAD, UNIMIB-SHAR, PAMAP2, and HARBOX, which are collected in both controlled environments and real-world scenarios. The results show that compared with the state-of-the-art FL algorithms, ProtoHAR achieves the best performance and faster convergence speed in HAR datasets.

Sensor-based Human Activity Recognition (HAR) has been a hot topic in pervasive computing for many years, as an enabling technology for several context-aware applications. However, the deployment of HAR in real-world scenarios is limited by some major challenges. Among those issues, privacy is particularly relevant, since activity patterns may reveal sensitive information about the users (e.g., personal habits, medical conditions). HAR solutions based on Federated Learning (FL) have been recently proposed to mitigate this problem. In FL, each user shares with a cloud server only the parameters of a locally trained model, while personal data are kept private. The cloud server is in charge of building a global model by aggregating the received parameters. Even though FL avoids the release of labelled sensor data, researchers have found that the parameters of deep learning models may still reveal sensitive information through specifically designed attacks. In this paper, we propose a first contribution in this line of research by introducing a novel framework to quantitatively evaluate the effectiveness of the Membership Inference Attack (MIA) for FL-based HAR. Our preliminary results on a public HAR dataset show how the global activity model may actually reveal sensitive information about the participating users and provide hints for future work on countering such attacks.

One of the major open problems in sensor-based Human Activity Recognition (HAR) is the scarcity of labeled data. Among the many solutions to address this challenge, semi-supervised learning approaches represent a promising direction. However, their centralized architecture incurs in the scalability and privacy problems that arise when the process involves a large number of users. Federated learning (FL) is a promising paradigm to address these problems. However, the FL methods that have been proposed for HAR assume that the participating users can always obtain labels to train their local models (i.e., they assume a fully supervised setting). In this work, we propose FedAR: a novel hybrid method for HAR that combines semi-supervised and federated learning to take advantage of the strengths of both approaches. FedAR combines active learning and label propagation to semi-automatically annotate the local streams of unlabeled sensor data, and it relies on FL to build a global activity model in a scalable and privacy-aware fashion. FedAR also includes a transfer learning strategy to fine-tune the global model on each user. We evaluated our method on two public datasets, showing that FedAR reaches recognition rates and personalization capabilities similar to state-of-the-art FL supervised approaches. As a major advantage, FedAR only requires a very limited number of annotated data to populate a pre-trained model and a small number of active learning questions that quickly decrease while using the system, leading to an effective and scalable solution for the data scarcity problem of HAR.

Federated learning (FL) is a decentralized approach that aims to establish a global model by aggregating updates from diverse clients without sharing their local data. However, the approach becomes complicated when Byzantine clients join with arbitrary manipulation, referred to as malicious clients. Classical techniques, such as Federated Averaging (FedAvg), are insufficient to incentivize reliable clients and discourage malicious clients. Other existing Byzantine FL schemes to address malicious clients are either incentive-reliable clients or need-to-provide server-labeled data as the public validation dataset, which increase time complexity. This study introduces a federated learning framework with an evaluator-based incentive mechanism (FedEach) that offers robustness with no dependency on server-labeled data. In this framework, we introduce evaluators and participants. Unlike the existing approaches, the server selects the evaluators and participants among the clients using model-based performance evaluation criteria such as test score and reputation. Afterward, the evaluators assess and evaluate whether a participant is reliable or malicious. Subsequently, the server exclusively aggregates models from these identified reliable participants and the evaluators for global model updates. After this aggregation, the server calculates each client’s contribution, prioritizing each client’s contribution to ensure the fair recognition of high-quality updates and penalizing malicious clients based on their contributions. Empirical evidence obtained from the performance in human activity recognition (HAR) datasets highlights FedEach’s effectiveness, especially in environments with a high presence of malicious clients. In addition, FedEach maintains computational efficiency so that it is reliable for efficient FL applications such as sensor-based HAR with wearable devices and mobile sensing.

Evaluation and comparison of federated learning algorithms for Human Activity Recognition on smartphones

Human Activity Recognition (HAR) from sensor measurements is still challenging due to noisy or lack of la-belled examples and issues concerning data privacy. Training a traditional centralized machine learning (ML) model for HAR is constrained by infrastructure availability, network connectivity, latency issues etc. Issues regarding labels of user measurements can be tackled by using semi-supervised learning while issues regarding privacy concerns can be addressed by increasingly popular Federated Learning (FL). In this work, we propose a novel algorithm GraFeHTy, a Graph Convolution Network (GCN) trained in a federated setting to alleviate these key obstructions for HAR. We construct a similarity graph from sensor measurements for each user and apply a GCN to perform semi-supervised classification of human activities by leveraging inter-relatedness and closeness of activities. The weights of the GCN are trained using federated learning where each user performs gradient descent using their local data and share only the updated weights with a central server for aggregation. The GCN helps in accurate detection of unlabelled or noisy labels in the activity sequence by borrowing information from similar labelled nodes. The federated setting for training these models ensures that user privacy is respected by transferring only the learned representations out of the device to a central server. By avoiding transfer of raw data, the algorithm also ensures that training a HAR model is not constrained by infrastructure availability in central server or low network bandwidth from edge devices. Our proposed algorithm performs better than the baseline feed-forward federated learning model in terms of both accuracy and computational complexity.

FederatedTrust: A solution for trustworthy federated learning

Currently, federated learning (FL) can enable users to collaboratively train a global model while protecting the privacy of user data, which has been applied to human activity recognition (HAR) tasks. However, in real HAR scenarios, deploying an FL system needs to consider multiple aspects, including system accuracy, fairness, robustness, and scalability. Most existing FL frameworks aim to solve specific problems while ignoring other properties. In this paper, we propose FedCHAR, a personalized FL framework with a hierarchical clustering method for robust and fair HAR, which not only improves the accuracy and the fairness of model performance by exploiting the intrinsically similar relationship between users but also enhances the robustness of the system by identifying malicious nodes through clustering in attack scenarios. In addition, to enhance the scalability of FedCHAR, we also propose FedCHAR-DC, a scalable and adaptive FL framework which is featured by dynamic clustering and adapting to the addition of new users or the evolution of datasets for realistic FL-based HAR scenarios. We conduct extensive experiments to evaluate the performance of FedCHAR on seven datasets of different sizes. The results demonstrate that FedCHAR could obtain better performance on different datasets than the other five state-of-the-art methods in terms of accuracy, robustness, and fairness. We further validate that FedCHAR-DC exhibits satisfactory scalability on three large-scale datasets regardless of the number of participants.

Mass data generation occurring in the Internet-of-Things (IoT) requires processing to extract meaningful information. Deep learning is commonly used to perform such processing. However, due to the sensitive nature of these data, it is important to consider data privacy. As such, federated learning (FL) has been proposed to address this issue. FL pushes training to the client devices and tasks a central server with aggregating collected model weights to update a global model. However, the transmission of these model weights can be costly, gradually. The trade-off between communicating model weights for aggregation and the loss provided by the global model remains an open problem. In this work, we cast this trade-off problem of client selection in FL as an optimization problem. We then design a Distributed Client Selection (DCS) algorithm that allows client devices to decide to participate in aggregation in hopes of minimizing overall communication cost — while maintaining low loss. We evaluate the performance of our proposed client selection algorithm against standard FL and a state-of-the-art client selection algorithm, called Power-of-Choice (PoC), using CIFAR-10, FMNIST, and MNIST datasets. Our experimental results confirm that our DCS algorithm is able to closely match the loss provided by the standard FL and PoC, while on average reducing the overall communication cost by nearly 32.67% and 44.71% in comparison to standard FL and PoC, respectively.

Training of machine learning models in a Datacen-ter, with data originated from edge nodes, incurs high communication overheads and violates a user's privacy. These challenges may be tackled by employing Federated Learning (FL) machine learning technique to train a model across multiple decentralized edge devices (workers) using local data. In this paper, we explore an approach that identifies the most representative updates made by workers and those are only uploaded to the central server for reducing network communication costs. Based on this idea, we propose a FL model that can mitigate communication overheads via clustering analysis of the worker local updates. The Cluster Analysis-based Federated Learning (CA-FL) model is studied and evaluated in human activity recognition (HAR) datasets. Our evaluation results show the robustness of CA - FL in comparison with traditional FL in terms of accuracy and communication costs on both IID and non-IID cases.

Sensor-based Human Activity Recognition (HAR) has been a hot topic in pervasive computing for several years mainly due to its applications in healthcare and well-being. Centralized supervised approaches reach very high recognition rates, but they incur privacy and scalability issues. Federated Learning (FL) has been recently proposed to mitigate these issues. Each subject only shares the weights of a personal model trained locally, instead of sharing data. A cloud server is in charge of aggregating the weights to generate a global model. However, since activity data is non-independently and identically distributed (non-IID), a single model may not be sufficiently accurate for a large number of diverse users. In this work, we propose FedCLAR, a novel federated clustering method for HAR. Based on the similarity of the local model updates, the cloud server in FedCLAR derives groups of users that exhibit similar ways of performing activities. For each group, FedCLAR uses a specialized global model to mitigate the non-IID problem. We evaluated FedCLAR on two well-known public datasets, showing that it outperforms state-of-the-art FL solutions.

Deep learning-based Human Activity Recognition (HAR) systems received a lot of interest for health monitoring and activity tracking on wearable devices. The availability of large and representative datasets is often a requirement for training accurate deep learning models. To keep private data on users' devices while utilizing them to train deep learning models on huge datasets, Federated Learning (FL) was introduced as an inherently private distributed training paradigm. However, standard FL (FedAvg) lacks the capability to train heterogeneous model architectures. In this paper, we propose Federated Learning via Augmented Knowledge Distillation (FedAKD) for distributed training of heterogeneous models. FedAKD is evaluated on two HAR datasets: A waist-mounted tabular HAR dataset and a wrist-mounted time-series HAR dataset. FedAKD is more flexible than standard federated learning (FedAvg) as it enables collaborative heterogeneous deep learning models with various learning capacities. In the considered FL experiments, the communication overhead under FedAKD is 200X less compared with FL methods that communicate models' gradients/weights. Relative to other model-agnostic FL methods, results show that FedAKD boosts performance gains of clients by up to 20 percent. Furthermore, FedAKD is shown to be relatively more robust under statistical heterogeneous scenarios.

Federated Learning (FL) enables distributed training of machine learning models while keeping personal data on user devices private. While we witness increasing applications of FL in the area of mobile sensing, such as human activity recognition (HAR), FL has not been studied in the context of a multi-device environment (MDE), wherein each user owns multiple data-producing devices. With the proliferation of mobile and wearable devices, MDEs are increasingly becoming popular in ubicomp settings, therefore necessitating the study of FL in them. FL in MDEs is characterized by being not independent and identically distributed (non-IID) across clients, complicated by the presence of both user and device heterogeneities. Further, ensuring efficient utilization of system resources on FL clients in a MDE remains an important challenge. In this paper, we propose FLAME, a user-centered FL training approach to counter statistical and system heterogeneity in MDEs, and bring consistency in inference performance across devices. FLAME features (i) user-centered FL training utilizing the time alignment across devices from the same user; (ii) accuracy- and efficiency-aware device selection; and (iii) model personalization to devices. We also present an FL evaluation testbed with realistic energy drain and network bandwidth profiles, and a novel class-based data partitioning scheme to extend existing HAR datasets to a federated setup. Our experiment results on three multi-device HAR datasets show that FLAME outperforms various baselines by 4.3-25.8% higher F1 score, 1.02-2.86x greater energy efficiency, and up to 2.06x speedup in convergence to target accuracy through fair distribution of the FL workload.

Federated Learning (FL) is an emerging privacy-aware machine learning technique that applies successfully to the collaborative learning of global models for Human Activity Recognition (HAR). As of now, the applications of FL for HAR assume that the data associated with diverse individuals follow the same distribution. However, this assumption is impractical in real-world scenarios where the same activity is frequently performed differently by different individuals. To tackle this issue, we propose FedMAT, a Federated Multi-task ATtention framework for HAR, which extracts and fuses shared as well as individual-specific multi-modal sensor data features. Specifically, we treat the HAR problem associated with each individual as a different task and train a federated multi-task model, composed of a shared feature representation network in a central server plus multiple individual-specific networks with attention modules stored in decentralized nodes. In this architecture, the attention module operates as a mask that allows to learn individual-specific features from the global model, whilst simultaneously allowing for features to be shared among different individuals. We conduct extensive experiments based on publicly available HAR datasets, which are collected in both controlled environments and real-world scenarios. Numeric results verify that our proposed FedMAT significantly outperforms baselines not only in generalizing to existing individuals but also in adapting to new individuals.

Federated learning (FL) refers to a system in which a central aggregator coordinates the efforts of several clients to solve the issues of machine learning. This setting allows the training data to be dispersed in order to protect the privacy of each device. This paper provides an overview of federated learning systems, with a focus on healthcare. FL is reviewed in terms of its frameworks, architectures and applications. It is shown here that FL solves the preceding issues with a shared global deep learning (DL) model via a central aggregator server. Inspired by the rapid growth of FL research, this paper examines recent developments and provides a comprehensive list of unresolved issues. Several privacy methods including secure multiparty computation, homomorphic encryption, differential privacy and stochastic gradient descent are described in the context of FL. Moreover, a review is provided for different classes of FL such as horizontal and vertical FL and federated transfer learning. FL has applications in wireless communication, service recommendation, intelligent medical diagnosis system and healthcare, which we review in this paper. We also present a comprehensive review of existing FL challenges for example privacy protection, communication cost, systems heterogeneity, unreliable model upload, followed by future research directions.