Federated Learning Research Articles

Federated learning: Impact of different algorithms and models on prediction results based on fashion-MNIST data set

The realm of federated learning is rapidly advancing amid the era of big data. Therefore, how to select a suitable federated learning algorithm to achieve realistic tasks has become particularly critical. In this study, we explore the impact of different algorithms and models on the prediction results of Federated Learning (FL) using the Fashion-MNIST data set. Federated Learning enhances data privacy and reduces latency by training models directly on local devices since it is a decentralized machine learning approach. We analyze the performance of several FL algorithms including Federated Averaging (FedAvg), Federated Stochastic Gradient Descent (FedSGD), Federated Proximal (FedProx), and SCAFFOLD. Our experiments reveal significant differences in accuracy and stability among these algorithms, highlighting their strengths and weaknesses in handling non-IID (Non-Independent and Identically Distributed) data. FedProx demonstrate superior performance in terms of accuracy and robustness, making them suitable for complex federated learning environments. These discoveries offer crucial insights for choosing suitable FL algorithms and models in practical applications.

Self-adaptive asynchronous federated optimizer with adversarial sharpness-aware minimization

The past years have witnessed the success of a distributed learning system called Federated Learning (FL). Recently, asynchronous FL (AFL) has demonstrated its potential in concurrency compared to mainstream synchronous FL. However, the inherent systematic and statistical heterogeneity has presented several impediments to AFL: On the client side, the discrepancies in trips and local model drift impede global performance enhancement; On the server side, dynamic communication leads to significant fluctuations in gradient arrival time, while asynchronous arrival gradients with ambiguous value are not fully leveraged. In this paper, we propose an adaptive AFL framework, ARDAGH, which systematically addresses the aforementioned challenges: Firstly, to address the discrepancies in client trips, ARDAGH ensures their convergence by incorporating only 1-bit feedback information into the downlink. Secondly, to counter the drift of clients, ARDAGH generalizes the local models by employing our novel adversarial sharpness-aware minimization, which does not necessitate reliance on additional global variables. Thirdly, in the face of gradient latency issues, ARDAGH employs a communication-aware dropout strategy to adaptively compress gradients to ensure similar transmission times. Finally, to fully unleash the potential of each gradient, we establish a consistent optimal direction by conceptualizing the aggregation as an optimizer with successive momentum. In light of the comprehensive solution provided by ARDAGH, an algorithm named FedAMO is derived, and its superiority is confirmed by experimental results obtained under challenging prototype and simulation settings. Particularly in typical sentiment analysis tasks, FedAMO demonstrates an improvement of up to 5.351% with a 20.056-fold acceleration compared to conventional asynchronous methods.

A review of federated learning algorithms in image classification

Image classification is one of the most popular applications of machine learning. It has shown its potential in fields like healthcare, auto-driving and face recognition. Federated learning (FL) emerged in 2017, creating a major innovation to the field. The new structure brings new possibilities, but create new challenges such as data heterogeneity, privacy leakage and communication burdens in parameter updating. The paper solves the problem that there are relatively few papers providing a complete analysis relating to the application of FL in image classification. The paper contributes to describe the new challenges of FL in image classification and show the related reasons behind respectively, then analyze the current state-of-the-art algorithms designed for solving the challenges and improving image model performance by discussing the basic ideas and steps of algorithms and showing their pros and cons. The paper further more contributes to compare the performance of each model in accuracy and communication speed, and outline several possible directions for future advancement.

Federated learning enhanced AR/VR integration in sleep wearable devices

As a crucial aspect of human health, sleep remains a problem to many individuals. Many still struggle with going to sleep or poor sleeping quality. In terms of ways to interfere with this issue, traditional sleep measurement methods, such as polysomnography(PSG), are intrusive, expensive, and inconvenient for long-term monitoring. Wearable devices offer a practical alternative but often face challenges related to accuracy, comfort, and data privacy. Advances in technology present new opportunities to address these issues.This systematic review focuses on examining the drawbacks of current sleeping measurements and wearable devices, and further explores the integration of federated learning (FL) with AR, VR in sleep wearable devices in order to promote sleep facilitation. The study addresses key challenges in sleep research, including accurate and non-intrusive monitoring, data privacy, and personalized sleep coaching. Federated learning enables decentralized data analysis, enhancing model accuracy while preserving user privacy. AR and VR technologies offer immersive environments for sleep facilitation through tailored relaxation techniques. The combination of these technologies provides innovative solutions to improve sleep quality and intervention efficacy. Future research should focus on enhancing device comfort, accuracy, and the integration of FL algorithms for better user experiences.

Comparative analysis of federated learning algorithms under non-IID data

With the increasing global attention to data privacy and security issues, how to effectively utilize distributed data while protecting personal privacy has become an important research topic. Federated learning (FL) aims to address data silos and privacy issues by enabling multiple devices or servers to train shared models in collaboration without submitting raw data to a central server. Although a variety of federated learning algorithms have been proposed, there is still a gap in the research on their performance differences under the same model. The goal of this study is to compare and analyze the performance differences of different federated learning algorithms on the same model through experiments. Based on the Fashion-MNIST dataset, this paper compares four commonly used federated learning algorithms in detail: Federal Averaging (FedAvg), Federated Stochastic Gradient Descent (FedSGD), Stochastic Controlled Averaging for Federated Learning (SCAFFOLD), and Federated Proximal (FedProx). The experimental results show that FedProx performs best in all evaluation indicators, followed by SCAFFOLD and FedAvg, while FedSGD performs the worst. These insights into algorithm performance with non-IID data inform practical application suitability and guide future research.

2FAKA-C/S: A Robust Two-Factor Authentication and Key Agreement Protocol for C/S Data Transmission in Federated Learning

As a hot technology trend, the federated learning (FL) cleverly combines data utilization and privacy protection by processing data locally on the client and only sharing model parameters with the server, embodying an efficient and secure collaborative learning model between clients and aggregated Servers. During the process of uploading parameters in FL models, there is susceptibility to unauthorized access threats, which can result in training data leakage. To ensure data security during transmission, the Authentication and Key Agreement (AKA) protocols are proposed to authenticate legitimate users and safeguard training data. However, existing AKA protocols for client–server (C/S) architecture show security deficiencies, such as lack of user anonymity and susceptibility to password guessing attacks. In this paper, we propose a robust 2FAKA-C/S protocol based on ECC and Hash-chain technology. Our security analysis shows that the proposed protocol ensures the session keys are semantically secure and can effectively resist various attacks. The performance analysis indicates that the proposed protocol achieves a total running time of 62.644 ms and requires only 800 bits of communication overhead, showing superior computational efficiency and lower communication costs compared to existing protocols. In conclusion, the proposed protocol securely protects the training parameters in a federated learning environment and provides a reliable guarantee for data transmission.

Fine-grained personalized federated learning via transformer in the transformer framework

Federated learning (FL) stands out as one of the most decentralized and privacy-preserving machine learning techniques. FL enables collaborative model training across multiple clients while preserving the privacy of their data. An extension of this method, namely, personalized federated learning (PFL), seeks to address client data heterogeneity by tailoring individualized models. Existing PFL methods that use transformers do not always take into account how federated averaging algorithms (a common FL algorithm that takes a weighted average of model parameters) can aggregate the self-attention mechanism. This makes it harder for transformers to work in federated settings. In addition, these approaches typically underperform on fine-grained tasks. To overcome these limitations, this study develops a novel transformer-in-transformer facilitated FL framework that not only aggregates global model parameters but also pinpoints personalized parameters for each client. This framework departs from traditional methods by implementing an “adaptive-personalization” mechanism, which not only enhances inter-client cooperation but also improves the model's scalability and adaptability. On the server side, a hypernetwork is developed to generate personalized projection matrices for use in the self-attention layers. These matrices determine the query, key, and value parameters for each client. Experimental results demonstrate that the model equipped with this adaptive-personalization mechanism considerably outperforms traditional models in scenarios with heterogeneous data, thus validating its effectiveness and efficiency.

Increasing trust in AI through privacy preservation and model explainability: Federated Learning of Fuzzy Regression Trees

Federated Learning (FL) lets multiple data owners collaborate in training a global model without any violation of data privacy, which is a crucial requirement for enhancing users’ trust in Artificial Intelligence (AI) systems. Despite the significant momentum recently gained by the FL paradigm, most of the existing approaches in the field neglect another key pillar for the trustworthiness of AI systems, namely explainability. In this paper, we propose a novel approach for FL of fuzzy regression trees (FRTs), which are generally acknowledged as highly interpretable by-design models. The proposed FL procedure is designed for the scenario of horizontally partitioned data and is based on the transmission of aggregated statistics from the clients to a central server for the tree induction procedure. It is shown that the proposed approach faithfully approximates the ideal case in which the tree induction algorithm is applied on the union of all local datasets, while still ensuring privacy preservation. Furthermore, the FL approach brings benefits, in terms of generalization capability, compared to the local learning setting in which each participant learns its own FRT based only on the private, local, dataset. The adoption of linear models in the leaf nodes ensures a competitive level of performance, as assessed by an extensive experimental campaign on benchmark datasets. The analysis of the results covers both the aspects of accuracy and interpretability of FRT. Finally, we discuss the application of the proposed federated FRT to the task of Quality of Experience forecasting in an automotive case-study.

Enhancing Energy Consumption Prediction in Smart Homes: A Convergence-Aware Federated Transfer Learning Approach

Achieving accurate energy consumption prediction can be challenging, particularly in residential buildings, which experience highly variable consumption behavior due to changes in occupants and the construction of new buildings. This variability, combined with the potential for privacy breaches through conventional data collection methods, underscores the need for novel approaches to energy consumption forecasting. The proposed study suggests a new approach to predict energy consumption, utilizing Federated Learning (FL) to train a global model while ensuring local data privacy and transferring knowledge from information-rich to information-poor buildings. The proposed method learns the transferable knowledge from the source building without any privacy leakage and utilizes it for target buildings. Since the performance of the global model could be negatively affected by some participating nodes with poor performance due to noisy or limited data, we propose a client selection strategy on the server based on the normal distribution for choosing the best possible participants for the global model. Our method enables clients to participate selectively in the aggregation procedure to avoid model divergence due to poor performance. The proposed model is evaluated and conducts in-depth analyses of energy consumption patterns. We validate the performance by comparing its MAE, MSE, and R2 values to those of existing traditional and ensemble models. Our findings indicate that the proposed FL-based model with selective client participation outperforms its counterpart methods regarding predictive accuracy and robustness. The source code is available on Github

Developing a privacy-preserving deep learning model for glaucoma detection: a multicentre study with federated learning

BackgroundDeep learning (DL) is promising to detect glaucoma. However, patients’ privacy and data security are major concerns when pooling all data for model development. We developed a privacy-preserving DL model...

EAFL: Equilibrium Augmentation Mechanism to Enhance Federated Learning for Aspect Category Sentiment Analysis

Aspect Category Sentiment Analysis (ACSA) involves identifying sentiment categories for specific aspects of a sentence. Despite the progress made in pre-trained language models for extracting semantic and categorical representations, the sharing of sensitive data remains challenging due to data privacy protection. Federated Learning (FL) offers a solution by enabling decentralized model training without accessing raw datasets. However, isolated data encounters difficulties with text heterogeneity, which is crucial for aspect-based sentiment analysis (ABSA) subtask like ACSA. To address these problems, this paper presents a novel method called the Equilibrium Augmentation Mechanism to Enhance Federated Learning for Aspect Category Sentiment Analysis (EAFL-ACSA). The EAFL mechanism seamlessly integrates FL’s data privacy preservation capabilities with a variety of augmented data representations. Our unified model safeguards data privacy through FL and incorporates multiple features through data augmentation to enhance encrypted data. Furthermore, the parameter-efficient tuning mechanism incorporated in our framework enhances model performance and generates fine-tuned augmented sentences. Extensive experiments on five benchmark datasets demonstrates that EAFL consistently outperforms the best models, achieving a competitive level of performance while protecting data privacy. The equilibrium data augmentation mechanism not only advances aspect category detection but also enhances semantic data representations, effectively addressing critical concerns surrounding data privacy preservation within a unified framework.

Privacy-preserving edge federated learning for intelligent mobile-health systems

Machine Learning (ML) algorithms are generally designed for scenarios in which all data is stored in one data center, where the training is performed. However, in many applications, e.g., in the healthcare domain, the training data is distributed among several entities, e.g., different hospitals or patients’ mobile devices/sensors. At the same time, transferring the data to a central location for learning is certainly not an option, due to privacy concerns and legal issues, and in certain cases, because of the communication and computation overheads. Federated Learning (FL) is the state-of-the-art collaborative ML approach for training an ML model across multiple parties holding local data samples, without sharing them. However, enabling learning from distributed data over such edge Internet of Things (IoT) systems (e.g., mobile-health and wearable technologies, involving sensitive personal/medical data) in a privacy-preserving fashion presents a major challenge mainly due to their stringent resource constraints, i.e., limited computing capacity, communication bandwidth, memory storage, and battery lifetime. In this paper, we propose a privacy-preserving edge FL framework for resource-constrained mobile-health and wearable technologies over the IoT infrastructure. We evaluate our proposed framework extensively and provide the implementation of our technique on Amazon’s AWS cloud platform based on the seizure detection application in epilepsy monitoring using wearable technologies.

DDP-FedFV: A Dual-Decoupling Personalized Federated Learning Framework for Finger Vein Recognition.

Finger vein recognition methods, as emerging biometric technologies, have attracted increasing attention in identity verification due to their high accuracy and live detection capabilities. However, as privacy protection awareness increases, traditional centralized finger vein recognition algorithms face privacy and security issues. Federated learning, a distributed training method that protects data privacy without sharing data across endpoints, is gradually being promoted and applied. Nevertheless, its performance is severely limited by heterogeneity among datasets. To address these issues, this paper proposes a dual-decoupling personalized federated learning framework for finger vein recognition (DDP-FedFV). The DDP-FedFV method combines generalization and personalization. In the first stage, the DDP-FedFV method implements a dual-decoupling mechanism involving model and feature decoupling to optimize feature representations and enhance the generalizability of the global model. In the second stage, the DDP-FedFV method implements a personalized weight aggregation method, federated personalization weight ratio reduction (FedPWRR), to optimize the parameter aggregation process based on data distribution information, thereby enhancing the personalization of the client models. To evaluate the performance of the DDP-FedFV method, theoretical analyses and experiments were conducted based on six public finger vein datasets. The experimental results indicate that the proposed algorithm outperforms centralized training models without increasing communication costs or privacy leakage risks.

A comprehensive survey on client selection strategies in federated learning

Federated learning (FL) has emerged as a promising paradigm for collaborative model training while preserving data privacy. Client selection plays a crucial role in determining the overall performance and efficiency of the FL training process. However, the heterogeneity of clients in terms of computational resources, data quality, and network conditions poses significant challenges in selecting the optimal set of clients for each training round. This survey provides a comprehensive review of client selection methodologies in FL, addressing the challenges and opportunities in this field. We present a systematic categorization of existing client selection techniques based on their underlying principles and objectives, and discuss the key challenges, including resource constraints, data heterogeneity, and security concerns. Additionally, we provide a comparative analysis of the surveyed techniques, highlighting their strengths, limitations, and suitability for various FL scenarios. Furthermore, we identify open research problems and propose future research directions, emphasizing the need for more efficient, adaptive, and secure selection strategies. This survey serves as a comprehensive framework for understanding client selection in FL, bridging gaps in the existing literature and providing guidance for future research, including ethical considerations and domain-specific applications. Our work aims to support practitioners in making informed decisions and to stimulate further research in this critical area of FL.

TSFed: A three-stage optimization mechanism for secure and efficient federated learning in industrial IoT networks

This paper presents a three-stage optimization mechanism designed to enhance Federated Learning (FL) in Industrial Internet of Things (IIoT) networks. Traditional FL optimizations, which typically focus on a single aspect, fall short in IIoT environments. Our approach integrates a multi-criteria enhancement: first, an Ensembled Client Selection Mechanism (ECSM) selects participants based on accuracy, reputation, and randomness. Second, Adaptive Distributed Client Training (ADCT) dynamically adjusts based on participant performance. Lastly, a Secure and Efficient Communication Channel (SECC), backed by blockchain, meets IIoT’s stringent security demands. The evaluation shows TSFed outperforms baseline methods, enhancing FL performance by increasing accuracy and F1-score. Importantly, TSFed improves the efficiency of achieving 80% accuracy on the MNIST dataset by 29.09% over baseline methods, showcasing significant gains in both security and efficiency. This mechanism also exhibits robustness against malicious attacks, setting a new benchmark for FL in IIoT environments.

DeepSneak: User GPS Trajectory Reconstruction from Federated Route Recommendation Models

Decentralized machine learning, such as Federated Learning (FL), is widely adopted in many application domains. Especially in domains like recommendation systems, sharing gradients instead of private data has recently caught the research community’s attention. Personalized travel route recommendation utilizes users’ location data to recommend optimal travel routes. Location data is extremely privacy sensitive, presenting increased risks of exposing behavioural patterns and demographic attributes. FL for route recommendation can mitigate the sharing of location data. However, this paper shows that an adversary can recover the user trajectories used to train the federated recommendation models with high proximity accuracy. To this effect, we propose a novel attack called DeepSneak, which uses shared gradients obtained from global model training in FL to reconstruct private user trajectories. We formulate the attack as a regression problem and train a generative model by minimizing the distance between gradients. We validate the success of DeepSneak on two real-world trajectory datasets. The results show that we can recover the location trajectories of users with reasonable spatial and semantic accuracy.

When Federated Learning Meets Privacy-Preserving Computation

Nowadays, with the development of artificial intelligence (AI), privacy issues attract wide attention from society and individuals. It is desirable to make the data available but invisible, i.e., to realize data analysis and calculation without disclosing the data to unauthorized entities. Federated learning (FL) has emerged as a promising privacy-preserving computation method for AI. However, new privacy issues have arisen in FL-based application because various inference attacks can still infer relevant information about the raw data from local models or gradients. This will directly lead to the privacy disclosure. Therefore, it is critical to resist these attacks to achieve complete privacy-preserving computation. In light of the overwhelming variety and a multitude of privacy-preserving computation protocols, we survey these protocols from a series of perspectives to supply better comprehension for researchers and scholars. Concretely, the classification of attacks is discussed including four kinds of inference attacks as well as malicious server and poisoning attack. Besides, this paper systematically captures the state of the art of privacy-preserving computation protocols by analyzing the design rationale, reproducing the experiment of classic schemes, and evaluating all discussed protocols in terms of efficiency and security properties. Finally, this survey identifies a number of interesting future directions.

Improving Global Generalization and Local Personalization for Federated Learning.

Federated learning aims to facilitate collaborative training among multiple clients with data heterogeneity in a privacy-preserving manner, which either generates the generalized model or develops personalized models. However, existing methods typically struggle to balance both directions, as optimizing one often leads to failure in another. To address the problem, this article presents a method named personalized federated learning via cross silo prototypical calibration (pFedCSPC) to enhance the consistency of knowledge of clients by calibrating features from heterogeneous spaces, which contributes to enhancing the collaboration effectiveness between clients. Specifically, pFedCSPC employs an adaptive aggregation method to offer personalized initial models to each client, enabling rapid adaptation to personalized tasks. Subsequently, pFedCSPC learns class representation patterns on clients by clustering, averages the representations within each cluster to form local prototypes, and aggregates them on the server to generate global prototypes. Meanwhile, pFedCSPC leverages global prototypes as knowledge to guide the learning of local representation, which is beneficial for mitigating the data imbalanced problem and preventing overfitting. Moreover, pFedCSPC has designed a cross-silo prototypical calibration (CSPC) module, which utilizes contrastive learning techniques to map heterogeneous features from different sources into a unified space. This can enhance the generalization ability of the global model. Experiments were conducted on four datasets in terms of performance comparison, ablation study, in-depth analysis, and case study, and the results verified that pFedCSPC achieves improvements in both global generalization and local personalization performance via calibrating cross-source features and strengthening collaboration effectiveness, respectively.

Privacy-preserving AI for early diagnosis of thoracic diseases using IoTs: A federated learning approach with multi-headed self-attention for facilitating cross-institutional study

Our study recognized the crucial role of early diagnosis of pulmonary radiological abnormalities such as pneumothorax, effusion, pneumonia, cardiomegaly, and COVID-19. We proposed FedXNet, which is a collaborative deep learning model based on federated learning (FL) exploiting edge computing resources efficiently to accurately deal with them and ensure privacy. Our developed model is notable for its integration of Multi-Headed Self-Attention, a complex technique that allows the model to focus on several parts of the input data at once. This improves the model’s capacity to uncover complex patterns and correlations within the medical images. This multi-class CNN system uses a thorough four-pronged approach: (1) facilitating cross-institutional, federated training without sacrificing the integrity of individual data, (2) image preprocessing to achieve robust model accuracy, (3) efficient Feature extraction using pre-trained models and our dedicated FedXNet architecture, as well as (4) a variety of classifiers tailored to each disease, resulting in impressive diagnostic performance for a range of thoracic diseases, including COVID-19. This model paves the way for a future where timely diagnosis and better patient outcomes become a reality, empowered by the collaborative spirit of FL exploiting edge computing resources of IoT for implementing robust deep learning models.

A comparative Study on AI-Driven Anonymization Techniques for Protecting Personal Data

In the Artificial Intelligence era, protecting individual users' data has become crucial. The collected data is stored in multiple databases having personally identifiable information (PII). This may provide a significant privacy concern for the database. Several privacy-preserving approaches have been proposed, including Differential Privacy, Homomorphic Encryption, Generative Adversarial Network and Federated Learning. In this paper, the above four anonymization techniques are compared. In addition, this study will review the strengths and weaknesses of these techniques. We also discuss the trade-off between data utility and privacy. The results of this study aim to guide researchers and practitioners in selecting suitable AI-driven anonymization techniques.