Federated Learning Research Articles

Application of quantum-inspired tensor networks to optimize federated learning systems

Federated learning (FL) has gained significant traction across diverse industries, which allows multiple clients or institutions to enhance model performance and outcomes while preserving data privacy collaboratively. In recent years, tensor networks (TNs) have become important in machine learning because they allow the compact representation of high-dimensional tensors by decomposing them into lower-dimensional components with polynomial complexity. The application of TNs in FL is a natural extension because of its flexible framework for representing and optimizing models. Inspired by quantum computing principles, we have integrated a quantum-inspired tensor network into the FL framework. This framework focuses on a one-dimensional matrix product state (MPS) tensor network (TN) in a federated setting (FedTN), with data distributed across homogeneous and heterogeneous partitions among clients. Our experiments demonstrate that tensor network-based federated learning can be made practical, as FedTN is robust to the unbalanced and non-IID data distributions typically encountered in such settings. Our research assessed the effectiveness and feasibility of comparing quantum-inspired TN and conventional methods, evaluating their performance, and exploring the benefits of incorporating quantum principles in FL settings. Furthermore, we have investigated its performance when training for many local epochs (large E) between the averaging steps.

A Novel Data Obfuscation Framework Integrating Probability Density and Information Entropy for Privacy Preservation

Data privacy protection is increasingly critical in fields like healthcare and finance, yet existing methods, such as Fully Homomorphic Encryption (FHE), differential privacy (DP), and federated learning (FL), face limitations like high computational complexity, noise interference, and communication overhead. This paper proposes a novel data obfuscation method based on probability density and information entropy, leveraging a probability density extraction module for global data distribution modeling and an information entropy fusion module for dynamically adjusting the obfuscation intensity. In medical image classification, the method achieved precision, recall, and accuracy of 0.93, 0.89, and 0.91, respectively, with a throughput of 57 FPS, significantly outperforming FHE (0.82, 23 FPS) and DP (0.84, 25 FPS). Similarly, in financial prediction tasks, it achieved precision, recall, and accuracy of 0.95, 0.91, and 0.93, with a throughput of 54 FPS, surpassing traditional approaches. These results highlight the method’s ability to balance privacy protection and task performance effectively, offering a robust solution for advancing privacy-preserving technologies.

Toward Enhancing Privacy Preservation of a Federated Learning CNN Intrusion Detection System in IoT: Method and Empirical Study

Enormous risks and hidden dangers of information security exist in the applications of Internet of Things (IoT) technologies. To secure IoT software systems, software engineers have to deploy advanced security software such as Intrusion Detection Systems (IDS) that are able to keep track of how the IoT devices behave within the network and detect any malicious activity that may be occurring. Considering that IoT devices generate large amounts of data, Artificial Intelligence (AI) is often regarded as the best method for implementing IDS, thanks to AI’s high capability in processing large amounts of IoT data. To tackle these security concerns, specifically the ones tied to the privacy of data used in IoT systems, the software implementation of a Federated Learning (FL) method is often used to improve both privacy preservation (PP) and scalability in IoT networks. In this article, we present an FL IDS that leverages a 1-Dimensional Convolutional Neural Network (CNN) for efficient and accurate intrusion detection in IoT networks. To address the critical issue of PP in FL, we incorporate three techniques: Differential Privacy, Diffie–Hellman Key Exchange, and Homomorphic Encryption. To evaluate the effectiveness of our solution, we conduct experiments on seven publicly available IoT datasets: TON-IoT, IoT-23, BoT-IoT, CIC IoT 2023, CIC IoMT 2024, RT-IoT 2022, and EdgeIIoT. Our CNN-based approach achieves outstanding performance with an average accuracy, precision, recall, and F1-score of 97.31%, 95.59%, 92.43%, and 92.69%, respectively, across these datasets. These results demonstrate the effectiveness of our approach in accurately identifying and detecting intrusions in IoT networks. Furthermore, our experiments reveal that implementing all three PP techniques only incurs a minimal increase in computation time, with a 10% overhead compared to our solution without any PP mechanisms. This finding highlights the feasibility and efficiency of our solution in maintaining privacy while achieving high performance. Finally, we show the effectiveness of our solution through a comparison study with other recent IDS trained and tested on the same datasets we use.

A Personalized Federated Learning Algorithm Based on Dynamic Weight Allocation

Federated learning is a privacy-preserving distributed machine learning paradigm. However, due to client data heterogeneity, the global model trained by a traditional federated averaging algorithm often exhibits poor generalization ability. To mitigate the impact of data heterogeneity, some existing research has proposed clustered federated learning, where clients with similar data distributions are grouped together to reduce interference from dissimilar clients. However, since the data distribution of clients is unknown, determining the optimal number of clusters is difficult, leading to reduced model convergence efficiency. To address this issue, this paper proposes a personalized federated learning algorithm based on dynamic weight allocation. First, each client is allowed to obtain a global model tailored to fit its local data distribution. During the client model aggregation process, the server first computes the similarity of model updates between clients and dynamically allocates aggregation weights to client models based on these similarities. Secondly, clients use the received exclusive global model to train their local models via the personalized federated learning algorithm. Extensive experimental results demonstrate that, compared to other personalized federated learning algorithms, the proposed method effectively improves model accuracy and convergence speed.

FL-W3S: Cross-domain federated learning for weakly supervised semantic segmentation of white blood cells.

Segmentation models for clinical data experience severe performance degradation when trained on a single client from one domain and distributed to other clients from different domain. Federated Learning (FL) provides a solution by enabling multi-party collaborative learning without compromising the confidentiality of clients' private data. In this paper, we propose a cross-domain FL method for Weakly Supervised Semantic Segmentation (FL-W3S) of white blood cells in microscopic images. We perform model training on multiple clients with different data distributions to obtain a global aggregated model using only image-level class labels for semantic segmentation of white blood cells. A multi-class token transformer model learns the relationship between patch tokens and class tokens during collaborative learning and generates class-specific localization maps for mask predictions. To rectify the localization maps, we use patch-level pairwise affinity obtained from patch-to-patch transformer attention. We evaluate performance of the proposed semantic segmentation method on two different datasets of white blood cells from different domains. Our experimental results show that for two datasets, there is 2.56% and 1.39% increase in performance of the proposed method over existing state-of-the-art methods. The combination of federated learning for collaborative model training while preserving data privacy, alongside white blood cell segmentation techniques for precise cell identification, enhances diagnostic accuracy and personalized treatment strategies in clinical applications, particularly in hematology and pathology. More specifically, it involves isolating white blood cell from blood smear for further analysis such as automated blood cell counting, morphological analysis, cell classification, disease diagnosis and monitoring.

Federated Learning for IoT: A Survey of Techniques, Challenges, and Applications

Federated Learning (FL) has emerged as a pivotal approach for decentralized Machine Learning (ML), addressing the unique demands of the Internet of Things (IoT) environments where data privacy, bandwidth constraints, and device heterogeneity are paramount. This survey provides a comprehensive overview of FL, focusing on its integration with the IoT. We delve into the motivations behind adopting FL for IoT, the underlying techniques that facilitate this integration, the unique challenges posed by IoT environments, and the diverse range of applications where FL is making an impact. Finally, this submission also outlines future research directions and open issues, aiming to provide a detailed roadmap for advancing FL in IoT settings.

Dynamic selectout and voting-based federated learning for enhanced medical image analysis

Abstract Federated Learning (FL) is a promising technique for training machine learning models on distributed, privacy-aware datasets. Nevertheless, FL faces difficulties with agent/client participation, model performance, and the heterogeneous nature of networked data sources when it comes to distributed healthcare systems. When these agents work together in the system, it is imperative to tackle the complexities of distributed deep learning. We suggest a novel approach that uses a voting mechanism and dynamic SelectOut inside the FL framework to address these problems. Local medical imaging datasets frequently show diversity in distribution and data imbalances. In certain situations, traditional FL techniques like FedProx and Federated Averaging (FedAvg), which depend on data size to weight contributions, might not be the optimal choice. We improve parameter aggregation and client selection unpredictability and increase the model's adaptability to imbalanced and heterogeneous datasets, our proposed FedVoteNet model introduces SelectOut techniques based on Voting methodology. Based on how much their local performance has improved from the last communication cycle, we arbitrarily remove clients. Additionally eliminated are clients whose model weights when combined with the global model adversely affect its performance. After each communication cycle, clients that improve both their local performance and their contribution to the global model are awarded higher voting values. This encourages more significant and effective contributions from clients by providing incentives for them to actively increase the diversity of their training data. We assess our approach on a dataset of medical images, including MRI scans, and find that the FL model performs noticeably better (F1 Score=0.968, Sensitivity=0.977, Specificity=0.945, and AUC=0.950). The voting system and the dynamic SelectOut algorithms improve the convergence of the FL model and successfully handle the difficulties presented by uneven and heterogeneous datasets. To sum up, our proposed approach uses voting and dynamic SelectOut techniques to improve FL performance on a variety of uneven, distributed, and varied datasets. This strategy has a lot of potential to improve FL across a range of applications, especially those that prioritize data privacy, diversity, and performance.

Federated Learning for Privacy-Preserving AI in Hybrid Clouds: A Technical Overview

Federated Learning (FL) represents a transformative approach to machine learning in hybrid cloud environments, addressing critical challenges in data privacy and security while enabling collaborative model training across distributed environments. This comprehensive technical overview explores the architecture, implementation, and benefits of FL in hybrid cloud deployments. The article explores how organizations can leverage FL to maintain data sovereignty while achieving comparable or superior model performance to traditional centralized approaches. By analyzing various aspects including communication optimization, model initialization, local training, and global aggregation, this article demonstrates FL's effectiveness in preserving privacy while enabling cross-organizational collaboration. The article encompasses privacy preservation mechanisms, security protocols, and scalability considerations, providing insights into both current capabilities and future directions for enterprise implementations.

Out-of-Distribution Detection via outlier exposure in federated learning.

Among various out-of-distribution (OOD) detection methods in neural networks, outlier exposure (OE) using auxiliary data has shown to achieve practical performance. However, existing OE methods are typically assumed to run in a centralized manner, and thus are not feasible for a standard federated learning (FL) setting where each client has low computing power and cannot collect a variety of auxiliary samples. To address this issue, we propose a practical yet realistic OE scenario in FL where only the central server has a large amount of outlier data and a relatively small amount of in-distribution (ID) data is given to each client. For this scenario, we introduce an effective OE-based OOD detection method, called internal separation & backstage collaboration, which makes the best use of many auxiliary outlier samples without sacrificing the ultimate goal of FL, that is, privacy preservation as well as collaborative training performance. The most challenging part is how to make the same effect in our scenario as in joint centralized training with outliers and ID samples. Our main strategy (internal separation) is to jointly train the feature vectors of an internal layer with outliers in the back layers of the global model, while ensuring privacy preservation. We also suggest an collaborative approach (backstage collaboration) where multiple back layers are trained together to detect OOD samples. Our extensive experiments demonstrate that our method shows remarkable detection performance, compared to baseline approaches in the proposed OE scenario.

Federated Learning: An Approach for Managing Data Privacy and Security in Collaborative Learning

Abstract: In the field of machine learning, federated learning (FL) has become a breakthrough paradigm, provided a decentralized method of training models while solving issues with data security, privacy, and scalability. This study offers a thorough analysis of FL, including an examination of its underlying theories, its varieties, and a comparison with more conventional machine learning techniques. We explore the drawbacks of conventional machine learning techniques, especially when sensitive and distributed data is involved. We also explain how FL addresses these drawbacks by leveraging collaborative learning across decentralized devices or servers. We also highlight the various fields in which FL finds application, including healthcare, industries, IoT, mobile devices, and education, demonstrating its potential to deliver tailored services and predictive analytics while maintaining data privacy. Furthermore, we address the main obstacles to FL adoption, such as costly communication, heterogeneous systems, statistical heterogeneity, and privacy concerns, and we suggest possible directions for future research to effectively overcome these obstacles. In order to facilitate future study and growth in this quickly developing discipline, this review attempts to shed light on the advances and challenges of FL.

Efficient algorithm for resource optimization in optical communication networks.

Beyond 5 G and 6 G, communication systems should be able to deliver high throughput, low latency, high dependability, and high energy efficiency services. The creation of hybrid systems that can meet and largely satisfy these needs is promised by the merging of systems based on optical communication and radio frequency (RF). Smart devices may work together to cooperatively train Machine Learning (ML) models in a distributed fashion using Federated Learning (FL), all without disclosing personal information to a central server. This paper proposes a new solution to optimize the network resources in optical-RF communication network. The main idea is to optimize user selection, transmission power and channel estimation based on multilayer perception. Then, the loss function is minimized through joint optimization of user selection and transmission power. Simulation results show that, the proposed algorithm has better performance as compared with existing algorithms.

The Data Heterogeneity Issue Regarding COVID-19 Lung Imaging in Federated Learning: An Experimental Study

Federated learning (FL) has emerged as a transformative framework for collaborative learning, offering robust model training across institutions while ensuring data privacy. In the context of making a COVID-19 diagnosis using lung imaging, FL enables institutions to collaboratively train a global model without sharing sensitive patient data. A central manager aggregates local model updates to compute global updates, ensuring secure and effective integration. The global model’s generalization capability is evaluated using centralized testing data before dissemination to participating nodes, where local assessments facilitate personalized adaptations tailored to diverse datasets. Addressing data heterogeneity, a critical challenge in medical imaging, is essential for improving both global performance and local personalization in FL systems. This study emphasizes the importance of recognizing real-world data variability before proposing solutions to tackle non-independent and non-identically distributed (non-IID) data. We investigate the impact of data heterogeneity on FL performance in COVID-19 lung imaging across seven distinct heterogeneity settings. By comprehensively evaluating models using generalization and personalization metrics, we highlight challenges and opportunities for optimizing FL frameworks. The findings provide valuable insights that can guide future research toward achieving a balance between global generalization and local adaptation, ultimately enhancing diagnostic accuracy and patient outcomes in COVID-19 lung imaging.

Federated Learning and Reputation-Based Node Selection Scheme for Internet of Vehicles

With the rapid development of in-vehicle communication technology, the Internet of Vehicles (IoV) is gradually becoming a core component of next-generation transportation networks. However, ensuring the activity and reliability of IoV nodes remains a critical challenge. The emergence of blockchain technology offers new solutions to the problem of node selection in IoV. Nevertheless, traditional blockchain networks may suffer from malicious nodes, which pose security threats and disrupt normal blockchain operations. To address the issues of low participation and security risks among IoV nodes, this paper proposes a federated learning (FL) scheme based on blockchain and reputation value changes. This scheme encourages active involvement in blockchain consensus and facilitates the selection of trustworthy and reliable IoV nodes. First, we avoid conflicts between computing power for training and consensus by constructing state-channel transitions to move training tasks off-chain. Task rewards are then distributed to participating miner nodes based on their contributions to the FL model. Second, a reputation mechanism is designed to measure the reliability of participating nodes in FL, and a Proof of Contribution Consensus (PoCC) algorithm is proposed to allocate node incentives and package blockchain transactions. Finally, experimental results demonstrate that the proposed incentive mechanism enhances node participation in training and successfully identifies trustworthy nodes.

Advancements in securing federated learning with IDS: a comprehensive review of neural networks and feature engineering techniques for malicious client detection

Federated Learning (FL) is a technique that can learn a global machine-learning model at a central server by aggregating locally trained models. This distributed machine-learning approach preserves the privacy of local models. However, FL systems are inherently vulnerable to significant security challenges such as cyber-attacks, handling non-independent and identically distributed (non-IID) data, and data privacy concerns. This systematic literature review addresses these issues by examining advanced neural network models, feature engineering methods, and privacy-preserving techniques within intrusion detection systems (IDS) for FL environments. These are key elements for improving the security of FL systems. To the best of our knowledge, this review is among the first to comprehensively explore the combined impacts of these technologies. We analyzed 88 studies published between 2021 and October 2024. This study offers valuable insights for future research directions, including scaling FL in a real-world environment.

Federated Learning Unveiled: From Practical Insights to Bold Predictions

Federated Learning (FL) is a collaborative machine learning technique that lets multiple entities train a shared model without exchanging their data, keeping privacy intact. Over the past decade, FL has evolved, scaling to millions of devices in various domains, all while maintaining strong differential privacy (DP) protections. Companies like Google, Apple, and Meta have successfully brought FL to life in their systems, proving its real-world value. However, challenges remain. Verifying server-side DP guarantees and managing training across a mix of device types are complex issues that need attention. Emerging trends like large multi-modal models and blurred lines between training and personalization are pushing traditional FL to its limits. To address these challenges, we propose a more flexible FL framework focused on privacy principles, using trusted execution environments and open-source collaboration to drive future innovation.

Decentralized Federated Learning with Prototype Exchange

As AI applications become increasingly integrated into daily life, protecting user privacy while enabling collaborative model training has become a crucial challenge, especially in decentralized edge computing environments. Traditional federated learning (FL) approaches, which rely on centralized model aggregation, struggle in such settings due to bandwidth limitations, data heterogeneity, and varying device capabilities among edge nodes. To address these issues, we propose PearFL, a decentralized FL framework that enhances collaboration and model generalization by introducing prototype exchange mechanisms. PearFL allows each client to share lightweight prototype information with its neighbors, minimizing communication overhead and improving model consistency across distributed devices. Experimental evaluations on benchmark datasets, including MNIST, CIFAR-10, and CIFAR-100, demonstrate that PearFL achieves superior communication efficiency, convergence speed, and accuracy compared to conventional FL methods. These results confirm PearFL’s efficacy as a scalable solution for decentralized learning in heterogeneous and resource-constrained environments.

Collaboration of IoT devices in smart home scenarios: algorithm research based on graph neural networks and federated learning

Traditional IoT device collaboration is usually static and cannot adjust the collaboration mode between devices according to various changes, which limits work efficiency. To this end, an IoT device collaboration optimization algorithm based on graph neural network and federated learning is studied. This method abstracts various IoT device nodes and their communication relationships into graph structured data for storage, and then uses federated learning to train the graph convolutional network with graph structured data. The obtained model can be used to optimize the collaboration mode of IoT devices. During the training process, the total average MSE (mean square error) between the output and the label of the graph convolutional network model based on federated learning is 0.968; the total standard deviation of MSE is 0.0353; the total time from training to model convergence is 435.82 s, of which data transmission time accounts for 27.1% and model training time accounts for 72.9%. In a 2-h practical experiment, the graph convolutional network model based on federated learning was used to optimize the collaboration mode of smart homes, achieving a target environment residence time of 87 min and a total power consumption reduction of 0.69 kW·h. The results show that this method can effectively optimize the collaboration efficiency of IoT devices, reduce training time and network overhead, but it fails to improve the prediction accuracy of the model and may also lead to a decrease in stability.

Hierarchical Aggregation for Federated Learning in Heterogeneous IoT Scenarios: Enhancing Privacy and Communication Efficiency

Federated Learning (FL) is a distributed machine-learning paradigm that enables models to be trained across multiple decentralized devices or servers holding local data without transferring the raw data to a central location. However, applying FL to heterogeneous IoT scenarios comes with several challenges due to the diverse nature of these devices in terms of hardware capabilities, communications, and data heterogeneity. Furthermore, the conventional parameter server-based FL paradigm aggregates the trained parameters of devices directly, which incurs high communication overhead. To this end, this paper designs a hierarchical federated-learning framework for heterogeneous IoT systems, focusing on enhancing communication efficiency and ensuring data security through lightweight encryption. By leveraging hierarchical aggregation, lightweight stream encryption, and adaptive device participation, the proposed framework provides an efficient and robust solution for federated learning in dynamic and resource-constrained IoT environments. The extensive experimental results show that the proposed FL paradigm significantly reduces round time by 20%.

A conceptual approach of optimization in federated learning

Federated learning (FL) is an emerging approach to distributed learning from decentralized data, designed with privacy concerns in mind. FL has been successfully applied in several fields, such as the internet of things (IoT), human activity recognition (HAR), and natural language processing (NLP), showing remarkable results. However, the development of FL in real-world applications still faces several challenges. Recent optimizations of FL have been made to address these issues and enhance the FL settings. In this paper, we categorize the optimization of FL into five main challenges: Communication Efficiency, Heterogeneity, Privacy and Security, Scalability, and Convergence Rate. We provide an overview of various optimization frameworks for FL proposed in previous research, illustrated with concrete examples and applications based on these five optimization goals. Additionally, we propose two optional integrated conceptual frameworks (CFs) for optimizing FL by combining several optimization methods to achieve the best implementation of FL that addresses the five challenges.

Federated learning meets Bayesian neural network: Robust and uncertainty-aware distributed variational inference.

Federated Learning (FL) is a popular framework for data privacy protection in distributed machine learning. However, current FL faces some several problems and challenges, including the limited amount of client data and data heterogeneity. These lead to models trained on clients prone to drifting and overfitting, such that we just obtain suboptimal performance of the aggregated model. To tackle the aforementioned problems, we introduce a novel approach explicitly integrating Bayesian neural networks (BNNs) into the FL framework. The proposed approach is able to enhance the robustness. We refer to this approach as FedUAB, standing for FL with uncertainty-aware BNNs. In the FedUAB algorithm, each FL client independently trains a BNN using the Bayes by backprop algorithm. The weights of approximating model are modeled as Gaussian distributions, which mitigates the overfitting issue and also ensures better data privacy. Besides, we apply novel methods to overcome other key challenges in the fusion of BNNs and FL, such as selecting an optimal prior distribution, aggregating weights characterized by Gaussian forms across multiple clients, and rigorously managing weights variances. In the simulation of a FL environment, FedUAB demonstrated superior performance with both its server-side global model and client-side personalized models, outperforming traditional FL and other Bayesian FL methods. Moreover, it possesses the capability to quantify and decompose uncertainties. We have open-sourced our project at https://github.com/lpf111222/FedUAB/.