Federated Learning Model Research Articles

Accuracy Rate Maximization in Edge Federated Learning With Delay and Energy Constraints

Deploying deep learning models on edge servers can effectively alleviate the pressure on cloud data centers in computing, communication, and energy, etc. In this study, we attempt to extend this deployment scheme from edge servers to mobile terminals to further save spectrum resources, utilize communication resources, and provide intelligent applications. Unfortunately, the learning model experiences a slow convergence speed, and the mobile terminal consumes much energy if the model is only trained on the local devices with weak computing power. Therefore, we deploy deep learning algorithms (both in edge servers and mobile terminals) by considering the delay and energy consumption constraints of terminals in task processing. Specifically, a lightweight federated learning model is first proposed through the cooperative training between edge servers and mobile terminals. The trained model is then migrated to mobile terminals. Finally, image identification is considered to verify the effectiveness and efficiency of our proposed model. The experimental results show that the accuracy of the model reaches 90% using two datasets and three network models. In addition, it further reduces the processing delay and saves energy.

Joint Device Selection and Bandwidth Allocation for Cost-Efficient Federated Learning in Industrial Internet of Things

Along with the deployment of Industrial Internet of Things (IIoT), massive amounts of industrial data have been generated at the network edge, driving the evolution of edge machine learning (ML). But during the ML model training, it may bring privacy leakage by traditional central methods. To address this issue, federated learning (FL) has been proposed as a distributed learning framework for training a global model without uploading raw data to protect data privacy. Since the communication and computing resources are usually limited in IIoT networks, how to reasonably select device and allocate bandwidth is crucial for the FL model training. Therefore, this paper proposes a joint edge device selection and bandwidth allocation scheme for FL to minimize the time-averaged cost under the given long-term energy budget and delay constraints in IIoT system. To tackle with this long-term optimization problem, we construct a virtual energy deficit queue and leverage the Lyapunov optimization theory to transform it into a list of round-wise drift-plus-cost minimization problems firstly. Then, we design an iterative algorithm to allocate reasonable bandwidth and select appropriate devices to achieve cost minimization while satisfying the energy consumption constraints. Besides, we develop an optimality analysis of the average cost and energy violation for our proposed scheme. Extensive experiments verify that our proposed scheme can achieve superior performance in cost efficiency over other schemes while guaranteeing FL training performance.

FBN: Federated Bert Network with client-server architecture for cross-lingual signature verification

Online signature verification has a great challenge due to the poor performance of deep learning techniques on cross-lingual datasets under privacy constraints. In this paper, we propose a novel Federated Bert Network (FBN) by embedding the Bidirectional Encoder Representations from Transformers (Bert) into a Federated Learning (FL) framework with client-server architecture. A new Length Alignment Algorithm is employed to unify the signature pairs’ sequence length, and the input representations are fed into the different clients to complete the independent learning of local-models. In addition, the server (coordinator) uses the improved Federated Average Algorithm with Reward-Punishment Mechanism (FedAvgRP) to aggregate these local-models and further generate a global-model. After multiple iterations, the optimal model can be obtained and cross-tested on four datasets (SVC 2004, MCYT-330, BioecurID, and Ours) with skilled forged (random forged) EERs of 7.65% (4.76%), 10.73% (8.46%), 10.09% (7.13%), and 8.28% (5.74%), respectively, far higher than that of the independent learning of state-of-the-art methods. Compared with the domain adaptation and improved FL models, our FBN model performs best in random and skilled forgery scenarios. Moreover, the FedAvgRP algorithm helps our model maintain high performance in the face of data attacks.

A Novel Disease Prediction Model Based on Blockchain and Federated Learning

With the advent of the era of big data, the model architecture of disease prediction has been transformed from cloud computing architecture to edge computing architecture to alleviate problems, such as single point of failure, huge burden of computing and long communication delays.However, due to the lack of trust among edge computing participants (herein referred to as medical institutions with data resources or data storage nodes), the patients’ personal privacy and security issues are still challenging, and there is an urgent need to propose new solutions with higher security. In this paper, we proposed a new disease prediction model with the edge computing framework based on blockchain and federated learning technology to realise full mining of data value, as well as protecting local data sharing security and privacy. In the process of federated learning with blockchain as the underlying platform working environment, we propose the concept of model accuracy as the selection indicator of data nodes in federated learning, which improves the accuracy of the federated learning model and optimizes the training delay.

Federated Learning for Medical Image Analysis with Deep Neural Networks.

Medical image analysis using deep neural networks (DNN) has demonstrated state-of-the-art performance in image classification and segmentation tasks, aiding disease diagnosis. The accuracy of the DNN is largely governed by the quality and quantity of the data used to train the model. However, for the medical images, the critical security and privacy concerns regarding sharing of local medical data across medical establishments precludes exploiting the full DNN potential for clinical diagnosis. The federated learning (FL) approach enables the use of local model's parameters to train a global model, while ensuring data privacy and security. In this paper, we review the federated learning applications in medical image analysis with DNNs, highlight the security concerns, cover some efforts to improve FL model performance, and describe the challenges and future research directions.

Aligning Federated Learning with Existing Trust Structures in Health Care Systems.

Patient-centered health care information systems (PHSs) on peer-to-peer (P2P) networks (e.g., decentralized personal health records) enable storing data locally at the edge to enhance data sovereignty and resilience to single points of failure. Nonetheless, these systems raise concerns on trust and adoption in medical workflow due to non-alignment to current health care processes and stakeholders' needs. The distributed nature of the data makes it more challenging to train and deploy machine learning models (using traditional methods) at the edge, for instance, for disease prediction. Federated learning (FL) has been proposed as a possible solution to these limitations. However, the P2P PHS architecture challenges current FL solutions because they use centralized engines (or random entities that could pose privacy concerns) for model update aggregation. Consequently, we propose a novel conceptual FL framework, CareNetFL, that is suitable for P2P PHS multi-tier and hybrid architecture and leverages existing trust structures in health care systems to ensure scalability, trust, and security. Entrusted parties (practitioners' nodes) are used in CareNetFL to aggregate local model updates in the network hierarchy for their patients instead of random entities that could actively become malicious. Involving practitioners in their patients' FL model training increases trust and eases access to medical data. The proposed concepts mitigate communication latency and improve FL performance through patient-practitioner clustering, reducing skewed and imbalanced data distributions and system heterogeneity challenges of FL at the edge. The framework also ensures end-to-end security and accountability through leveraging identity-based systems and privacy-preserving techniques that only guarantee security during training.

FedIPR: Ownership Verification for Federated Deep Neural Network Models.

Federated learning models are collaboratively developed upon valuable training data owned by multiple parties. During the development and deployment of federated models, they are exposed to risks including illegal copying, re-distribution, misuse and/or free-riding. To address these risks, the ownership verification of federated learning models is a prerequisite that protects federated learning model intellectual property rights (IPR) i.e., FedIPR. We propose a novel federated deep neural network (FedDNN) ownership verification scheme that allows private watermarks to be embedded and verified to claim legitimate IPR of FedDNN models. In the proposed scheme, each client independently verifies the existence of the model watermarks and claims respective ownership of the federated model without disclosing neither private training data nor private watermark information. The effectiveness of embedded watermarks is theoretically justified by the rigorous analysis of conditions under which watermarks can be privately embedded and detected by multiple clients. Moreover, extensive experimental results on computer vision and natural language processing tasks demonstrate that varying bit-length watermarks can be embedded and reliably detected without compromising original model performances. Our watermarking scheme is also resilient to various federated training settings and robust against removal attacks.

316 Neurosurgical Federated Learning: A Multi-Center Collaboration to Detect Intracranial Hemorrhage Without Directly Sharing Data

INTRODUCTION: The development of accurate and generalizable machine learning algorithms requires sufficient quantities of diverse data. This poses a challenge in healthcare due to the sensitive and siloed nature of biomedical information. Decentralized algorithms through a federated learning (FL) paradigm avoid the need for data aggregation by instead distributing algorithms to the data itself before centrally updating one global model. METHODS: Five academic neurosurgery departments across the US collaborated to establish a federated network using computed tomography (CT) scans for prototyping. We trained a convolutional neural network to detect the presence of ICH through FL and benchmarked this against a standard, centralized training approach. Models were validated on each site’s data and tested on a held-out dataset to determine the area under the ROC curve (AUROC). RESULTS: A federated network of practicing neurosurgeon-scientists was successfully initiated and used to train a model for predicting ICH across the US. The federated model achieved an average AUROC of 0.9487 at predicting all types of ICH compared to a benchmark non-FL model AUROC of 0.9753, although performance varied by hemorrhage subtype. Subdural bleeds had the lowest AUROC (0.9257) while intraventricular bleeds had the highest (0.9751) on hold-out data. The global FL model consistently achieved top-three performance (of five) when validated on data from each site, suggesting improved generalizability. A qualitative survey of participants revealed that a majority found the process of connecting to the federated network easy. CONCLUSIONS: This study demonstrates the feasibility of implementing a federated network for multi-institutional collaboration amongst clinicians and using FL to conduct machine learning research without the transfer of data between sites, thereby opening up a new paradigm for neurosurgical collaboration.

Federated learning for scam classification in small Indonesian language dataset: an initial study

Most digital phishing or scam trick users into fraudulent links and is more effective against users with low technology literacy, like in Indonesia. Machine learning is widely used for scam classification, but most require sending the messages to a centralized server. This induces privacy concern as messages might contain private data. Federated learning (FL) was proposed to allow user devices to train models locally without sending data to server. In this work, we examined the use of FL with gated recurrent unit (GRU) model for classifying scam messages in Indonesian language with small dataset. We provided two FL-based baseline models (FedAvg and daisy-chained algorithms) and a dataset for scam classification in Indonesian language. We examined the models based on these performance metrics; precision, recall, F1, selectivity, and balanced accuracy. Despite the performance, we pointed out characteristics of the FL algorithms and the hyperparameters for this use case as pointers towards fine-tuning these baseline models. Overall, the FL model with FedAvg algorithm performed better in all metrics except recall.

Decentralized P2P Federated Learning for Privacy-Preserving and Resilient Mobile Robotic Systems

Swarms of mobile robots are being widely applied for complex tasks in various practical scenarios toward modern smart industry. Federated learning (FL) has been developed as a promising privacy-preserving paradigm to tackle distributed machine learning tasks for mobile robotic systems in 5G and beyond networks. However, unstable wireless network conditions of the complex and harsh working environment may lead to poor communication quality and bring big challenges to traditional centralized global training in FL models. In this article, a Peer-to-Peer (P2P) based Privacy-Perceiving Asynchronous Federated Learning (PPAFL) framework is introduced to realize the decentralized model training for secure and resilient modern mobile robotic systems in 5G and beyond networks. Specifically, a reputation-aware coordination mechanism is designed and addressed to coordinate a group of smart devices dynamically into a virtual cluster, in which the asynchronous model aggregation is conducted in a decentralized P2P manner. A secret sharing based communication mechanism is developed to ensure an encrypted P2P FL process, while a Secure Stochastic Gradient Descent (SSGD) scheme is integrated with an Autoencoder and a Gaussian mechanism is developed to ensure an anonymized local model update, communicating within a few neighboring clients. The case study based experiment and evaluation in three different application scenarios demonstrate that the PPAFL can effectively improve the security and resilience issues compared with the traditional centralized approaches for smart mobile robotic applications in 5G and beyond networks.

A Clustered Federated Learning Method of User Behavior Analysis Based on Non-IID Data

Federated learning (FL) is a novel distributed machine learning paradigm. It can protect data privacy in distributed machine learning. Hence, FL provides new ideas for user behavior analysis. User behavior analysis can be modeled using multiple data sources. However, differences between different data sources can lead to different data distributions, i.e., non-identically and non-independently distributed (Non-IID). Non-IID data usually introduce bias in the training process of FL models, which will affect the model accuracy and convergence speed. In this paper, a new federated learning algorithm is proposed to mitigate the impact of Non-IID data on the model, named federated learning with a two-tier caching mechanism (FedTCM). First, FedTCM clustered similar clients based on their data distribution. Clustering reduces the extent of Non-IID between clients in a cluster. Second, FedTCM uses asynchronous communication methods to alleviate the problem of inconsistent computation speed across different clients. Finally, FedTCM sets up a two-tier caching mechanism on the server for mitigating the Non-IID data between different clusters. In multiple simulated datasets, compared to the method without the federated framework, the FedTCM is maximum 15.8% higher than it and average 12.6% higher than it. Compared to the typical federated method FedAvg, the accuracy of FedTCM is maximum 2.3% higher than it and average 1.6% higher than it. Additionally, FedTCM achieves more excellent communication performance than FedAvg.

Multi-party Diabetes Mellitus risk prediction based on secure federated learning

At present, there is a lack of data exchange between medical institutions at the municipal level in the prevention and control of chronic disease. In this paper, we study Diabetes Mellitus risk prediction based on data securely shared between hospital and medical examination center, where federated learning algorithms are used and security challenges are analyzed. Targeted at a general disease prediction problem, we design customized secure protocols for regression family and tree family algorithms to solve the problem of potential private data leakage. Prediction of diabetes risks is taken as a typical example, and XGBoost, LightGBM, Neural Network, Logistic Regression algorithms are used for joint data modeling between different organizations, with the support from the “Seceum” Federated Learning Platform we first introduced in this paper. Extensive results on two datasets, including computational costs, recall and precision metrics, significant tests, show that using federated learning models we can make better use of the patient data between different organizations and deliver a reliable and improved prediction of Diabetes Mellitus risks.

Federated learning for performance prediction in multi-operator environments

Telecom vendors and operators deliver services with strict requirements on performance, over complex and sometimes partly shared network infrastructures. A key enabler for network and service management in such environments is knowledge sharing, and the use of data-driven models for performance prediction, forecasting, and troubleshooting. In this paper, we outline a multi-operator service metrics prediction framework using federated learning that allows privacy-preserved knowledge-sharing across operators for improved model performance, and also reduced requirements on data transfer within an operator network. Federated learning is compared against local and central learning strategies for multi-operator performance prediction, and it is shown to balance the requirements on data privacy, model performance, and the network overhead. Further, the paper provides insights on how data heterogeneity affects model performance, where the conclusion is that standard federated learning has certain robustness to data heterogeneity. Finally, we discuss the challenges related to training a federated learning model with a limited budget on the communication rounds. The evaluation is performed using a set of realistic publicly available data traces, that are adapted specifically for the purpose of studying multi-operator service performance prediction.

Doubly contrastive representation learning for federated image recognition

This paper focuses on the problem of personalized federated learning (FL) with the schema of contrastive learning (CL), which is to implement collaborative pattern classification by many clients. The traditional FL frameworks mostly facilitate the global model for the server and the local models for the clients to be similar, often ignoring the data heterogeneity of the clients. Aiming at achieving better performance in clients, this study introduces a personalized federated contrastive learning model, dubbed PerFCL, by proposing a new approach to doubly contrastive representation learning (DCL). Concretely, PerFCL borrows the DCL scheme, where one CL loss compares the shared parts of local models with the global model and the other CL loss compares the personalized parts of local models with the global model. To encourage the difference between the two parts, we created a double optimization problem composed of maximizing the comparison agreement for the former and minimizing the comparison agreement for the latter. We evaluated the proposed model on three publicly available data sets for federated image classification. Experiment results show that PerFCL benefits from the proposed DCL strategy and performs better than the state-of-the-art federated-learning models.

Misbehavior detection system with semi-supervised federated learning

V2X communications can enhance transportation safety by exchanging safety information between vehicles, road infrastructures, networks, and pedestrians. However, the safety messages are vulnerable to disruption from faulty components or an attack that can cause misinformation. Recently, a machine learning-based misbehavior detection system (MDS) has been widely investigated to detect the misbehaving vehicles to secure the V2X communications. Nonetheless, machine learning models need sufficient labeled data for learning purposes. However, the volume of unlabeled data is usually larger than that of labeled data in practice. Moreover, transferring the large dataset to a centralized learning model will consume much bandwidth. Thus, we propose a semi-supervised federated learning MDS to overcome the limitations of unlabeled data and bring the training close to the data sources to reduce the bandwidth to the core network. Overall, our model with only limited labeled data training (5%–30%) can achieve the F1-score up to 0.96 and the recall up to 0.95. The F1-score is up to 0.26 higher and the recall is up to 0.29 higher than the performance of centralized supervised learning. The federated learning model can reduce the core network bandwidth utilization by up to 95%.

CAreFL: Enhancing smart healthcare with Contribution‐Aware Federated Learning

AbstractArtificial intelligence (AI) is a promising technology to transform the healthcare industry. Due to the highly sensitive nature of patient data, federated learning (FL) is often leveraged to build models for smart healthcare applications. Existing deployed FL frameworks cannot address the key issues of varying data quality and heterogeneous data distributions across multiple institutions in this sector. In this paper, we report our experience developing and deploying the Contribution‐Aware Federated Learning (CAreFL) framework for smart healthcare. It provides an efficient and accurate approach to fairly evaluate FL participants’ contributions to model performance without exposing their private data, and improves the FL model training protocol by allowing the best performing intermediate sub‐models to be distributed to participants for FL training. Since its deployment by Yidu Cloud Technology Inc. in March 2021, CAreFL has served eight well‐established medical institutions in China to build healthcare decision support models. It can perform contribution evaluations close to three times faster than the best existing approach and has improved the average accuracy of the resulting models by more than 2% compared to the previous system (which is significant in industrial settings). To the best of our knowledge, it is the first CAreFL successfully deployed in the healthcare industry.

FedTAR: Task and Resource-Aware Federated Learning for Wireless Computing Power Networks

In the 6G era, the proliferation of data and data-intensive applications poses unprecedented challenges on the current communication and computing networks. The collaboration among cloud computing, edge computing, and networking is imperative to process such massive data, eventually realizing ubiquitous computing and intelligence. In this article, we propose a wireless computing power network (WCPN) by orchestrating the computing and networking resources of heterogeneous nodes toward specific computing tasks. To enable intelligent service in WCPN, we design a task and resource-aware federated learning model, coined FedTAR, which minimizes the sum energy consumption of all computing nodes by the joint optimization of the computing strategies of individual computing nodes and their collaborative learning strategy. Based on the solution of the optimization problem, the neural network depth of computing nodes and the collaboration frequency among nodes are adjustable according to specific computing task requirements and resource constraints. To further adapt to heterogeneous computing nodes, we then propose an energy-efficient asynchronous aggregation algorithm for FedTAR, which accelerates the convergence speed of federated learning in WCPN. Numerical results show that the proposed scheme outperforms the existing studies in terms of learning accuracy, convergence rate, and energy saving.

Adaptive Upgrade of Client Resources for Improving the Quality of Federated Learning Model

Conventional systems are usually constrained to store data in a centralized location. This restriction has either precluded sensitive data from being shared or put its privacy on the line. Alternatively, federated learning (FL) has emerged as a promising privacy-preserving paradigm for exchanging model parameters instead of private data of Internet of Things (IoT) devices known as clients. FL trains a global model by communicating local models generated by selected clients throughout many communication rounds until ensuring high learning performance. In these settings, the FL performance highly depends on selecting the best available clients. This process is strongly related to the quality of their models and their training data. Such selection-based schemes have not been explored yet, particularly regarding participating clients having high-quality data yet with limited resources. To address these challenges, we propose in this article FedAUR, a novel approach for an adaptive upgrade of clients resources in FL. We first introduce a method to measure how a locally generated model affects and improves the global model if selected for aggregation without revealing raw data. Next, based on the significance of each client parameters and the resources of their devices, we design a selection scheme that manages and distributes available resources on the server among the appropriate subset of clients. This client selection and resource allocation problem is thus formulated as an optimization problem, where the purpose is to discover and train in each round the maximum number of samples with the highest quality in order to target the desired performance. Moreover, we present a Kubernetes-based prototype that we implemented to evaluate the performance of the proposed approach.

Federated learning using game strategies: State-of-the-art and future trends

Federated learning (FL) is a new and promising paradigm that allows devices to learn without sharing data with the centralized server. It is often built on decentralized data where edge nodes use the internet of everything to mitigate the malicious attacks. The server gives incentive to all the participants according to their individual contributions. For profit maximization, each participating node balances between training rewards and costs. Game theory (GT) is a mathematical optimization technique that can be used to solve problems in wireless communication including security, resource allocation, power management, node rewards and punishments, and balancing numerous trade-offs. The complicated interactions between the server and the edge devices are interpreted using GT to maximize their utility. In this review article, we present an overview of the latest research on GT-based FL models for profit maximization, authentication, privacy management, trust management, and threat detection. This study also investigates the bibliometric analysis covering the period from 2019 to 2022 with an emphasis on various mechanisms of FL for GT applications. This article seeks to fill the gap by exploring the significant works highlighting the authors, citations, algorithms used, findings, and applications in this field. Based on the findings, we conclude this article with several key challenges and future approaches for researchers to implement an efficient GT-based FL model.

Federated Learning-Based Resource Management with Blockchain Trust Assurance in Smart IoT

Resource management is a key issue that needs to be addressed in the future smart Internet of Things (IoT). This paper focuses on a Federated Learning (FL)-based resource management mechanism in IoT. It incorporates blockchain technology to guarantee the security of the FL model parameters exchange. We propose an IoT resource management framework incorporating blockchain and federated learning technologies; then, a specific FL-based resource management with a blockchain trust assurance algorithm is given. We use a Support Vector Machine (SVM) classifier to detect malicious nodes in order to avoid the impact on the performance of the FL-based algorithm. Finally, we perform simulation to verify the SVM classification effect and the proposed algorithm performance. The results show that the SVM-based malicious node identification accuracy can be acceptable. Moreover, the proposed algorithm obtains better performance when malicious nodes are excluded from the FL selected participant.