Federated Learning Model Research Articles

Adaptive Resource Allocation for Blockchain-Based Federated Learning in Internet of Things

The fast development of mobile communication and artificial intelligence (AI) technologies greatly promotes the prosperity of the Internet of Things (IoT), where various types of IoT devices can perform more intelligent tasks. Considering the privacy leakage and limited communication resources, federated learning (FL) has emerged to enable devices to collaboratively train AI models based on their local data without raw data exchanges. Nevertheless, it is still challenging for guaranteeing any FL models to be effective due to the sluggish willingness of IoT devices and the model poisoning attacks in the FL. To address these issues, in this paper, we introduce blockchain technology and propose a blockchain-based FL framework for supporting a trustworthy and reliable FL paradigm in IoT. In the proposed framework, we design a committee-based participant selection mechanism that selects the aggregate node and local model updates dynamically to construct the global model. Moreover, considering the trade-off between the energy consumption and the convergence rate of the FL model, we perform the channel allocation, block size adjustment, and block producer selection jointly. Since the remaining resources, handling transactions, and channel conditions are dynamically varying (i.e., stochastic environment), we formulate the problem as a Markov decision process (MDP) and adopt a deep reinforcement learning (DRL)-based algorithm to solve it. The simulation results demonstrate the effectiveness of the proposed framework and show the superior performance of the DRL-based resource allocation algorithm compared with other baseline methods in terms of energy consumption.

Facilitating Early-Stage Backdoor Attacks in Federated Learning With Whole Population Distribution Inference

The development of the Internet of Things (IoT) combined with the emergence of federated learning (FL) makes it possible for mobile edge computing (MEC) to gain insight from physically separated data without violating privacy or burdening communication. Due to the distributed nature of MEC devices, researchers have uncovered that the FL is vulnerable to backdoor attacks, which aim at injecting a subtask into the FL without corrupting the performance of the main task. The backdoor attack achieves high accuracy on both the main task and the backdoor subtask when injected at FL model convergence. However, the effectiveness of the backdoor is weak when injected in early training stage. In this paper, we strengthen the early-injected backdoor attack by using information leakage. We show that FL convergence can be expedited if the clients dataset mimicks the distribution and gradients of the whole population. Based on this observation, we propose a two-phase backdoor attack, which includes a preliminary phase for the subsequent backdoor attack. Taking advantage of the preliminary phase, the later injected backdoor achieves better effectiveness, as the backdoor effect is less likely to be diluted by normal model updates. Extensive experiments are conducted on the MNIST dataset under various data heterogeneity settings to evaluate the effectiveness of the proposed backdoor attack. The results show that the proposed backdoor outperforms existing backdoor attacks in both success rate and longevity, even when defense mechanisms are in place.

Improve individual fairness in federated learning via adversarial training

Federated learning (FL) has been widely investigated these years. Since FL will be universally applied in the real world, the fairness issue involved is worthy of attention, while there are few relevant studies. Unlike previous work on group fairness in FL or fairness in centralized machine learning, this paper firstly considers both privacy and individual fairness and proposes promoting individual fairness in FL through distributionally adversarial training without violating data privacy. Specifically, we assume a model satisfying individual fairness as one robust to certain sensitive perturbations, which aligns with the goal of adversarial training. Then we transform the task of training an individually fair FL model into an adversarial training task. To obey the FL requirement of keeping data on clients privately, we execute the adversarial training task on the client side distributionally. Extensive experimental results on two real datasets collectively demonstrate the effectiveness of our proposed method, which not only improves individual fairness significantly but improves group fairness at the same time.

Asynchronous adaptive federated learning for distributed load forecasting with smart meter data

Load forecasting is essential for the operation and planning of a utility company. Recent large-scale smart meter deployments enabled the collection of fine-grained load data and created opportunities for sensor-based load forecasting. Machine learning has achieved great successes in load forecasting; however, conventional machine learning techniques require data transfer to the cloud or another centralized location for model training. This not only exposes data to privacy and security risks but also, with a large number of smart meters, increases network traffic. Federated Learning (FL) has a potential to alleviate mentioned concerns by training a single ML model in a distributed manner without requiring participants to share their data. Consequently, this paper proposes FedNorm, a novel asynchronous FL approach for load forecasting with smart meter data. While most FL strategies are synchronous and require all clients to complete local training in each round of aggregation, FedNorm is asynchronous and aggregates updates without waiting for lagging clients. To achieve this, FedNorm measures the clients contributions considering similarities of local and global models as well as the loss function magnitudes. The experiments demonstrate that FedNorm achieves higher accuracy than seven state-of-the-art FL techniques. Moreover, experiments show that FedNorm converges in fewer communication rounds compared to other FL models.

CF4FL: A Communication Framework for Federated Learning in Transportation Systems

Federated Learning (FL) is a promising technique to enhance the safety and efficiency of intelligent transportation systems. While FL has been extensively studied, the communication and networking challenges related to the operations of FL in dynamic yet dense vehicular networks remain under-explored. Limited storage and communication capacities of individual vehicles throttle the timely training of an FL model in distributed vehicular networks. In this paper, we present a communication framework for FL (CF4FL) in transportation systems. CF4FL aims to accelerate the convergence of FL training process through the innovation of two complementary networking components: (i) a deadline-driven vehicle scheduler (DDVS), and (ii) a concurrent vehicle polling scheme (CVPS). DDVS identifies a subset of vehicles for local model training in each iteration of FL, with the aim of minimizing data loss while respecting the deadline constraints derived from vehicles’ storage, computation, and energy budgets. CVPS takes advantage of multiple antennas on an edge server to enable concurrent local model transmissions in dynamic vehicular networks, thereby reducing the airtime overhead of each FL iteration. We have evaluated CF4FL through a blend of experimentation and simulation. Trace-driven simulation shows that, compared to existing scheduling and transmission schemes, CF4FL reduces the convergence time of FL training by 39%.

Worker-Centric Model Allocation for Federated Learning in Mobile Edge Computing

Federated Learning (FL) is believed as a promising manner of distributed machine learning for 5G and future 6G networks in the context of mobile edge computing (MEC). From the worker’s viewpoint, a problem is that no incentive drives them to participate in FL. Thus, different from the conventional server-centric model allocation methods, we study the worker-centric strategy in this paper. Because communicating with the FL server and training an FL model locally are energy-hungry, a dilemma for each worker is how to make the tradeoff between participating in FL training and the volume restriction of its battery. To address this issue, we formulate a WorkerFirst problem. Its NP-hardness is also proved. Next, we devise a DDQN algorithm and a DQL algorithm to strive for near-optimal decisions for each worker. These two algorithms are proposed by leveraging the deep reinforcement learning framework, while considering the energy consumption, training timespan, and the communication overheads of workers, simultaneously. The benefit of adopting such a DRL-based decision-making framework is that workers can execute the algorithms locally and dynamically adapt to the varying MEC environment. The evaluation results show that the proposed DDQN and DQL algorithms can learn good policies without knowing any prior knowledge of network conditions.

Network Flow based Abnormal Behavior feature Extraction for DDoS Attack Classification used Adaptive Federated Learning Model

Network Flow based Abnormal Behavior feature Extraction for DDoS Attack Classification used Adaptive Federated Learning Model

Accuracy Rate Maximization in Edge Federated Learning With Delay and Energy Constraints

Deploying deep learning models on edge servers can effectively alleviate the pressure on cloud data centers in computing, communication, and energy, etc. In this study, we attempt to extend this deployment scheme from edge servers to mobile terminals to further save spectrum resources, utilize communication resources, and provide intelligent applications. Unfortunately, the learning model experiences a slow convergence speed, and the mobile terminal consumes much energy if the model is only trained on the local devices with weak computing power. Therefore, we deploy deep learning algorithms (both in edge servers and mobile terminals) by considering the delay and energy consumption constraints of terminals in task processing. Specifically, a lightweight federated learning model is first proposed through the cooperative training between edge servers and mobile terminals. The trained model is then migrated to mobile terminals. Finally, image identification is considered to verify the effectiveness and efficiency of our proposed model. The experimental results show that the accuracy of the model reaches 90% using two datasets and three network models. In addition, it further reduces the processing delay and saves energy.

Joint Device Selection and Bandwidth Allocation for Cost-Efficient Federated Learning in Industrial Internet of Things

Along with the deployment of Industrial Internet of Things (IIoT), massive amounts of industrial data have been generated at the network edge, driving the evolution of edge machine learning (ML). But during the ML model training, it may bring privacy leakage by traditional central methods. To address this issue, federated learning (FL) has been proposed as a distributed learning framework for training a global model without uploading raw data to protect data privacy. Since the communication and computing resources are usually limited in IIoT networks, how to reasonably select device and allocate bandwidth is crucial for the FL model training. Therefore, this paper proposes a joint edge device selection and bandwidth allocation scheme for FL to minimize the time-averaged cost under the given long-term energy budget and delay constraints in IIoT system. To tackle with this long-term optimization problem, we construct a virtual energy deficit queue and leverage the Lyapunov optimization theory to transform it into a list of round-wise drift-plus-cost minimization problems firstly. Then, we design an iterative algorithm to allocate reasonable bandwidth and select appropriate devices to achieve cost minimization while satisfying the energy consumption constraints. Besides, we develop an optimality analysis of the average cost and energy violation for our proposed scheme. Extensive experiments verify that our proposed scheme can achieve superior performance in cost efficiency over other schemes while guaranteeing FL training performance.

FBN: Federated Bert Network with client-server architecture for cross-lingual signature verification

Online signature verification has a great challenge due to the poor performance of deep learning techniques on cross-lingual datasets under privacy constraints. In this paper, we propose a novel Federated Bert Network (FBN) by embedding the Bidirectional Encoder Representations from Transformers (Bert) into a Federated Learning (FL) framework with client-server architecture. A new Length Alignment Algorithm is employed to unify the signature pairs’ sequence length, and the input representations are fed into the different clients to complete the independent learning of local-models. In addition, the server (coordinator) uses the improved Federated Average Algorithm with Reward-Punishment Mechanism (FedAvgRP) to aggregate these local-models and further generate a global-model. After multiple iterations, the optimal model can be obtained and cross-tested on four datasets (SVC 2004, MCYT-330, BioecurID, and Ours) with skilled forged (random forged) EERs of 7.65% (4.76%), 10.73% (8.46%), 10.09% (7.13%), and 8.28% (5.74%), respectively, far higher than that of the independent learning of state-of-the-art methods. Compared with the domain adaptation and improved FL models, our FBN model performs best in random and skilled forgery scenarios. Moreover, the FedAvgRP algorithm helps our model maintain high performance in the face of data attacks.

A Novel Disease Prediction Model Based on Blockchain and Federated Learning

With the advent of the era of big data, the model architecture of disease prediction has been transformed from cloud computing architecture to edge computing architecture to alleviate problems, such as single point of failure, huge burden of computing and long communication delays.However, due to the lack of trust among edge computing participants (herein referred to as medical institutions with data resources or data storage nodes), the patients’ personal privacy and security issues are still challenging, and there is an urgent need to propose new solutions with higher security. In this paper, we proposed a new disease prediction model with the edge computing framework based on blockchain and federated learning technology to realise full mining of data value, as well as protecting local data sharing security and privacy. In the process of federated learning with blockchain as the underlying platform working environment, we propose the concept of model accuracy as the selection indicator of data nodes in federated learning, which improves the accuracy of the federated learning model and optimizes the training delay.

Federated Learning for Medical Image Analysis with Deep Neural Networks.

Medical image analysis using deep neural networks (DNN) has demonstrated state-of-the-art performance in image classification and segmentation tasks, aiding disease diagnosis. The accuracy of the DNN is largely governed by the quality and quantity of the data used to train the model. However, for the medical images, the critical security and privacy concerns regarding sharing of local medical data across medical establishments precludes exploiting the full DNN potential for clinical diagnosis. The federated learning (FL) approach enables the use of local model's parameters to train a global model, while ensuring data privacy and security. In this paper, we review the federated learning applications in medical image analysis with DNNs, highlight the security concerns, cover some efforts to improve FL model performance, and describe the challenges and future research directions.

Aligning Federated Learning with Existing Trust Structures in Health Care Systems.

Patient-centered health care information systems (PHSs) on peer-to-peer (P2P) networks (e.g., decentralized personal health records) enable storing data locally at the edge to enhance data sovereignty and resilience to single points of failure. Nonetheless, these systems raise concerns on trust and adoption in medical workflow due to non-alignment to current health care processes and stakeholders' needs. The distributed nature of the data makes it more challenging to train and deploy machine learning models (using traditional methods) at the edge, for instance, for disease prediction. Federated learning (FL) has been proposed as a possible solution to these limitations. However, the P2P PHS architecture challenges current FL solutions because they use centralized engines (or random entities that could pose privacy concerns) for model update aggregation. Consequently, we propose a novel conceptual FL framework, CareNetFL, that is suitable for P2P PHS multi-tier and hybrid architecture and leverages existing trust structures in health care systems to ensure scalability, trust, and security. Entrusted parties (practitioners' nodes) are used in CareNetFL to aggregate local model updates in the network hierarchy for their patients instead of random entities that could actively become malicious. Involving practitioners in their patients' FL model training increases trust and eases access to medical data. The proposed concepts mitigate communication latency and improve FL performance through patient-practitioner clustering, reducing skewed and imbalanced data distributions and system heterogeneity challenges of FL at the edge. The framework also ensures end-to-end security and accountability through leveraging identity-based systems and privacy-preserving techniques that only guarantee security during training.

FedIPR: Ownership Verification for Federated Deep Neural Network Models.

Federated learning models are collaboratively developed upon valuable training data owned by multiple parties. During the development and deployment of federated models, they are exposed to risks including illegal copying, re-distribution, misuse and/or free-riding. To address these risks, the ownership verification of federated learning models is a prerequisite that protects federated learning model intellectual property rights (IPR) i.e., FedIPR. We propose a novel federated deep neural network (FedDNN) ownership verification scheme that allows private watermarks to be embedded and verified to claim legitimate IPR of FedDNN models. In the proposed scheme, each client independently verifies the existence of the model watermarks and claims respective ownership of the federated model without disclosing neither private training data nor private watermark information. The effectiveness of embedded watermarks is theoretically justified by the rigorous analysis of conditions under which watermarks can be privately embedded and detected by multiple clients. Moreover, extensive experimental results on computer vision and natural language processing tasks demonstrate that varying bit-length watermarks can be embedded and reliably detected without compromising original model performances. Our watermarking scheme is also resilient to various federated training settings and robust against removal attacks.

316 Neurosurgical Federated Learning: A Multi-Center Collaboration to Detect Intracranial Hemorrhage Without Directly Sharing Data

INTRODUCTION: The development of accurate and generalizable machine learning algorithms requires sufficient quantities of diverse data. This poses a challenge in healthcare due to the sensitive and siloed nature of biomedical information. Decentralized algorithms through a federated learning (FL) paradigm avoid the need for data aggregation by instead distributing algorithms to the data itself before centrally updating one global model. METHODS: Five academic neurosurgery departments across the US collaborated to establish a federated network using computed tomography (CT) scans for prototyping. We trained a convolutional neural network to detect the presence of ICH through FL and benchmarked this against a standard, centralized training approach. Models were validated on each site’s data and tested on a held-out dataset to determine the area under the ROC curve (AUROC). RESULTS: A federated network of practicing neurosurgeon-scientists was successfully initiated and used to train a model for predicting ICH across the US. The federated model achieved an average AUROC of 0.9487 at predicting all types of ICH compared to a benchmark non-FL model AUROC of 0.9753, although performance varied by hemorrhage subtype. Subdural bleeds had the lowest AUROC (0.9257) while intraventricular bleeds had the highest (0.9751) on hold-out data. The global FL model consistently achieved top-three performance (of five) when validated on data from each site, suggesting improved generalizability. A qualitative survey of participants revealed that a majority found the process of connecting to the federated network easy. CONCLUSIONS: This study demonstrates the feasibility of implementing a federated network for multi-institutional collaboration amongst clinicians and using FL to conduct machine learning research without the transfer of data between sites, thereby opening up a new paradigm for neurosurgical collaboration.

Federated learning for scam classification in small Indonesian language dataset: an initial study

Most digital phishing or scam trick users into fraudulent links and is more effective against users with low technology literacy, like in Indonesia. Machine learning is widely used for scam classification, but most require sending the messages to a centralized server. This induces privacy concern as messages might contain private data. Federated learning (FL) was proposed to allow user devices to train models locally without sending data to server. In this work, we examined the use of FL with gated recurrent unit (GRU) model for classifying scam messages in Indonesian language with small dataset. We provided two FL-based baseline models (FedAvg and daisy-chained algorithms) and a dataset for scam classification in Indonesian language. We examined the models based on these performance metrics; precision, recall, F1, selectivity, and balanced accuracy. Despite the performance, we pointed out characteristics of the FL algorithms and the hyperparameters for this use case as pointers towards fine-tuning these baseline models. Overall, the FL model with FedAvg algorithm performed better in all metrics except recall.

Decentralized P2P Federated Learning for Privacy-Preserving and Resilient Mobile Robotic Systems

Swarms of mobile robots are being widely applied for complex tasks in various practical scenarios toward modern smart industry. Federated learning (FL) has been developed as a promising privacy-preserving paradigm to tackle distributed machine learning tasks for mobile robotic systems in 5G and beyond networks. However, unstable wireless network conditions of the complex and harsh working environment may lead to poor communication quality and bring big challenges to traditional centralized global training in FL models. In this article, a Peer-to-Peer (P2P) based Privacy-Perceiving Asynchronous Federated Learning (PPAFL) framework is introduced to realize the decentralized model training for secure and resilient modern mobile robotic systems in 5G and beyond networks. Specifically, a reputation-aware coordination mechanism is designed and addressed to coordinate a group of smart devices dynamically into a virtual cluster, in which the asynchronous model aggregation is conducted in a decentralized P2P manner. A secret sharing based communication mechanism is developed to ensure an encrypted P2P FL process, while a Secure Stochastic Gradient Descent (SSGD) scheme is integrated with an Autoencoder and a Gaussian mechanism is developed to ensure an anonymized local model update, communicating within a few neighboring clients. The case study based experiment and evaluation in three different application scenarios demonstrate that the PPAFL can effectively improve the security and resilience issues compared with the traditional centralized approaches for smart mobile robotic applications in 5G and beyond networks.

A Clustered Federated Learning Method of User Behavior Analysis Based on Non-IID Data

Federated learning (FL) is a novel distributed machine learning paradigm. It can protect data privacy in distributed machine learning. Hence, FL provides new ideas for user behavior analysis. User behavior analysis can be modeled using multiple data sources. However, differences between different data sources can lead to different data distributions, i.e., non-identically and non-independently distributed (Non-IID). Non-IID data usually introduce bias in the training process of FL models, which will affect the model accuracy and convergence speed. In this paper, a new federated learning algorithm is proposed to mitigate the impact of Non-IID data on the model, named federated learning with a two-tier caching mechanism (FedTCM). First, FedTCM clustered similar clients based on their data distribution. Clustering reduces the extent of Non-IID between clients in a cluster. Second, FedTCM uses asynchronous communication methods to alleviate the problem of inconsistent computation speed across different clients. Finally, FedTCM sets up a two-tier caching mechanism on the server for mitigating the Non-IID data between different clusters. In multiple simulated datasets, compared to the method without the federated framework, the FedTCM is maximum 15.8% higher than it and average 12.6% higher than it. Compared to the typical federated method FedAvg, the accuracy of FedTCM is maximum 2.3% higher than it and average 1.6% higher than it. Additionally, FedTCM achieves more excellent communication performance than FedAvg.

Multi-party Diabetes Mellitus risk prediction based on secure federated learning

At present, there is a lack of data exchange between medical institutions at the municipal level in the prevention and control of chronic disease. In this paper, we study Diabetes Mellitus risk prediction based on data securely shared between hospital and medical examination center, where federated learning algorithms are used and security challenges are analyzed. Targeted at a general disease prediction problem, we design customized secure protocols for regression family and tree family algorithms to solve the problem of potential private data leakage. Prediction of diabetes risks is taken as a typical example, and XGBoost, LightGBM, Neural Network, Logistic Regression algorithms are used for joint data modeling between different organizations, with the support from the “Seceum” Federated Learning Platform we first introduced in this paper. Extensive results on two datasets, including computational costs, recall and precision metrics, significant tests, show that using federated learning models we can make better use of the patient data between different organizations and deliver a reliable and improved prediction of Diabetes Mellitus risks.

Federated learning for performance prediction in multi-operator environments

Telecom vendors and operators deliver services with strict requirements on performance, over complex and sometimes partly shared network infrastructures. A key enabler for network and service management in such environments is knowledge sharing, and the use of data-driven models for performance prediction, forecasting, and troubleshooting. In this paper, we outline a multi-operator service metrics prediction framework using federated learning that allows privacy-preserved knowledge-sharing across operators for improved model performance, and also reduced requirements on data transfer within an operator network. Federated learning is compared against local and central learning strategies for multi-operator performance prediction, and it is shown to balance the requirements on data privacy, model performance, and the network overhead. Further, the paper provides insights on how data heterogeneity affects model performance, where the conclusion is that standard federated learning has certain robustness to data heterogeneity. Finally, we discuss the challenges related to training a federated learning model with a limited budget on the communication rounds. The evaluation is performed using a set of realistic publicly available data traces, that are adapted specifically for the purpose of studying multi-operator service performance prediction.