Federated Learning Scheme Research Articles

Federated Learning with Pareto Optimality for Resource Efficiency and Fast Model Convergence in Mobile Environments.

Federated learning (FL) is an emerging distributed learning technique through which models can be trained using the data collected by user devices in resource-constrained situations while protecting user privacy. However, FL has three main limitations: First, the parameter server (PS), which aggregates the local models that are trained using local user data, is typically far from users. The large distance may burden the path links between the PS and local nodes, thereby increasing the consumption of the network and computing resources. Second, user device resources are limited, but this aspect is not considered in the training of the local model and transmission of the model parameters. Third, the PS-side links tend to become highly loaded as the number of participating clients increases. The links become congested owing to the large size of model parameters. In this study, we propose a resource-efficient FL scheme. We follow the Pareto optimality concept with the biased client selection to limit client participation, thereby ensuring efficient resource consumption and rapid model convergence. In addition, we propose a hierarchical structure with location-based clustering for device-to-device communication using k-means clustering. Simulation results show that with prate at 0.75, the proposed scheme effectively reduced transmitted and received network traffic by 75.89% and 78.77%, respectively, compared to the FedAvg method. It also achieves faster model convergence compared to other FL mechanisms, such as FedAvg and D2D-FedAvg.

ESVFL: Efficient and secure verifiable federated learning with privacy-preserving

Federated learning has been widely applied as a distributed machine learning method in various fields, allowing a global model to be trained by sharing local gradients instead of raw data. However, direct sharing of local gradients still carries the risk of privacy data leakage, and the malicious server might falsify aggregated result to disrupt model updates. To address these issues, a lot of privacy-preserving and verifiable federated learning schemes have been proposed. However, existing schemes suffer from significant computation overhead in either encryption or verification. In this paper, we present ESVFL, an efficient and secure verifiable federated learning scheme with privacy-preserving. This scheme can simultaneously achieve low computation overhead for encryption and verification on the user side. We design an efficient privacy-preserving method to encrypt the users’ local gradients. Using this method, the computation and communication overheads of encryption on the user side is independent of the number of users. Users can efficiently verify the correctness of aggregated results returned by the cloud servers using cross-verification. During the verification process, there is no interaction among users and no additional computation is required. Furthermore, we also construct an efficient method to address the issue of user dropout. When some users drop out, online users do not incur any additional computation and communication overheads, while guaranteeing the correctness of the aggregated result of online users’ encrypted gradients. The security analysis and the performance evaluation demonstrate that ESVFL is secure and can achieve efficient encryption and verification.

PT-ADP: A personalized privacy-preserving federated learning scheme based on transaction mechanism

Differential privacy (DP) is a widely used technique for enhancing privacy in federated learning (FL) frameworks, whereby noise is added to the datasets or learning parameters to prevent attackers from discovering which client (i.e., the data owner) the sensitive information originated from. More noise implies higher privacy protection but also leads to a decrease in model accuracy. The privacy budget is a key parameter that controls the amount of noise. Most existing methods focus on the same privacy budget, ignoring the various privacy protection requirements of the clients. In this study, we propose PT-ADP, which is a personalized privacy-preserving federated learning scheme. First, a privacy transaction mechanism (PT) is proposed to realize personalized allocation of privacy budget for clients through “quotation-allocation.” Subsequently, an adaptive transmission data perturbation mechanism (ADP) is proposed to save a large amount of privacy budget and improve accuracy under the premise of providing sufficient privacy protection. The security, convergence, efficiency, and specific parameters of the proposed scheme were theoretically analyzed and verified through extensive experiments. PT-ADP can improve model availability while providing the same level of privacy protection compared to other schemes, without increasing complexity and communication overhead.

Privacy-preserving federated learning based on partial low-quality data

AbstractTraditional machine learning requires collecting data from participants for training, which may lead to malicious acquisition of privacy in participants’ data. Federated learning provides a method to protect participants’ data privacy by transferring the training process from a centralized server to terminal devices. However, the server may still obtain participants’ privacy through inference attacks and other methods. In addition, the data provided by participants varies in quality, and the excessive involvement of low-quality data in the training process can render the model unusable, which is an important issue in current mainstream federated learning. To address the aforementioned issues, this paper proposes a Privacy Preserving Federated Learning Scheme with Partial Low-Quality Data (PPFL-LQDP). It can achieve good training results while allowing participants to utilize partial low-quality data, thereby enhancing the privacy and security of the federated learning scheme. Specifically, we use a distributed Paillier cryptographic mechanism to protect the privacy and security of participants’ data during the Federated training process. Additionally, we construct composite evaluation values for the data held by participants to reduce the involvement of low-quality data, thereby minimizing the negative impact of such data on the model. Through experiments on the MNIST dataset, we demonstrate that this scheme can complete the model training of federated learning with the participation of partial low-quality data, while effectively protecting the security and privacy of participants’ data. Comparisons with related schemes also show that our scheme has good overall performance.

Federated Pseudo-Sample Clustering Algorithm: A Label-Personalized Federated Learning Scheme Based on Image Clustering

In the realm of personalized federated learning, some current methods substitute shared parameters with shared samples created by Generative Adversarial Networks (GANs). This enables each client to independently design the architecture of their neural network model. However, this approach still fails to overcome the restriction of enforcing uniform labels on client models that partake in training. To tackle this problem, we suggest the Federated Pseudo-Sample Clustering Algorithm (LPFL-GD). This method allows clients to train cooperatively under personalized labeling conditions. The approach uses the local model as a discriminator and forms a GAN network with the generator to produce a sample set. This set is then uploaded to the central server. The uploaded shared samples are clustered and divided into several clusters by introducing the DBSCAN algorithm on the central server. When filtering client samples, we obtain the labels of shared samples from each client in a cluster and correct the label of the entire cluster. We then merge the corrected samples with the local dataset to extend it. Our approach improves model performance, even when different clients label the same type of data differently. Compared with their performance before participating in federated learning, our approach can improve client model accuracy by up to 13.4%. We replicated other methods in the same environment and found that the local model accuracy of those methods not only improved very little but even decreased by up to 34.5%.

IODeep: An IOD for the introduction of deep learning in the DICOM standard

Background and objectiveIn recent years, Artificial Intelligence (AI) and in particular Deep Neural Networks (DNN) became a relevant research topic in biomedical image segmentation due to the availability of more and more data sets along with the establishment of well known competitions. Despite the popularity of DNN based segmentation on the research side, these techniques are almost unused in the daily clinical practice even if they could support effectively the physician during the diagnostic process. Apart from the issues related to the explainability of the predictions of a neural model, such systems are not integrated in the diagnostic workflow, and a standardization of their use is needed to achieve this goal. MethodsThis paper presents IODeep a new DICOM Information Object Definition (IOD) aimed at storing both the weights and the architecture of a DNN already trained on a particular image dataset that is labeled as regards the acquisition modality, the anatomical region, and the disease under investigation. ResultsThe IOD architecture is presented along with a DNN selection algorithm from the PACS server based on the labels outlined above, and a simple PACS viewer purposely designed for demonstrating the effectiveness of the DICOM integration, while no modifications are required on the PACS server side. Also a service based architecture in support of the entire workflow has been implemented. ConclusionIODeep ensures full integration of a trained AI model in a DICOM infrastructure, and it is also enables a scenario where a trained model can be either fine-tuned with hospital data or trained in a federated learning scheme shared by different hospitals. In this way AI models can be tailored to the real data produced by a Radiology ward thus improving the physician decision making process.Source code is freely available at https://github.com/CHILab1/IODeep.git.

PPBRFL: Privacy-Preserving Byzantine-Robust Federated Learning

Federated learning is a distributed machine learning approach that allows the neural network to be trained without exposing private user data. Despite its advantages, federated learning schemes still face two critical security challenges: user privacy disclosure and Byzantine robustness. The adversary may try to infer the private data from the trained local gradients or compromise the global model update. To tackle the above challenges, we propose PPBRFL, a privacy-preserving Byzantine-robust federated learning scheme. To resist Byzantine attacks, we design a novel Byzantine-robust aggregation method based on
 cosine similarity, which can guarantee the global model update and improve the model’s classification accuracy. Furthermore, we introduce a reward and penalty mechanism that considers users’ behavior to mitigate the impact of Byzantine users on the global model. To protect user privacy, we utilize symmetric homomorphic encryption to encrypt the users’ trained local models, which requires low computation cost while maintaining model accuracy. We conduct the experimental assessment of the performance of PPBRFL. The experimental results show that PPBRFL maintains model classification accuracy while ensuring privacy preservation and Byzantine robustness compared to traditional federated learning scheme.

Federated Medical Learning Framework Based on Blockchain and Homomorphic Encryption

Federated learning-based medical data privacy sharing can promote the development of medical industry intelligence, but limited by its own security and privacy deficiencies, federated learning still suffers from a single point of failure and privacy leakage of intermediate parameters. To address these problems, this paper proposes a privacy protection framework for medical data based on blockchain and cross-silo federated learning, using cross-silo federated learning to establish a collaborative training platform for multiple medical institutions to enhance the privacy of medical data, introducing blockchain and smart contracts to realize decentralized federated learning to enhance trust between distrustful medical institutions and solve the problem of a single point of failure. In addition, a secure aggregation scheme is designed using threshold homomorphic encryption to prevent the privacy leakage problem during parameter transmission. The experimental and analytical results show that the accuracy of this paper’s scheme is consistent with the original federated learning scheme, effectively deals with the problems of single-point failure and inference attacks of federated learning, improves system robustness, and is suitable for medical scenarios with more stringent requirements on security and accuracy.

FedDP: Dual Personalization in Federated Medical Image Segmentation.

Personalized federated learning (PFL) addresses the data heterogeneity challenge faced by general federated learning (GFL). Rather than learning a single global model, with PFL a collection of models are adapted to the unique feature distribution of each site. However, current PFL methods rarely consider self-attention networks which can handle data heterogeneity by long-range dependency modeling and they do not utilize prediction inconsistencies in local models as an indicator of site uniqueness. In this paper, we propose FedDP, a novel fed erated learning scheme with d ual p ersonalization, which improves model personalization from both feature and prediction aspects to boost image segmentation results. We leverage long-range dependencies by designing a local query (LQ) that decouples the query embedding layer out of each local model, whose parameters are trained privately to better adapt to the respective feature distribution of the site. We then propose inconsistency-guided calibration (IGC), which exploits the inter-site prediction inconsistencies to accommodate the model learning concentration. By encouraging a model to penalize pixels with larger inconsistencies, we better tailor prediction-level patterns to each local site. Experimentally, we compare FedDP with the state-of-the-art PFL methods on two popular medical image segmentation tasks with different modalities, where our results consistently outperform others on both tasks. Our code and models are available at https://github.com/jcwang123/PFL-Seg-Trans.

Auditable Federated Learning With Byzantine Robustness

Machine learning (ML) has led to disruptive innovations in many fields, such as medical diagnoses. A key enabler for ML is large training data, but existing data, such as medical data, are not fully exploited by ML because of data silos and privacy concerns. Federated learning (FL) is a promising distributed learning paradigm to address this problem. On the other hand, existing FL approaches are vulnerable to poisoning attacks or privacy leakage from a malicious aggregator or client. This article proposes an auditable FL scheme with Byzantine robustness against the aggregator and client: The aggregator is malicious but available, and the client could perform poisoning attacks. First, the Pedersen commitment scheme (PCS) for homomorphic encryption was applied to preserve privacy and for commitments to the FL process to achieve auditability. The auditability enables clients to verify the correctness and consistency of the entire FL process and to identify parties that misbehave. Second, an efficient technique of divide and conquer was designed based on PCS to allow parties to cooperate and securely aggregate gradients to defend against poisoning attacks. This technique enables clients to share no common secret key and cooperate to decrypt ciphertext, guaranteeing a client’s privacy even if some other clients are corrupted by adversaries. This technique was optimized to tolerate the dropout of clients. This article reports a formal analysis concerning privacy, efficiency, and auditability against malicious participants. Extensive experiments on various benchmark datasets show that the scheme is robust with high model accuracy against poisoning attacks.

A decentralised federated learning scheme for heterogeneous devices in cognitive IoT

Cognitive Internet of Things (IoT) technologies typically rely on substantial data collected from edge devices for data analysis and decision-making. However this reliance often leads to the inadvertent exposure of private data from smart edge devices. Federated learning (FL) is a distributed machine learning framework that protects user privacy by performing collaborative training without uploading private data. Nevertheless, applying classical FL to cognitive IoT systems to preserve privacy preservation faces significant challenges, such as central server failure and communication burden. Furthermore, when edge devices with heterogeneous data and systems participate in federated training, the learning process becomes slow and the performance of edge devices is compromised. To address these challenges, we propose a decentralised FL framework for cognitive IoT, termed DFL–MKF. In DFL– MKF, the centralised server is eliminated and each edge device is dynamically connected. We initialised models for edge devices based on computational and storage capabilities to accommodate system heterogeneity. Edge devices learned from the knowledge of multiple neighbours via knowledge transfer, and knowledge fusion was employed to aggregate the knowledge of multiple neighbours, thereby improving the performance of local models, and addressing data heterogeneity. Comprehensive experiments were performed on three image classification tasks. The results of these experiments demonstrate that the proposed method achieved superior performance compared to various baselines and improved communication efficiency.

APDPFL: Anti-Poisoning Attack Decentralized Privacy Enhanced Federated Learning Scheme for Flight Operation Data Sharing

APDPFL: Anti-Poisoning Attack Decentralized Privacy Enhanced Federated Learning Scheme for Flight Operation Data Sharing

Boosted Multi-Task Learning for Inter-District Collaborative Load Forecasting

This paper proposes a boosted multi-task learning framework for inter-district collaborative load forecasting. The proposed framework involves two subsequent stages: in the first stage, districts would collaborate under a seamlessly-integrated federated learning scheme to capture the global load pattern; in the second stage, districts would withdraw and perform local training to capture the local load patterns. The probabilistic Gradient-Boosted Regression Tree (GBRT) is applied as the bottom-level machine learning algorithm, which would allow for an easy and intuitive embodiment of the generalized multi-task learning framework. We further propose two candidate district withdrawal mechanisms to connect the two stages: the simultaneous withdrawal, which prioritizes prediction accuracy, and the dynamic withdrawal, which prioritizes training efficiency and district incentivization. The follow-up performance analyses and the case study on 11 districts of the Zhuhai city confirm the superiority of the proposed framework and district withdrawal mechanisms.

On the Security of “LSFL: A Lightweight and Secure Federated Learning Scheme for Edge Computing”

On the Security of “LSFL: A Lightweight and Secure Federated Learning Scheme for Edge Computing”

Byzantine Robust Federated Learning Scheme Based on Backdoor Triggers

Byzantine Robust Federated Learning Scheme Based on Backdoor Triggers

An Robust Secure Blockchain-Based Hierarchical Asynchronous Federated Learning Scheme for Internet of Things

An Robust Secure Blockchain-Based Hierarchical Asynchronous Federated Learning Scheme for Internet of Things

PASS: A Parameter Audit-Based Secure and Fair Federated Learning Scheme Against Free-Rider Attack

Federated Learning (FL) as a secure distributed learning framework gains interests in Internet of Things (IoT) due to its capability of protecting the privacy of participant data. However, traditional FL systems are vulnerable to Free-Rider (FR) attacks, which causes unfairness, privacy leakage and inferior performance to FL systems. The prior defense mechanisms against FR attacks assumed that malicious clients (namely, adversaries) declare less than 50% of the total amount of clients. Moreover, they aimed for Anonymous FR (AFR) attacks and lost effectiveness in resisting Selfish FR (SFR) attacks. In this paper, we propose a Parameter Audit-based Secure and fair federated learning Scheme (PASS) against FR attack. PASS has the following key features: (a) prevent from privacy leakage with less accuracy loss; (b) be effective in countering both AFR and SFR attacks; (c) work well no matter whether AFR and SFR adversaries occupy the majority of clients or not. Extensive experimental results validate that PASS: (a) has the same level as the State-Of-The-Art method in mean square error against privacy leakage; (b) defends against AFR and SFR attacks in terms of a higher defense success rate, lower false positive rate, and higher F1-score; (c) is still effective where adversaries exceed 50%, with F1-score 89% against AFR attack and F1-score 87% against SFR attack. Note that PASS produces no negative effect on FL accuracy when there is no FR adversary.

HAFedL: A Hessian-Aware Adaptive Privacy Preserving Horizontal Federated Learning Scheme for IoT Applications

HAFedL: A Hessian-Aware Adaptive Privacy Preserving Horizontal Federated Learning Scheme for IoT Applications

Federify: A Verifiable Federated Learning Scheme Based on zkSNARKs and Blockchain

Federify: A Verifiable Federated Learning Scheme Based on zkSNARKs and Blockchain

A Dynamic Clustering-Based Hierarchical Federated Learning Scheme in Internet of Vehicles

A Dynamic Clustering-Based Hierarchical Federated Learning Scheme in Internet of Vehicles