Federated Learning Process Research Articles

Federated Learning's Dynamic Defense Against Byzantine Attacks: Integrating SIFT-Wavelet and Differential Privacy for Byzantine Grade Levels Detection

Federated learning, which enables decentralized training across multiple devices while maintaining data privacy, is susceptible to Byzantine poisoning attacks. This paradigm reduces the need for centralized data storage and transmission, thereby mitigating privacy risks associated with traditional data aggregation methods. However, FL introduces new challenges, notably susceptibility to Byzantine poisoning attacks, where rogue participants can tamper with model updates, threatening the consistency and security of the aggregated model. Our approach addresses this vulnerability by implementing robust aggregation methods, sophisticated pre-processing techniques, and a novel Byzantine grade-level detection mechanism. We introduce a federated aggregation operator designed to mitigate the impact of malicious clients. Our pre-processing includes data loading and transformation, data augmentation, and feature extraction using SIFT and wavelet transforms. Additionally, we employ differential privacy and model compression to improve the robustness and performance of the federated learning framework. Our approach is assessed using a tailored neural network model applied to the MNIST dataset, achieving 97% accuracy in detecting Byzantine attacks. Our results demonstrate that robust aggregation significantly improves the resilience and performance. This comprehensive approach ensures the integrity of the federated learning process, effectively filtering out adversarial influences and sustaining high accuracy even when faced with adversarial Byzantine clients.

Telemedicine data secure sharing scheme based on heterogeneous federated learning

The forward triage characteristic of telemedicine highlights its importance again in the COVID-19 pandemic. Telemedicine can provide timely emergency response in the case of environmental or biological hazards, and the patient’s medical privacy data generated in this process can also accelerate the establishment of models for preventing and treating infectious diseases. However, the reuse process of telemedicine user privacy data based on federated learning also faces significant challenges. Differences in regions, economic levels, and grades lead to heterogeneous data and resource-constrained environments, seriously damaging the federated learning process. Besides, the weak password authentication of medical terminals and eavesdropping attacks on transmission channels may cause illegal access to terminals and platforms and leakage of sensitive data. This paper proposed a telemedicine data secure-sharing scheme based on heterogeneous federated learning. Specifically, we proposed a heterogeneous federated learning scheme with model alignment to guide telemedicine practice through the reuse of telemedicine data; in addition, we designed an SM9 threshold identity authentication scheme to guarantee that the patient’s medical privacy data is protected from leakage during the federated learning process. We evaluated our scheme using two third-party medical datasets. The evaluation results indicate that this scheme can still assist the federated learning process in resisting data heterogeneity and resource constraints with almost no performance cost.

Enhancing Federated Learning in Heterogeneous Internet of Vehicles: A Collaborative Training Approach

The current Internet of Vehicles (IoV) faces significant challenges related to resource heterogeneity, which adversely impacts the convergence speed and accuracy of federated learning models. Existing studies have not adequately addressed the problem of resource-constrained vehicles that slow down the federated learning process particularly under conditions of high mobility. To tackle this issue, we propose a model partition collaborative training mechanism that decomposes training tasks for resource-constrained vehicles while retaining the original data locally. By offloading complex computational tasks to nearby service vehicles, this approach effectively accelerates the slow training speed of resource-limited vehicles. Additionally, we introduce an optimal matching method for collaborative service vehicles. By analyzing common paths and time delays, we match service vehicles with similar routes and superior performance within mobile service vehicle clusters to provide effective collaborative training services. This method maximizes training efficiency and mitigates the negative effects of vehicle mobility on collaborative training. Simulation experiments demonstrate that compared to benchmark methods, our approach reduces the impact of mobility on collaboration, achieving large improvements in the training speed and the convergence time of federated learning.

RETRACTION: Improving Mobile Cloud Computing Service Efficiency Using Energy Competent and Federated Learning Process

RETRACTION: M. Thirunavukkarasu, and P. Shanmugapriya, “,” Trans Emerg Telecommun Technol (Early View): doi:10.1002/ett.4643.The above article, published online on 8 September 2022 in Wiley Online Library (wileyonlinelibrary.com), has been retracted by agreement between the journal Editor‐in‐Chief, Changqiao Xu; and John Wiley & Sons Ltd. The article was submitted as part of a guest‐edited special issue. Following publication, it has come to the attention of the journal that the article was accepted solely on the basis of compromised editorial handling and peer review processes. Furthermore, several articles, this one included, submitted as part of the same special issue exhibited an unusually high degree of similarity in both concept and style. Therefore, the decision to retract this article was taken.

Federated Learning-Oriented Edge Computing Framework for the IIoT.

With the maturity of artificial intelligence (AI) technology, applications of AI in edge computing will greatly promote the development of industrial technology. However, the existing studies on the edge computing framework for the Industrial Internet of Things (IIoT) still face several challenges, such as deep hardware and software coupling, diverse protocols, difficult deployment of AI models, insufficient computing capabilities of edge devices, and sensitivity to delay and energy consumption. To solve the above problems, this paper proposes a software-defined AI-oriented three-layer IIoT edge computing framework and presents the design and implementation of an AI-oriented edge computing system, aiming to support device access, enable the acceptance and deployment of AI models from the cloud, and allow the whole process from data acquisition to model training to be completed at the edge. In addition, this paper proposes a time series-based method for device selection and computation offloading in the federated learning process, which selectively offloads the tasks of inefficient nodes to the edge computing center to reduce the training delay and energy consumption. Finally, experiments carried out to verify the feasibility and effectiveness of the proposed method are reported. The model training time with the proposed method is generally 30% to 50% less than that with the random device selection method, and the training energy consumption under the proposed method is generally 35% to 55% less.

Wireless federated learning for PR identification and analysis based on generalized information

This paper introduces a novel approach to personal risk (PR) identification using federated learning (FL) in wireless communication scenarios, leveraging generalized information. The primary focus is on harnessing the power of distributed data across various wireless devices while ensuring data privacy and security, a critical concern in PR assessment. To this end, we propose an FL-based model that effectively aggregates learning from diverse, decentralized data sources to analyze the PR factors. The proposed method involves training local models on individual devices, which are then aggregated to form a comprehensive global model. This process not only preserves data privacy by keeping sensitive information on the device but also utilizes the widespread availability and connectivity of wireless devices to enhance data richness and model robustness. To address the challenges posed by the wireless environment, such as data heterogeneity and communication constraints, we further implement advanced aggregation algorithms and optimization techniques tailored to these unique conditions. We finally evaluate the performance of our proposed method based on two primary metrics of identification accuracy and convergence rate of the federated learning process. Through extensive simulations and real-world experiments, we demonstrate that our approach not only achieves high accuracy in PR identification but also ensures rapid convergence, making it a viable solution for real-time risk assessment in wireless networks.

Privacy-preserving and scalable federated blockchain scheme for healthcare 4.0

IoT plays a significant role in the growth of clinical data for identifying hazardous diseases and building drugs for patient diagnosis and medical care. Blockchain and federated learning are widely proposed in existing studies for localized data training and storage in a secure, decentralized environment. However, several studies rely on the federated averaging method in federated learning, which introduces high energy consumption during several training rounds. Frequent transactions for storing patient records in blockchain and smart contract processing present network congestion challenges. In this paper, we propose a privacy-preserving federated learning and scalable blockchain scheme. First, we present a satisfaction scoring method for model aggregation to improve energy efficiency during the federated learning process. Secondly, we design an Ethereum-based blockchain network with sidechains to process smart contract transactions separately and reduce computation overload in the blockchain mainchain. Evaluation of the proposed scheme is compared with the baseline federated average method and transaction processing of smart contracts with the Ethereum main chain. Results demonstrate reduced CPU consumption using the proposed model aggregation method, an improvement over the federated average method, and an improvement of 43.65 % in transaction processing speed using two sidechains ensuring side.

Blind Federated Learning without initial model

Federated learning is an emerging machine learning approach that allows the construction of a model between several participants who hold their own private data. This method is secure and privacy-preserving, suitable for training a machine learning model using sensitive data from different sources, such as hospitals. In this paper, the authors propose two innovative methodologies for Particle Swarm Optimisation-based federated learning of Fuzzy Cognitive Maps in a privacy-preserving way. In addition, one relevant contribution this research includes is the lack of an initial model in the federated learning process, making it effectively blind. This proposal is tested with several open datasets, improving both accuracy and precision.

Towards Energy-Aware Federated Learning via Collaborative Computing Approach

This research delves into the consequences of the high complexity of on-device operations executed during the federated learning process. We investigate how the varying computational capabilities and battery levels among mobile devices can introduce performance disparities and influence training quality. Hence, in order to deal with these challenges, we propose EAFL+, a novel energy optimization technique, that focuses on managing power consumption in devices with limited battery capacity. EAFL+ is a cloud–edge–terminal collaborative approach that provides a new architectural design for achieving power-aware FL training by leveraging resource diversity and computation offloading. The innovative scheme enables the efficient selection of an approximately-optimal offloading target, from a set of Cloud-tier, Edge-tier, and Terminal-tier resources and achieves the best cost-quality tradeoff for the devices taking part in the FL system. Our evaluation shows EAFL+ can help conserve the devices’ energy participating in training, which improves the participation rates and increases the clients’ contributions, hence achieving higher accuracy and faster convergence. Through experiments on real datasets and traces in an emulated FL environment, EAFL+ reduces the drop-out of clients to zero and enhances accuracy by up to 24% and 9% compared to EAFL and Oort, respectively.

DeFTA: A plug-and-play peer-to-peer decentralized federated learning framework

Federated learning (FL) is a pivotal catalyst for enabling large-scale privacy-preserving distributed machine learning (ML). By eliminating the need for local raw dataset sharing, FL substantially reduces privacy concerns and alleviates the isolated data problem. However, in reality, the success of FL is predominantly attributed to a centralized framework called FedAvg [1], in which workers are responsible for model training, and servers are in control of model aggregation. Nevertheless, FedAvg's centralized worker-server architecture has raised new concerns, including low scalability of the cluster, risk of data leakage, and central server failure or even defection. To overcome these challenges, we propose Decentralized Federated Trusted Averaging (DeFTA), a decentralized FL framework that serves as a plug-and-play replacement for FedAvg, bringing instant improvements to security, scalability, and fault-tolerance in the federated learning process. In essence, it primarily consists of a novel model aggregating formula with theoretical performance analysis, and a decentralized trust system (DTS) to significantly enhance system robustness. Extensive experiments conducted on six datasets and six basic models suggest that DeFTA not only exhibits comparable performance with FedAvg in a more realistic setting, but also achieves remarkable resilience even when 67% of workers are malicious.

Enhancing Federated Learning Security, Scalability, and Future Incentives

This study delves into the integration of Storj and blockchain technology within the context of federated learning (FL) and its implications for scalability and efficiency. By leveraging blockchain, we aimed to bolster security and transparency, while also addressing storage challenges through the integration of Incremental Learning. Our findings revealed that while the utilization of Storj led to marginally higher federated server storage requirements compared to local storage, particularly as the number of clients increased, there was also a slight increase in the time required for the federated learning process when Storj was integrated, especially with a larger number of clients.

Reliable federated learning based on dual-reputation reverse auction mechanism in Internet of Things

Federated learning, a promising distributed machine learning paradigm, has been used in various Internet of Things (IoT) environments to solve isolated data island issues and protect data privacy. However, since the central server in federated learning cannot detect the local training process of the client, it is vulnerable to adversarial attacks against its security and privacy by malicious clients during the training process. To address this problem, this work proposes a federated learning system model based on dual-reputation reverse auction in IoT. Specifically, due to resource constraints, not all clients participate in the federated learning process, so we propose the reputation-bid ratio based greedy half-splitting algorithm to select some clients to participate in the federated learning, which can guarantee that each client has the chance to be selected while selecting as many honest and high-quality clients as possible. Then, we propose an adaptive dropout aggregation method based on a training quality score, which can effectively defend against malicious workers’ attacks. After the completion of federated learning, we put forward a subjective evaluation incentive mechanism based on a second reputation to realize the fair incentive. Furthermore, we store and manage reputations through blockchain in our model to ensure their availability. Theoretical analysis deduces the complexity and security of our proposed model. Finally, simulation results indicate that our proposed model can achieve the highest accuracy across all test datasets compared to the baseline. Particularly, on the CIFAR10 dataset, the accuracy of our proposed model surpasses the baseline by 5% to 30%. In responding to sudden attacks initiated by normally participating workers, our model exhibits the fastest reaction time, with accuracy surpassing the baseline by 8% to 40%.

A methodology for evaluating the effectiveness of intrusion detection system based on federated learning

The article investigates the problem of applying federated learning for building intrusion detection systems. Federated learning is a way of organizing distributed learning, in which data are not transferred to a single cloud storage, and the training of models is performed locally, the formation of a common global model is performed based on the parameters of local models. The article proposes a methodology for evaluating the performance of federated trained models, including the evaluation of the training throughput. The training throughput depends on the federated learning settings, the computational resources of the interacting devices and the network bandwidth, and obviously, the use of federated learning can significantly affect the performance of the device itself. This article presents a description of the experimental setup, and the results of a series of experiments that characterize the federated learning process with different settings.

Tri-branch feature pyramid network based on federated particle swarm optimization for polyp segmentation.

Deep learning technology has shown considerable potential in various domains. However, due to privacy issues associated with medical data, legal and ethical constraints often result in smaller datasets. The limitations of smaller datasets hinder the applicability of deep learning technology in the field of medical image processing. To address this challenge, we proposed the Federated Particle Swarm Optimization algorithm, which is designed to increase the efficiency of decentralized data utilization in federated learning and to protect privacy in model training. To stabilize the federated learning process, we introduced Tri-branch feature pyramid network (TFPNet), a multi-branch structure model. TFPNet mitigates instability during the aggregation model deployment and ensures fast convergence through its multi-branch structure. We conducted experiments on four different public datasets：CVC-ClinicDB, Kvasir, CVC-ColonDB and ETIS-LaribPolypDB. The experimental results show that the Federated Particle Swarm Optimization algorithm outperforms single dataset training and the Federated Averaging algorithm when using independent scattered data, and TFPNet converges faster and achieves superior segmentation accuracy compared to other models.

A Communication-Efficient, Privacy-Preserving Federated Learning Algorithm Based on Two-Stage Gradient Pruning and Differentiated Differential Privacy

There are several unsolved problems in federated learning, such as the security concerns and communication costs associated with it. Differential privacy (DP) offers effective privacy protection by introducing noise to parameters based on rigorous privacy definitions. However, excessive noise addition can potentially compromise the accuracy of the model. Another challenge in federated learning is the issue of high communication costs. Training large-scale federated models can be slow and expensive in terms of communication resources. To address this, various model pruning algorithms have been proposed. To address these challenges, this paper introduces a communication-efficient, privacy-preserving FL algorithm based on two-stage gradient pruning and differentiated differential privacy, named IsmDP-FL. The algorithm leverages a two-stage approach, incorporating gradient pruning and differentiated differential privacy. In the first stage, the trained model is subject to gradient pruning, followed by the addition of differential privacy to the important parameters selected after pruning. Non-important parameters are pruned by a certain ratio, and differentiated differential privacy is applied to the remaining parameters in each network layer. In the second stage, gradient pruning is performed during the upload to the server for aggregation, and the final result is returned to the client to complete the federated learning process. Extensive experiments demonstrate that the proposed method ensures a high communication efficiency, maintains the model privacy, and reduces the unnecessary use of the privacy budget.

FedFTN: Personalized federated learning with deep feature transformation network for multi-institutional low-count PET denoising

Low-count PET is an efficient way to reduce radiation exposure and acquisition time, but the reconstructed images often suffer from low signal-to-noise ratio (SNR), thus affecting diagnosis and other downstream tasks. Recent advances in deep learning have shown great potential in improving low-count PET image quality, but acquiring a large, centralized, and diverse dataset from multiple institutions for training a robust model is difficult due to privacy and security concerns of patient data. Moreover, low-count PET data at different institutions may have different data distribution, thus requiring personalized models. While previous federated learning (FL) algorithms enable multi-institution collaborative training without the need of aggregating local data, addressing the large domain shift in the application of multi-institutional low-count PET denoising remains a challenge and is still highly under-explored. In this work, we propose FedFTN, a personalized federated learning strategy that addresses these challenges. FedFTN uses a local deep feature transformation network (FTN) to modulate the feature outputs of a globally shared denoising network, enabling personalized low-count PET denoising for each institution. During the federated learning process, only the denoising network’s weights are communicated and aggregated, while the FTN remains at the local institutions for feature transformation. We evaluated our method using a large-scale dataset of multi-institutional low-count PET imaging data from three medical centers located across three continents, and showed that FedFTN provides high-quality low-count PET images, outperforming previous baseline FL reconstruction methods across all low-count levels at all three institutions.

PPeFL: Privacy-Preserving Edge Federated Learning With Local Differential Privacy

Since traditional federated learning algorithms cannot provide sufficient privacy guarantees, an increasing number of approaches apply local differential privacy (LDP) techniques to federated learning to provide strict privacy guarantees. However, the privacy budget heavily increases proportionally with the dimension of the parameters, and the large variance generated by the perturbation mechanisms leads to poor performance of the final model. In this paper, we propose a novel privacy-preserving edge federated learning framework based on LDP (PPeFL). Specifically, we present three LDP mechanisms to address the privacy problems in the federated learning process. The proposed filtering and screening with exponential mechanism (FS-EM) filters out the better parameters for global aggregation based on the contribution of weight parameters to the neural network. Thus, we can not only solve the problem of fast growth of privacy budget when applying perturbation mechanism locally, but also greatly reduce the communication costs. In addition, the proposed data perturbation mechanism with stronger privacy (DPM-SP) allows a secondary scrambling of the original data of participants and can provide strong security. Further, a data perturbation mechanism with enhanced-utility (DPM-EU) is proposed in order to reduce the variance introduced by the perturbation. Finally, extensive experiments are performed to illustrate that the PPeFL scheme is practical and efficient, providing stronger privacy protection while ensuring utility.

Blockchain-based fairness-enhanced federated learning scheme against label flipping attack

The federated learning technology provides a new method for data integration, which realize the sharing of a global model and prevent the leakage of the user’s original data. In order to resist label flipping attack, to ensure the reliability and accuracy of the global model, and to guarantee the fairness of federated learning, we propose a blockchain-based fairness enhanced federated learning scheme. The accuracy of global model and the fairness of the aggregation process are guaranteed by an adaptive aggregation algorithm. The reliability of federated learning process is ensured by recording the entire process of the model training on the blockchain and by using digital signature. The privacy of each participant of federated learning is protected by public key encryption combined with the use of random numbers. Theoretical analysis and experiments show that the scheme can protect the privacy of each participant, mitigate label flipping attack and ensure the reliability and fairness of the entire federated learning process.

Anonymous federated learning framework in the internet of things

AbstractWith the continuous development of the internet of things (IoT), federated learning is being widely applied. This technology keeps data locally to protect data security. However, during the process of uploading gradients from local clients, there is a possibility of leaking sensitive information such as identities. To address this issue, this paper presents a secure and efficient anonymous federated learning framework. In our proposed, we first design a lightweight key‐sharing protocol based on elliptic curve cryptography (ECC) to generate shared keys and ensure secure communication. We then describe the improved process of federated learning, where clients communicate with the cloud server using pseudonyms to achieve anonymity. The security of our protocol is analyzed from both formal and informal perspectives, demonstrating that our proposed protocol satisfies “session key security.” We also employ the formal verification tool ProVerif to validate the security of the protocol in terms of mutual authentication and key configuration in the Dolve–Yao threat model. Finally, the computational and communication costs of the proposed anonymous federated learning framework are evaluated, showing that both the computational and communication expenses are relatively low. A comparison was made between the proposed federated learning process and three other federated learning processes, demonstrating the clear advantages of our proposed.

Multi-player evolutionary game of federated learning incentive mechanism based on system dynamics

Federated learning has emerged as a new way of data sharing. The participants in the federation tend to choose different strategies based on their benefits, which is formalized into an evolutionary game model. Existing techniques can limit the malicious behavior of participants by detecting betrayers or weakening their influence. The problem that whether there is an incentive mechanism which makes participants spontaneously choose to cooperate honestly and maintains the stability of the federated learning system is urgent. In this paper, we develop a multi-player evolutionary game model in federated learning. We model the federated learning process by evaluating the payoffs of the central server, internal clients, and external clients. The stability of the federated learning system in the long-term dynamics process is assessed by seeking the evolutionarily stable equilibrium solutions. In this paper, mathematical reasoning and computer simulation are combined to analyze the impact of reward and punishment strategies in incentive mechanisms on the game process and game equilibrium. An incentive mechanism is designed to achieve evolutionarily stable equilibrium while make most clients join the federation spontaneously and cooperate honestly. Finally, the effectiveness, stability, and generalizability of this incentive mechanism are verified by sensitivity analysis and Lyapunov stability theory.