Federated Learning Model Research Articles

Resource Consumption for Supporting Federated Learning in Wireless Networks

Federated learning (FL) has recently become one of the hottest focuses in wireless edge networks with the ever-increasing computing capability of user equipment (UE). In FL, UEs train local machine learning models and transmit them to an aggregator, where a global model is formed and then sent back to UEs. In wireless networks, local training and model transmission can be unsuccessful due to constrained computing resources, wireless channel impairments, bandwidth limitations, etc., which degrades FL performance in model accuracy and/or training time. Moreover, we need to quantify the benefits and cost of deploying edge intelligence, as model training and transmission consume certain amount of resources. Therefore, it is imperative to deeply understand the relationship between FL performance and multiple-dimensional resources. In this paper, we construct an analytical model to investigate the relationship between the FL model accuracy and consumed resources in FL empowered wireless edge networks. Based on the analytical model, we explicitly quantify the model accuracy, available computing resources and communication resources. Numerical results validate the effectiveness of our theoretical modeling and analysis, and demonstrate the trade-off between the communication and computing resources for achieving a certain model accuracy.

Toward Secure Federated Learning for IoT Using DRL-Enabled Reputation Mechanism

Federated learning (FL) has emerged to leverage datasets from multiple devices to improve the performance of a machine learning (ML) model while providing privacy preservation for devices. The training data is collected at the devices, also known as FL workers, which collaboratively train a global learning model and share their local model updates with a central entity or server without sharing their data. However, FL can be susceptible to various adversarial attacks that target its security and privacy. In particular, the workers can upload unreliable local model updates, leading to corruption of the main FL task. Workers may intentionally contribute unreliable local updates by launching poisoning attacks or unintentionally by updating low-quality models caused by high device mobility, limited device resources, or unstable network connection. Consequently, identifying reliable and trustworthy workers becomes critical for FL security. In this article, the concept of reputation is adopted as a metric to evaluate workers’ reliability and trustworthiness. In addition, deep reinforcement learning (DRL)-based reputation mechanism is proposed for optimal selection and evaluation of reliable FL workers. Due to the dynamic nature of worker behavior in the FL environment, the DRL-based algorithm deep deterministic policy gradient (DDPG) is employed to improve the FL model accuracy and stability. We compare the performance of our proposed method with a conventional reputation method and deep <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$Q$ </tex-math></inline-formula> -networks (DQNs)-based reputation method. Our simulation results demonstrate that our proposed method can improve FL accuracy by more than 30% under various scenarios and achieves better convergence than the other methods.

Applications and Challenges of Federated Learning Paradigm in the Big Data Era with Special Emphasis on COVID-19

Federated learning (FL) is one of the leading paradigms of modern times with higher privacy guarantees than any other digital solution. Since its inception in 2016, FL has been rigorously investigated from multiple perspectives. Some of these perspectives are extensions of FL’s applications in different sectors, communication overheads, statistical heterogeneity problems, client dropout issues, the legitimacy of FL system results, privacy preservation, etc. Recently, FL is being increasingly used in the medical domain for multiple purposes, and many successful applications exist that are serving mankind in various ways. In this work, we describe the novel applications and challenges of the FL paradigm with special emphasis on the COVID-19 pandemic. We describe the synergies of FL with other emerging technologies to accomplish multiple services to fight the COVID-19 pandemic. We analyze the recent open-source development of FL which can help in designing scalable and reliable FL models. Lastly, we suggest valuable recommendations to enhance the technical persuasiveness of the FL paradigm. To the best of the authors’ knowledge, this is the first work that highlights the efficacy of FL in the era of COVID-19. The analysis enclosed in this article can pave the way for understanding the technical efficacy of FL in medical field, specifically COVID-19.

Coalition based utility and efficiency optimization for multi-task federated learning in Internet of Vehicles

With the emergence of the sixth generation (6G) communication technologies, massive infrastructures will be densely deployed and the number of data will be generated exponentially. Massive data collected by vehicles can be used to train a machine learning model, which is an effective way of implementing intelligent services in the Internet of Vehicles (IoV). However, centralized training leads to communication congestion and user privacy leakage. To address these problems and improve efficiency, federated learning has been proposed. Most current studies on federated learning consider all users performing only a single task at a certain time, which results in low network utility and long task execution time. Because some vehicles may have similar data, we propose a multi-task federated learning model where vehicles simultaneously execute tasks. To jointly maximize the network utility and efficiency of the multi-task federated learning in the 6G-enabled IoV, we design a task-driven vehicular coalition algorithm considering vehicle selection and wireless resource allocation. Moreover, we derive a convex objective function from network utility function and loss function with constraints through a series of mathematical analyses. Finally, we verify that our proposed algorithm with low complexity can improve the utility and efficiency of the multi-task federated learning in the 6G-enabled IoV.

The federated tumor segmentation (FeTS) tool: an open-source solution to further solid tumor research

Objective. De-centralized data analysis becomes an increasingly preferred option in the healthcare domain, as it alleviates the need for sharing primary patient data across collaborating institutions. This highlights the need for consistent harmonized data curation, pre-processing, and identification of regions of interest based on uniform criteria. Approach. Towards this end, this manuscript describes the Federated Tumor Segmentation (FeTS) tool, in terms of software architecture and functionality. Main results. The primary aim of the FeTS tool is to facilitate this harmonized processing and the generation of gold standard reference labels for tumor sub-compartments on brain magnetic resonance imaging, and further enable federated training of a tumor sub-compartment delineation model across numerous sites distributed across the globe, without the need to share patient data. Significance. Building upon existing open-source tools such as the Insight Toolkit and Qt, the FeTS tool is designed to enable training deep learning models targeting tumor delineation in either centralized or federated settings. The target audience of the FeTS tool is primarily the computational researcher interested in developing federated learning models, and interested in joining a global federation towards this effort. The tool is open sourced at https://github.com/FETS-AI/Front-End.

Graph-regularized federated learning with shareable side information

This study focuses on specifying local models in federated learning (FL), which allows a large number of clients to improve their corresponding models by training a shared global model. However, current FL models often fail to consider the difference between the data distributions in various clients while enforcing all local models to be identical, thus leading to a considerable loss of local personalization. To this end, this study proposes a graph-regularized federated learning framework, GraphFL, by exploiting the available client features commonly shared with other clients in the real world. Specifically, GraphFL achieves the similarity matrix of all clients using the permitted shareable side information and subsequently updates local models by returning a specific model from the server instead of an identical model. The proposed model iteratively learns the neural network parameters for each client. Compared with state-of-the-art FL models, GraphFL can benefit from the employed similarity and achieve improved classification performance in clients on three publicly available image datasets.

An Improved Federated Learning Algorithm for Privacy Preserving in Cybertwin-Driven 6G System

With the expected explosive use of the Internet of Everything in sixth generation (6G), the cybertwin network is able to convert user information to digital assets and provide extensive services. However, protecting and enhancing privacy of the processed and transmitted data in cybertwin-driven 6G is still in its infancy. Federated learning (FL) is a nascent distributed machine learning paradigm that is able to facilitate privacy protection in cybertwin networks. In a cybertwin network, imbalanced data distribution of the clients can increase the bias of the global model and sacrifice the performance of the FL model. Prior research work dealing with imbalanced data requires extra data information exchanged between clients and the server, which increases the risk of privacy leakage. To avoid privacy leakage, we design an estimation algorithm to determine the distribution of local data collected at the clients without the awareness of specific raw data. We consider two scenarios in FL: 1) the server could receive the individual trained model for each selected device and 2) the server could receive the aggregated model from the selected clients. We formulate two device selection problems to improve the training performance of the aforementioned scenarios. We develop two online learning algorithms to tackle the selection problems for both individual model uploading and aggregated model uploading. The proposed algorithms are conducted on the server, thereby avoiding privacy leakage and extra computation at the clients. We validate the effectiveness of the proposed client selection algorithms with sufficient experiments in cybertwin-driven 6G networks.

Accelerating Federated Learning via Parallel Servers: A Theoretically Guaranteed Approach

With the growth of participating clients, the centralized parameter server (PS) will seriously limit the scale and efficiency of Federated Learning (FL). A straightforward approach to scale up the FL system is to construct a Parallel FL (PFL) system with multiple parallel PSes. However, it is unclear whether PFL can really accelerate FL or reduce the training time of FL. Even if the answer is yes, it is non-trivial to design a highly efficient parameter average algorithm for a PFL system. In this paper, we propose a completely parallelizable FL algorithm called P-FedAvg under the PFL architecture. P-FedAvg extends the well-known FedAvg algorithm by allowing multiple PSes to cooperate and train a learning model together. In P-FedAvg, each PS is only responsible for a fraction of total clients, but PSes can mix model parameters in a dedicatedly designed way so that the FL model can well converge. Different from heuristic-based algorithms, P-FedAvg is with theoretical guarantees. To be rigorous, we theoretically analyze the convergence rate of P-FedAvg in terms of the number of conducted iterations, the communication cost of each global iteration and the optimal weights for each PS to mix parameters with its neighbors. Based on theoretical analysis, we conduct a case study on five typical overlay topolgoies formed by PSes to further examine the communication efficiency under different topologies, and investigate how the overlay topology affects the convergence rate, communication cost and robustness of a PFL system. Lastly, we perform extensive experiments with real datasets to verify our analysis and demonstrate that P-FedAvg can significantly speed up FL than traditional FedAvg and other competitive baselines. We believe that our work can help to lay a theoretical foundation for building more efficient PFL systems.

Urban Integrated Energy Edge Collaboration and Privacy Protection Based on the Federated Learning Framework

In the context of building a low-carbon integrated energy city, the energy data of electricity, water, heat, gas and other in city belong to different public utility management departments, and there are information barriers and data privacy protection issues between different energy systems. Under the premise of ensuring data security, aiming at the problem of training algorithm models for different energy data, the energy edge collaboration architecture based on homomorphic encrypted federated learning is proposed firstly, and then the urban comprehensive energy federated learning model is established to realize the local training model and transmit the updated model parameters to the server. Finally, the prediction of multivariate short-term loads by federated learning framework training and centralized training neural network models is verified by comparative experiments, and the results prove the feasibility and superiority of urban integrated energy edge collaboration and privacy protection based on federated learning framework in the field of urban integrated energy.

Privacy-preserving federated learning for residential short-term load forecasting

With high levels of intermittent power generation and dynamic demand patterns, accurate forecasts for residential loads have become essential. Smart meters can play an important role when making these forecasts as they provide detailed load data. However, using smart meter data for load forecasting is challenging due to data privacy requirements. This paper investigates how these requirements can be addressed through a combination of federated learning and privacy preserving techniques such as differential privacy and secure aggregation. For our analysis, we employ a large set of residential load data and simulate how different federated learning models and privacy preserving techniques affect performance and privacy. Our simulations reveal that combining federated learning and privacy preserving techniques can secure both high forecasting accuracy and near-complete privacy. Specifically, we find that such combinations enable a high level of information sharing while ensuring privacy of both the processed load data and forecasting models. Moreover, we identify and discuss challenges of applying federated learning, differential privacy and secure aggregation for residential short-term load forecasting.

A Hybrid Incentive Mechanism for Decentralized Federated Learning

Federated Learning (FL) presents a privacy-compliant approach by sharing model parameters instead of raw data. However, how to motivate data owners to participate in and stay within an FL ecosystem by continuously contributing their data to the FL model remains a challenge. In this article, we propose a hybrid incentive mechanism based on blockchain to address the above challenge. The proposed mechanism comprises two primary smart contract-based modules, namely the reputation module and the reverse auction module. The former is used to dynamically calculate the reputation score of each FL participant. It employs a trust-jointed reputation scheme to balance the weights between trust values of parameters and bid prices. The latter is responsible for initiating FL auction tasks, calculating price rankings, and assigning corresponding token rewards. Experiments are conducted to evaluate the feasibility and performance of the proposed mechanism against the three typical threats. Experimental results indicate that our mechanism can successfully reduce incentive costs while preventing participants from colluding and over-bidding in the data sharing auction.

Secure Smart Communication Efficiency in Federated Learning: Achievements and Challenges

Federated learning (FL) is known to perform machine learning tasks in a distributed manner. Over the years, this has become an emerging technology, especially with various data protection and privacy policies being imposed. FL allows for performing machine learning tasks while adhering to these challenges. As with the emergence of any new technology, there will be challenges and benefits. A challenge that exists in FL is the communication costs: as FL takes place in a distributed environment where devices connected over the network have to constantly share their updates, this can create a communication bottleneck. This paper presents the state-of-the-art of the conducted works on communication constraints of FL while maintaining the secure and smart properties that federated learning is known for. Overall, current challenges and possible methods for enhancing the FL models’ efficiency with a perspective on communication are discussed. This paper aims to bridge the gap in all conducted review papers by solely focusing on communication aspects in FL environments.

High-accuracy low-cost privacy-preserving federated learning in IoT systems via adaptive perturbation

With the rapid development of the Internet of Things (IoT), federated learning (FL) has been widely used to obtain insights from collected data while preserving data privacy. Differential privacy (DP) is an additive noise scheme that has been widely studied as a privacy-preserving approach on FL. However, privacy protection under DP usually comes at the cost of model accuracy for the underlying FL process. In this paper, we propose a novel low-cost (for both communication and computational overhead) adaptive noise perturbation/masking scheme to protect FL clients’ privacy without degrading the global model accuracy. In particular, we set the magnitude of the additive noise to adaptively change with the magnitude of the local model updates. Then, a direction-based filtering scheme is used to accelerate the convergence of the FL model. A maximum tolerable noise bound for local clients is derived using the central limit theorem (CLT). The designed noise maximizes privacy protection for clients while preserving the accuracy and convergence rate of the FL model , as a result of the noise cancelling out and forming a more concentrated distribution after the aggregation operation on the server. We theoretically prove that FL with the proposed noise perturbation scheme retains the same accuracy and convergence rate (O(1/T) for convex loss functions and O(1/T) for non-convex loss functions) as that of non-private FL with SGD. We also evaluate the performance of the proposed scheme in terms of convergence behavior, computational efficiency, and privacy protection against state-of-the-art privacy inference attacks on real-world datasets. Experimental results show that FL with our proposed perturbation scheme outperforms DP in the accuracy and convergence rate of the FL model in both client dropout and non-client dropout scenarios. Compared with DP, our proposed scheme does not incur additional computational and communication overhead. Our approach provides DP-comparable or better effectiveness in defending against privacy attacks under the same global model accuracy.

High performance of privacy-preserving acute myocardial infarction auxiliary diagnosis based on federated learning: a multicenter retrospective study.

BackgroundMulticenter medical research is becoming a new trend. However, interagency data privacy security protection has become a major bottleneck of multicenter research. Therefore, to overcome this privacy protection issue, the aim of the present study was to apply a self-developed privacy-preserving machine learning framework for researchers who can build models on medical data from multiple sources, while providing privacy protection for both sensitive data and the learned model.MethodsBased on Arya, a novel privacy computing platform developed by Healink, we constructed a privacy-preserving federated learning (FL) model using the fully connected neural network with datasets from 2–3 individual medical institutions. In the dataset, 80% of records were used for joint modeling on acute myocardial infarction (AMI) diagnosis. Modeling efficacy was evaluated with the remaining 20% of records. As the control, 1,500 medical records from 1 medical institution were used for single-center modeling and efficacy evaluation. During the process, the original data were still kept in individual hospital without moving or transferring out of the hospitial. The diagnostic efficacy (sensitivity, positive predictive value, and accuracy) was evaluated.ResultsOur privacy-preserving FL model gives reliable AMI diagnostic efficacy. Three-center modeling (79% sensitivity, 88% positive predictive value, and 82.3% accuracy) and two-center modeling (77.8% or 77.6% sensitivity, 86.7% or 85.30% positive predictive value, and 81% or 79.7% accuracy) achieved relative high diagnostic efficacy; and single-center modeling achieved relative low diagnostic efficacy (76% sensitivity, 84.7% positive predictive value, and 79% accuracy).ConclusionsThe Arya privacy computing platform is efficient and practical for the FL model, which could promote multicenter medical research securely without sacrificing diagnostic efficacy.

On Demand Fog Federations for Horizontal Federated Learning in IoV

Federated learning using fog computing can suffer from the dynamic behavior of some of the participants in its training process, especially in Internet-of-Vehicles where vehicles are the targeted participants. For instance, the fog might not be able to cope with the vehicles’ demands in some areas due to resource shortages when the vehicles gather for events, or due to traffic congestion. Moreover, the vehicles are exposed to unintentionally leaving the fog coverage area which can result in the task being dropped as the communications between the server and the vehicles weaken. The aforementioned limitations can affect the federated learning model accuracy for critical applications, such as autonomous driving, where the model inference could influence road safety. Recent works in the literature have addressed some of these problems through active sampling techniques, however, they suffer from many complications in terms of stability, scalability, and efficiency of managing the available resources. To address these limitations, we propose a horizontal-based federated learning architecture, empowered by fog federations, devised for the mobile environment. In our architecture, fog computing providers form stable fog federations using a Hedonic game-theoretical model to expand their geographical footprints. Hence, providers belonging to the same federations can migrate services upon demand in order to cope with the federated learning requirements in an adaptive fashion. We conduct the experiments using a road traffic signs dataset modeled with intermodal traffic systems. The simulation results show that the proposed model can achieve better accuracy and quality of service than other models presented in the literature.

Effectiveness of Federated Learning and CNN Ensemble Architectures for Identifying Brain Tumors Using MRI Images.

Medical institutions often revoke data access due to the privacy concern of patients. Federated Learning (FL) is a collaborative learning paradigm that can generate an unbiased global model based on collecting updates from local models trained by client’s data while keeping the local data private. This study aims to address the centralized data collection issue through the application of FL on brain tumor identification from MRI images. At first, several CNN models were trained using the MRI data and the best three performing CNN models were selected to form different variants of ensemble classifiers. Afterward, the FL model was constructed using the ensemble architecture. It was trained using model weights from the local model without sharing the client’s data (MRI images) using the FL approach. Experimental results show only a slight decline in the performance of the FL approach as it achieved 91.05% accuracy compared to the 96.68% accuracy of the base ensemble model. Additionally, same approach was taken for another slightly larger dataset to prove the scalability of the method. This study shows that the FL approach can achieve privacy-protected tumor classification from MRI images without compromising much accuracy compared to the traditional deep learning approach.

An explainable semi-personalized federated learning model

Training a model using batch learning requires uniform data storage in a repository. This approach is intrusive, as users have to expose their privacy and exchange sensitive data by sending them to central entities to be preprocessed. Unlike the aforementioned centralized approach, training of intelligent models via the federated learning (FEDL) mechanism can be carried out using decentralized data. This process ensures that privacy and protection of sensitive information can be managed by a user or an organization, employing a single universal model for all users. This model should apply average aggregation methods to the set of cooperative training data. This raises serious concerns for the effectiveness of this universal approach and, therefore, for the validity of FEDL architectures in general. Generally, it flattens the unique needs of individual users without considering the local events to be managed. This paper proposes an innovative hybrid explainable semi-personalized federated learning model, that utilizes Shapley Values and Lipschitz Constant techniques, in order to create personalized intelligent models. It is based on the needs and events that each individual user is required to address in a federated format. Explanations are the assortment of characteristics of the interpretable system, which, in the case of a specified illustration, helped to bring about a conclusion and provided the function of the model on both local and global levels. Retraining is suggested only for those features for which the degree of change is considered quite important for the evolution of its functionality.

ShareChain: Blockchain‐enabled model for sharing patient data using federated learning and differential privacy

AbstractEvery individual in our technologically evolved world needs proper data security. The procedure of exchanging medical information is increasingly concerned with data privacy. Many techniques have been offered for preserving data security. These techniques use approaches such as ‐anonymity, ‐diversity, and others. However, such solutions are vulnerable to attribute disclosure, homogeneity, and background knowledge risks due to their syntactic nature. In this work, we describe a safe and secure architecture and semantic approach for data sharing that is based on blockchain, local differential privacy (LDP), and federated learning (FL). The proposed framework generates an atmosphere devoid of trust in which data owners are no longer required to have trust in the controllers. The FL models enable the whole network to decentralize its data‐driven learning. Interplanetary file system (IPFS) is used to provide data security in a distributed environment because each file in IPFS has a digital fingerprint that is computed using a cryptographic hash function on the file's whole contents. Due to the rigorous privacy guarantee, data owners no longer need to be worried about the security of their data. The proposed model's assessment parameters include latency, throughput, privacy, and accuracy. The data privacy of the proposed model is protected via LDP and FL, and its latency and throughput communication transactions on permissioned blockchain are calculated and compared with those of the benchmark model. The findings indicate that the proposed model delivers 85% more accurate privacy than the benchmark model.

Federated Learning and Its Role in the Privacy Preservation of IoT Devices

Federated learning (FL) is a cutting-edge artificial intelligence approach. It is a decentralized problem-solving technique that allows users to train using massive data. Unprocessed information is stored in advanced technology by a secret confidentiality service, which incorporates machine learning (ML) training while removing data connections. As researchers in the field promote ML configurations containing a large amount of private data, systems and infrastructure must be developed to improve the effectiveness of advanced learning systems. This study examines FL in-depth, focusing on application and system platforms, mechanisms, real-world applications, and process contexts. FL creates robust classifiers without requiring information disclosure, resulting in highly secure privacy policies and access control privileges. The article begins with an overview of FL. Then, we examine technical data in FL, enabling innovation, contracts, and software. Compared with other review articles, our goal is to provide a more comprehensive explanation of the best procedure systems and authentic FL software to enable scientists to create the best privacy preservation solutions for IoT devices. We also provide an overview of similar scientific papers and a detailed analysis of the significant difficulties encountered in recent publications. Furthermore, we investigate the benefits and drawbacks of FL and highlight comprehensive distribution scenarios to demonstrate how specific FL models could be implemented to achieve the desired results.

Federated Neural Architecture Search for Medical Data Security

Medical data widely exist in the hospital and personal life, usually across institutions and regions. They have essential diagnostic value and therapeutic significance. The disclosure of patient information causes people’s panic, therefore, medical data security solution is very crucial for intelligent health care. The emergence of federated learning (FL) provides an effective solution, which only transmits model parameters, breaking through the bottleneck of medical data sharing, protecting data security, and avoiding economic losses. Meanwhile, the neural architecture search (NAS) has become a popular method to automatically search the optimal neural architecture for solving complex practical problems. However, few papers have combined the FL and NAS for simultaneous privacy protection and model architecture selection. Convolutional neural network (CNN) has outstanding performance in the image recognition field. Combining CNN and fuzzy rough sets can effectively improve the interpretability of deep neural networks. This article aims to develop a multiobjective convolutional interval type-2 fuzzy rough FL model based on NAS (CIT2FR-FL-NAS) for medical data security with an improved multiobjective evolutionary algorithm. We test the proposed framework on the LC25000 lung and colon histopathological image dataset. Experimental verification demonstrates that the designed multiobjective CIT2FR-FL-NAS framework can achieve high accuracy superior to state-of-the-art models and reduce network complexity under the condition of protecting medical data security.