Federated Learning Model Research Articles

Quantized Distributed Federated Learning for Industrial Internet of Things

Federated learning (FL) enables multiple devices to collaboratively train a shared machine learning (ML) model while keeping all the local data private, which is a crucial enabler to implement artificial intelligence (AI) at the edge of the Industrial Internet of Things (IIoT) scenario. Distributed FL (DFL) based on Device-to-Device (D2D) communications can solve the single point of failure and scaling issue of centralized FL, but subject to the communication resource limitation of D2D links. Thus, it is crucial to reduce the data transmission volume of FL models between devices. In this article, we propose a quantization-based DFL (Q-DFL) mechanism in a D2D network and prove its convergence. Q-DFL contains two phases: 1) in phase I, a local model is trained with the stochastic gradient descent (SGD) algorithm on each IIoT device and then exchanges the quantified model parameters between neighboring nodes and 2) in phase II, a quantitative consensus mechanism is designed to ensure the local models converge to the same global model. We also propose an adaptive stopping mechanism and a synchronization protocol to fulfill the phase transition from phase I to phase II. Simulation results reveal that with Q-DFL, a 1-bit quantizer can be employed without affecting the model convergence at the price of slight accuracy reduction, which achieves significant transmission bandwidth saving. Further simulation of Q-DFL for the MobileNet model is fulfilled with different quantization bit levels, which reveals their performance tradeoff among the system information flow consumption, the system time delay, and the system energy cost.

RingFFL: A Ring-Architecture-Based Fair Federated Learning Framework

In the ring-architecture-based federated learning framework, security and fairness are severely compromised when dishonest clients abort the training process after obtaining useful information. To solve the problem, we propose a Ring- architecture-based Fair Federated Learning framework called RingFFL, in which we design a penalty mechanism for FL. Before the training starts in each round, all clients that will participate in the training pay deposits in a set order and record the transactions on the blockchain to ensure that they are not tampered with. Subsequently, the clients perform the FL training process, and the correctness of the models transmitted by the clients is guaranteed by the HASH algorithm during the training process. When all clients perform honestly, each client can obtain the final model, and the number of digital currencies in each client’s wallet is kept constant; otherwise, the deposits of clients who leave halfway will be compensated to the clients who perform honestly during the training process. In this way, through the penalty mechanism, all clients either obtain the final model or are compensated, thus ensuring the fairness of federated learning. The security analysis and experimental results show that RingFFL not only guarantees the accuracy and security of the federated learning model but also guarantees the fairness.

Building Trusted Federated Learning: Key Technologies and Challenges

Federated learning (FL) provides convenience for cross-domain machine learning applications and has been widely studied. However, the original FL is still vulnerable to poisoning and inference attacks, which will hinder the landing application of FL. Therefore, it is essential to design a trustworthy federation learning (TFL) to eliminate users’ anxiety. In this paper, we aim to provide a well-researched picture of the security and privacy issues in FL that can bridge the gap to TFL. Firstly, we define the desired goals and critical requirements of TFL, observe the FL model from the perspective of the adversaries and extrapolate the roles and capabilities of potential adversaries backward. Subsequently, we summarize the current mainstream attack and defense means and analyze the characteristics of the different methods. Based on a priori knowledge, we propose directions for realizing the future of TFL that deserve attention.

Federated-Learning-Empowered Semi-Supervised Active Learning Framework for Intrusion Detection in ZSM

Exponential growth of novel radical applications and services in sixth-generation (6G) networks is expected to increase the complexity of managing existing network infrastructures. In this context, the zero touch network and service management (ZSM) paradigm, which leverages AI, SDN, and NFV techniques, is seen as a promising solution to automatically manage and orchestrate network resources. However, due to the closed-loop operation and automated end-to-end framework in a distributed 6G network, the ZSM architecture, along with its potential benefits, is exposed to various security threats. A recently proposed solution to address privacy concerns is federated learning (FL), whereby distributed training is performed, and the aggregated model parameters, instead of clients' raw data, are forwarded to the global server. However, most of the existing FL and semi-supervised learning (SSL) models for intrusion detection are based on the assumption that fully labeled data are always available at the server and client sides, which is not practical due to the high labeling costs and privacy constraints in the 6G network. In this article, we propose a novel FL-empowered semi-supervised active learning (FL-SSAL) security orchestration framework for the Label-at-Client scenario where, along with unlabeled samples, clients have a small portion of labeled data. The entropy-based active learning selects the most informative samples for data annotation and leverages the unlabeled data using a semi-supervised approach. The results of our experimental evaluations performed on the private, not independent and identically distributed (non-IID) dataset demonstrate that FL-SSAL achieves higher intrusion detection accuracy and has less communication overhead than baseline schemes with less labeled data.

Privacy-Preserving Federated Learning for Industrial Edge Computing via Hybrid Differential Privacy and Adaptive Compression

With the continuous improvement of hardware computing power, edge computing of industrial data has been gradually applied. In the past decade, the promotion of edge computing has also greatly improved the efficiency of industrial production. Compared with the conventional cloud computing, it not only saves the bandwidth consumption of data transmission, but also ensures the terminal data security to a certain extent. However, the continuous update of attack types also put forward new requirements for the privacy protection of industrial edge computing. So it should fundamentally solve the risk of industrial data leakage in the process of deep model training in edge terminal. In this article, we propose a new federated edge learning framework based on hybrid differential privacy and adaptive compression for industrial data processing. Specifically, it first completes the adaptive gradient compression preparation, then constructs the industrial federated learning model, and finally makes use of adaptive differential privacy model to optimize, so as to complete the privacy protection towards the transmission of gradient parameters in industrial environment. By optimizing the hybrid differential privacy and adaptive compression, we can better prevent the terminal data privacy against inference attacks. The experimental results show that this method is very effective in the industrial edge computing situation, and it also opens up a new direction for the effect of differential privacy in federated learning.

Federated learning with hyper-network—a case study on whole slide image analysis

Federated learning(FL) is a new kind of Artificial Intelligence(AI) aimed at data privacy preservation that builds on decentralizing the training data for the deep learning model. This new technique of data security and privacy sheds light on many critical domains with highly sensitive data, including medical image analysis. Developing a strong, scalable, and precise deep learning model has proven to count on a variety of high-quality data from different centers. However, data holders may not willing to share their data considering the restriction of privacy. In this paper, we approach this challenge with a federated learning paradigm. Specifically, we present a case study on the whole slide image classification problem. At each local client center, a multiple-instance learning classifier is developed to conduct whole slide image classification. We introduce a privacy-preserving federated learning framework based on hyper-network to update the global model. Hyper-network is deployed at the global center that produces the weights of the local network conditioned on its input. In this way, hyper-networks can simultaneously learn a family of the local client networks. Instead of communicating raw data with the local client, only model parameters injected with noise are transferred between the local client and the global model. By using a large scale of whole slide images with only slide-level labels, we mensurated our way on two different whole slide image classification problems. The results demonstrate that our proposed federated learning model based on hyper-network can effectively leverage multi-center data to develop a more accurate model which can be used to classify a whole slide image. Its improvements in terms of over the isolated local centers and the commonly used federated averaging baseline are significant. Code will be available.

Blockchain-Based Decentralized Federated Learning Method in Edge Computing Environment

In recent years, federated learning has been able to provide an effective solution for data privacy protection, so it has been widely used in financial, medical, and other fields. However, traditional federated learning still suffers from single-point server failure, which is a frequent issue from the centralized server for global model aggregation. Additionally, it also lacks an incentive mechanism, which leads to the insufficient contribution of local devices to global model training. In this paper, we propose a blockchain-based decentralized federated learning method, named BD-FL, to solve these problems. BD-FL combines blockchain and edge computing techniques to build a decentralized federated learning system. An incentive mechanism is introduced to motivate local devices to actively participate in federated learning model training. In order to minimize the cost of model training, BD-FL designs a preference-based stable matching algorithm to bind local devices with appropriate edge servers, which can reduce communication overhead. In addition, we propose a reputation-based practical Byzantine fault tolerance (R-PBFT) algorithm to optimize the consensus process of global model training in the blockchain. Experiment results show that BD-FL effectively reduces the model training time by up to 34.9% compared with several baseline federated learning methods. The R-PBFT algorithm can improve the training efficiency of BD-FL by 12.2%.

A Federated Learning Framework against Data Poisoning Attacks on the Basis of the Genetic Algorithm

Obtaining the balance between information loss and training accuracy is crucial in federated learning. Nevertheless, inadequate data quality will affect training accuracy. Here, to improve the training accuracy without affecting information loss, we propose a malicious data detection model using the genetic algorithm to resist model poisoning attack. Specifically, the model consists of three modules: (1) Participants conduct single point training on data and upload accuracy to the third-party server; (2) Formulate data scoring formula based on data quantity and quality; (3) Use the genetic algorithm to obtain the threshold which makes the score highest. Data with accuracy which exceeds this threshold can participate in cooperative training of federated learning. Before participating in training, participants’ data is optimized to oppose data poisoning attacks. Experiments on two datasets validated the effectiveness of the proposed model. It was also verified in the fashion-MNIST data set and cifar10 data set that the training accuracy of GAFL is 7.45% higher than that of the federated learning model in the fashion-MNIST data set and 8.18% in the cifar10 data set.

Federated Learning via Attentive Margin of Semantic Feature Representations

Federated learning (FL) in Internet of Things (IoT) systems enables distributed model training using a large corpus of decentralized training data dispersed among multiple IoT clients. In this distributed setting, system and statistical heterogeneity, in the form of highly imbalanced, and nonindependent and identically distributed (non-i.i.d.) data stored on multiple devices, are likely to hinder model training. Existing methods aggregate models disregarding the internal representations being learned, which yet play an essential role to solve the pursued task, especially in the case of deep learning modules. To leverage feature representations in an FL framework, we introduce a method, called FedMargin, which computes client deviations using margins over feature representations learned on distributed data, and applies them to drive federated optimization via an attention mechanism. Local and aggregated margins are jointly exploited, taking into account local representation shift and representation discrepancy with the global model. In addition, we propose three methods to analyse statistical properties of feature representations learned in FL, in order to elucidate the relationship between accuracy, margins, and feature discrepancy of FL models. In experimental analyses, FedMargin demonstrates state-of-the-art accuracy and convergence rate across image classification and semantic segmentation benchmarks by enabling maximum margin training of FL models. Moreover, FedMargin reduces the uncertainty of predictions of FL models compared to the baseline. In this work, we also evaluate FL models on dense prediction tasks, such as semantic segmentation, proving the versatility of the proposed approach.

FEAT: A Federated Approach for Privacy-Preserving Network Traffic Classification in Heterogeneous Environments

Network traffic classification is the foundation for many network security and network management applications. Recently, to preserve the privacy of the data which are generated in the mobile ends, federated learning (FL)-based classification methods are being proposed. Unfortunately, the performance of FL-based methods can seriously degrade when the client data have skewness. This is particularly true for mobile network traffic classification where the environments in the mobile ends are highly heterogeneous. In this article, we first conduct a measurement study on traffic classification accuracy through FL using real-world network traffic trace and we observe serious accuracy degradation due to heterogeneous environments. We propose a novel federated analytics (FA) approach, FEAT, to improve the accuracy. Note that FL emphasizes on model training, yet our FA performs local analytic tasks that can estimate traffic data skewness and select appropriate clients for FL model training. Our analytics tasks are performed locally and in a federated manner; thus, we preserve privacy as well. Our approach has strong theoretical properties where we exploit Hoeffding inequality to infer traffic data skewness and we leverage the Thompson Sampling for client selection. We evaluate our approach through extensive experiments using real-world traffic data sets QUIC and ISCX. The extensive experiments demonstrate that FEAT can improve traffic classification accuracy in heterogeneous environments.

FedSlice: Protecting Federated Learning Models From Malicious Participants with Model Slicing

FedSlice: Protecting Federated Learning Models From Malicious Participants with Model Slicing

FedSlice: Protecting Federated Learning Models From Malicious Participants with Model Slicing

FedSlice: Protecting Federated Learning Models From Malicious Participants with Model Slicing

FedSlice: Protecting Federated Learning Models From Malicious Participants with Model Slicing

FedSlice: Protecting Federated Learning Models From Malicious Participants with Model Slicing

Personalized Federated Learning via Domain Adaptation with an Application to Distributed 3D Printing

Over the years, Internet of Things (IoT) devices have become more powerful. This sets forth a unique opportunity to exploit local computing resources to distribute model learning and circumvent the need to share raw data. The underlying distributed and privacy-preserving data analytics approach is often termed federated learning (FL). A key challenge in FL is the heterogeneity across local datasets. In this article, we propose a new personalized FL model, PFL-DA, by adopting the philosophy of domain adaptation. PFL-DA tackles two sources of data heterogeneity at the same time: a covariate and concept shift across local devices. We show, both theoretically and empirically, that PFL-DA overcomes intrinsic shortcomings in state of the art FL approaches and is able to borrow strength across devices while allowing them to retain their own personalized model. As a case study, we apply PFL-DA to distributed desktop 3D printing where we obtain more accurate predictions of printing speed, which can help improve the efficiency of the printers.

Fair detection of poisoning attacks in federated learning on non-i.i.d. data.

Reconciling machine learning with individual privacy is one of the main motivations behind federated learning (FL), a decentralized machine learning technique that aggregates partial models trained by clients on their own private data to obtain a global deep learning model. Even if FL provides stronger privacy guarantees to the participating clients than centralized learning collecting the clients' data in a central server, FL is vulnerable to some attacks whereby malicious clients submit bad updates in order to prevent the model from converging or, more subtly, to introduce artificial bias in the classification (poisoning). Poisoning detection techniques compute statistics on the updates to identify malicious clients. A downside of anti-poisoning techniques is that they might lead to discriminate minority groups whose data are significantly and legitimately different from those of the majority of clients. This would not only be unfair, but would yield poorer models that would fail to capture the knowledge in the training data, especially when data are not independent and identically distributed (non-i.i.d.). In this work, we strive to strike a balance between fighting poisoning and accommodating diversity to help learning fairer and less discriminatory federated learning models. In this way, we forestall the exclusion of diverse clients while still ensuring detection of poisoning attacks. Empirical work on three data sets shows that employing our approach to tell legitimate from malicious updates produces models that are more accurate than those obtained with state-of-the-art poisoning detection techniques. Additionally, we explore the impact of our proposal on the performance of models on non-i.i.d local training data.

Preventing Image Data Poisoning Attacks in Federated Machine Learning by an Encrypted Verification Key

Recent studies have uncovered security issues with most of the federated learning models. One common false assumption in the federated learning model is that participants are the attacker and would not use polluted data. This vulnerability enables attackers to train their models using polluted data and then send the polluted updates to the training server for aggregation, potentially poisoning the overall model. In such a setting, it is challenging for an edge server to thoroughly inspect the data used for model training and supervise any edge device. This study evaluates the vulnerabilities present in federated learning and explores various types of attacks that can occur. This paper presents a robust prevention scheme to address these vulnerabilities. The proposed prevention scheme enables federated learning servers to monitor participants actively in real time and identify infected individuals by introducing an encrypted verification scheme. The paper outlines the protocol design of this prevention scheme and presents experimental results that demonstrate its effectiveness.

Next Generation Infectious Diseases Monitoring Gages via Incremental Federated Learning: Current Trends and Future Possibilities.

Infectious diseases are always alarming for the survival of human life and are a key concern in the public health domain. Therefore, early diagnosis of these infectious diseases is a high demand for modern-era healthcare systems. Novel general infectious diseases such as coronavirus are infectious diseases that cause millions of human deaths across the globe in 2020. Therefore, early, robust recognition of general infectious diseases is the desirable requirement of modern intelligent healthcare systems. This systematic study is designed under Kitchenham guidelines and sets different RQs (research questions) for robust recognition of general infectious diseases. From 2018 to 2021, four electronic databases, IEEE, ACM, Springer, and ScienceDirect, are used for the extraction of research work. These extracted studies delivered different schemes for the accurate recognition of general infectious diseases through different machine learning techniques with the inclusion of deep learning and federated learning models. A framework is also introduced to share the process of detection of infectious diseases by using machine learning models. After the filtration process, 21 studies are extracted and mapped to defined RQs. In the future, early diagnosis of infectious diseases will be possible through wearable health monitoring cages. Moreover, these gages will help to reduce the time and death rate by detection of severe diseases at starting stage.

A Systematic Survey for Differential Privacy Techniques in Federated Learning

Federated learning is a distributed machine learning technique that trains a global model by exchanging model parameters or intermediate results among multiple data sources. Although federated learning achieves physical isolation of data, the local data of federated learning clients are still at risk of leakage under the attack of malicious individuals. For this reason, combining data protection techniques (e.g., differential privacy techniques) with federated learning is a sure way to further improve the data security of federated learning models. In this survey, we review recent advances in the research of differentially-private federated learning models. First, we introduce the workflow of federated learning and the theoretical basis of differential privacy. Then, we review three differentially-private federated learning paradigms: central differential privacy, local differential privacy, and distributed differential privacy. After this, we review the algorithmic optimization and communication cost optimization of federated learning models with differential privacy. Finally, we review the applications of federated learning models with differential privacy in various domains. By systematically summarizing the existing research, we propose future research opportunities.

Explainable Federated Learning: A Lifecycle Dashboard for Industrial Settings

As the adoption of federated learning (FL) in the manufacturing industry grows and systems get increasingly complex, a need to inspect their behavior arises. Stakeholders of the FL process want a more transparent system to understand the current state and analyze how its performance changed over time. However, current representation approaches are often not designed for industrial applications and do not cover the entire FL model lifecycle. We propose the lifecycle dashboard, which considers the different requirements and perspectives of industrial stakeholders by visualizing information from the FL server. In addition, our representation approach is generic enough to be applied to different use cases and industries. We evaluate the lifecycle dashboard in a semistructured expert interview, show improvements in the understandability of FL systems, and discuss possible use cases in the industry.

Distributed Anomaly Detection in Smart Grids: A Federated Learning-Based Approach

The smart grid integrates Information and Communication Technologies (ICT) into the traditional power grid to manage the generation, distribution, and consumption of electrical energy. Despite its many advantages, it faces significant challenges, such as detecting abnormal behaviours in the grid. Identifying anomalous behaviours helps to discover unusual user power consumption, faulty infrastructure, power outages, equipment failures, energy thefts, or cyberattacks. Machine learning (ML)-based techniques on smart meter data has shown remarkable results in anomaly detection. However, traditional ML-based anomaly detection requires smart meters to share local data with a central server, which raises concerns regarding data security and user privacy. Server-based model training faces additional challenges, such as the requirement of centralised computing power, reliable network communication, large bandwidth capacity, and latency issues, all of which affect the real-time anomaly detection performance. Motivated by these concerns, we propose a Federated Learning (FL)-based smart grid anomaly detection scheme where ML models are trained locally in smart meters without sharing data with a central server, thus ensuring user privacy. In the proposed approach, a global model is downloaded from the server to smart meters for on-device training. After local training, local model parameters are sent to the server to improve the global model. We secure the model parameter updates from adversaries using the SSL/TLS protocol. Using standard datasets, we investigate the anomaly detection performance of federated learning and observe that FL models achieve anomaly detection performance comparable to centralised ML models while ensuring user privacy. Further, our study shows that the proposed FL-based models perform efficiently in terms of memory, CPU usage, bandwidth and power consumption at edge devices and are suitable for implementation in resource-constrained environments, such as smart meters, for anomaly detection.