Federated Learning Scheme Research Articles

LSFL: A Lightweight and Secure Federated Learning Scheme for Edge Computing

Nowadays, many edge computing service providers expect to leverage the computational power and data of edge nodes to improve their models without transmitting data. Federated learning facilitates collaborative training of global models among distributed edge nodes without sharing their training data. Unfortunately, existing privacy-preserving federated learning applied to this scenario still faces three challenges: 1) It typically employs complex cryptographic algorithms, which results in excessive training overhead; 2) It cannot guarantee Byzantine robustness while preserving data privacy; and 3) Edge nodes have limited computing power and may drop out frequently. As a result, the privacy-preserving federated learning cannot be effectively applied to edge computing scenarios. Therefore, we propose a lightweight and secure federated learning scheme LSFL, which combines the features of privacy-preserving and Byzantine-Robustness. Specifically, we design the Lightweight Two-Server Secure Aggregation protocol, which utilizes two servers to enable secure Byzantine robustness and model aggregation. This scheme protects data privacy and prevents Byzantine nodes from influencing model aggregation. We implement and evaluate LSFL in a LAN environment, and the experiment results show that LSFL meets fidelity, security, and efficiency design goals, and maintains model accuracy compared to the popular FedAvg scheme.

A privacy preserving federated learning scheme using homomorphic encryption and secret sharing

A privacy preserving federated learning scheme using homomorphic encryption and secret sharing

Differentially Private Byzantine-Robust Federated Learning

Federated learning is a collaborative machine learning framework where a global model is trained by different organizations under the privacy restrictions. Promising as it is, privacy and robustness issues emerge when an adversary attempts to infer the private information from the exchanged parameters or compromise the global model. Various protocols have been proposed to counter the security risks, however, it becomes challenging when one wants to make federated learning protocols robust against Byzantine adversaries while preserving the privacy of the individual participant. In this article, we propose a differentially private Byzantine-robust federated learning scheme (DPBFL) with high computation and communication efficiency. The proposed scheme is effective in preventing adversarial attacks launched by the Byzantine participants and achieves differential privacy through a novel aggregation protocol in the shuffle model. The theoretical analysis indicates that the proposed scheme converges to the approximate optimal solution with the learning error dependent on the differential privacy budget and the number of Byzantine participants. Experimental results on MNIST, FashionMNIST and CIFAR10 demonstrate that the proposed scheme is effective and efficient.

G-VCFL: Grouped Verifiable Chained Privacy-Preserving Federated Learning

Federated learning, as a typical distributed learning paradigm, shows great potential in Industrial Internet of Things, Smart Home, Smart City, etc. It enables collaborative learning without data leaving local users. Despite the huge benefits, it still faces the risk of privacy breaches and a single point of failure for aggregation server. Adversaries can use intermediate models to infer user privacy, or even return incorrect global model by manipulating the aggregation server. To address these issues, several federated learning solutions focusing on privacy-preserving and security have been proposed. However, theses solutions still faces challenges in resource-limited scenarios. In this paper, we propose G-VCFL, a grouped verifiable chained privacy-preserving federated learning scheme. Specifically, we first use the grouped chain learning mechanism to guarantee the privacy of users, and then propose a verifiable secure aggregation protocol to guarantee the verifiability of the global model. G-VCFL does not require any complex cryptographic primitives and does not introduce noise, but enables verifiable privacy-preserving federated learning by utilizing lightweight pseudorandom generators. We conduct extensive experiments on real-world datasets by comparing G-VCFL with other state-of-the-art approaches. The experimental results and functional evaluation indicate that G-VCFL is efficient in the six experimental cases and satisfies all the intended design goals.

Misbehavior Detection in Vehicular Ad Hoc Networks Based on Privacy-Preserving Federated Learning and Blockchain

As an irreversible trend, connected vehicles have become increasingly more popular. They depend on the generation and sharing of data between vehicles to improve safety and efficiency of the transportation system. Due to the open feature of the vehicular ad hoc network (VANET), it is possible for dishonest and misbehaving vehicles to disrupt traffic by transmitting false information. In recent years, misbehavior detection systems have been developed to detect the malicious behaviour, and machine learning methods have been employed to make the detection more accurately. However, existing misbehavior detection systems typically require a single entity (e.g., a central server) for centralized data collection and training. Model updates are restricted due to data privacy and high overhead of data communication, which reduces the defensive capability of misbehavior detection systems. In this paper, we propose a blockchain-based federated learning scheme to detect misbehavior, which is trained collaboratively by coordinating multiple distributed edge devices while ensuring data security and privacy. In addition, to further protect the privacy of the model on the blockchain, differential privacy with the Gaussian mechanism is leveraged to provide strict privacy protection. Common data falsification attacks are studied in this paper. The experimental results show that our proposed scheme is feasible and effective, and demonstrate that our scheme achieves satisfied accuracy and efficiency.

2D Federated Learning for Personalized Human Activity Recognition in Cyber-Physical-Social Systems

The proliferation of the Internet of Things (IoT), wearable computing, and social media technologies bring forward the realization of the so-called Cyber-Physical-Social Systems (CPSS), which is capable of offering intelligent services in many different aspects of our day-to-day life. While CPSS offer a wide variety of data from wearable devices, challenges such as data silos and secure data sharing still remain. In this study, a 2-Dimensional Federated Learning (2DFL) framework, including the vertical and horizontal federated learning phases, is designed to cope with the insufficient training data and insecure data sharing issues in CPSS during a secure distributed learning process. Considering a specific application of Human Activity Recognition (HAR) across a variety of different devices from multiple individual users, the vertical federated learning scheme is developed to integrate shareable features from heterogeneous data across different devices into a full feature space, while the horizontal federated learning scheme is developed to effectively aggregate the encrypted local models among multiple individual users to achieve a high-quality global HAR model. A computationally efficient somewhat homomorphic encryption (SWHE) scheme is then improved and applied to support the parameter aggregation without giving access to it, which enables heterogeneous data sharing with privacy protection across different personal devices and multiple users in building a more precise personalized HAR model. Experiments are conducted based on two public datasets. Comparing with three conventional machine learning methods, evaluation results demonstrate the usefulness and effectiveness of our proposed model in achieving faster and smoother convergence, with better precision, recall, and F1 scores for HAR applications in CPSS.

Detection of False Data Injection Attacks in Smart Grid: A Secure Federated Deep Learning Approach

As an important cyber-physical system (CPS), smart grid is highly vulnerable to cyber attacks. Amongst various types of attacks, false data injection attack (FDIA) proves to be one of the top-priority cyber-related issues and has received increasing attention in recent years. However, so far little attention has been paid to privacy preservation issues in the detection of FDIAs in smart grids. Inspired by federated learning, a FDIA detection method based on secure federated deep learning is proposed in this paper by combining Transformer, federated learning and Paillier cryptosystem. The Transformer, as a detector deployed in edge nodes, delves deep into the connection between individual electrical quantities by using its multi-head self-attention mechanism. By using federated learning framework, our approach utilizes the data from all nodes to collaboratively train a detection model while preserving data privacy by keeping the data locally during training. To improve the security of federated learning, a secure federated learning scheme is designed by combing Paillier cryptosystem with federated learning. Through extensive experiments on the IEEE 14-bus and 118-bus test systems, the effectiveness and superiority of the proposed method are verified.

Homomorphic Encryption-Based Federated Privacy Preservation for Deep Active Learning.

Active learning is a technique for maximizing performance of machine learning with minimal labeling effort and letting the machine automatically and adaptively select the most informative data for labeling. Since the labels on records may contain sensitive information, privacy-preserving mechanisms should be integrated into active learning. We propose a privacy-preservation scheme for active learning using homomorphic encryption-based federated learning. Federated learning provides distributed computation from multiple clients, and homomorphic encryption enhances the privacy preservation of user data with a strong security level. The experimental result shows that the proposed homomorphic encryption-based federated learning scheme can preserve privacy in active learning while maintaining model accuracy. Furthermore, we also provide a Deep Leakage Gradient comparison. The proposed scheme has no gradient leakage compared to the related schemes that have more than 74% gradient leakage.

Joint Optimal Quantization and Aggregation of Federated Learning Scheme in VANETs

Vehicular ad hoc networks (VANETs) is one of the most promising approaches for the Intelligent Transportation Systems (ITS). With the rapid increase in the amount of traffic data, deep learning based algorithms have been used extensively in VANETs. The recently proposed federated learning is an attractive candidate for collaborative machine learning where instead of transferring a plethora of data to a centralized server, all clients train their respective local models and upload them to the server for model aggregation. Model quantization is an effective approach to address the communication efficiency issue in federated learning, and yet existing studies largely assume homogeneous quantization for all clients. However, in reality, clients are predominantly heterogeneous, where they support different quantization precision levels. In this work, we propose FedDO – Federated Learning with Double Optimization. Minimizing the drift term in the convergence analysis, which is a weighted sum of squared quantization errors (SQE) over all clients, leads to a double optimization at both clients and server sides. In particular, each client adopts a fully distributed, instantaneous (per learning round) and individualized (per client) quantization scheme that minimizes its own squared quantization error, and the server computes the aggregation weights that minimize the weighted sum of squared quantization errors over all clients. We show via numerical experiments that the minimal-SQE quantizer has a better performance than a widely adopted linear quantizer for federated learning. We also demonstrate the performance advantages of FedDO over the vanilla FedAvg with standard equal weights and linear quantization.

Federated stochastic configuration networks for distributed data analytics

Stochastic configuration networks (SCNs), as a class of randomized learning models, are incrementally built under a supervisory mechanism, and theoretically ensure error-free learning for training sets. This paper proposes a federated version of SCNs (FSCNs) for large-scale data, which are geographically distributed among different end-user clients with non-shareable data due to privacy and security concerns. Unlike centralized learning that needs to collect data from clients and store them collectively on a cloud server, FSCNs enable distributed analytics in a collaborative learning paradigm without centrally aggregating new data, thereby preventing the leakage of private information. Considering different supervisory and aggregate schemes of model parameters, two FSC algorithms with two aggregate strategies are presented. The experiment results on both data regression and classification show the effectiveness and feasibility of our proposed federated learning scheme.

VFLF: A verifiable federated learning framework against malicious aggregators in Industrial Internet of Things

AbstractAs an important approach to overcome data silos and privacy concerns in deep learning, federated learning, which can jointly train the global model and keep data local, has shown remarkable performance in a range of industrial applications. However, federated learning still suffers from the problem that shared gradients may be subject to tampering, inference functions, and falsification. To address this issue, we propose a verifiable federated learning framework to deal with malicious aggregators. Initially, we propose a reputation calculation mechanism to solve the problem of selecting a reliable aggregator based on a multiweight subjective logic model. Furthermore, we design a verifiable federated learning scheme to ensure data confidentiality, integrity, and verifiability, as well as support the client's dynamic withdrawal. Security analyses indicate that our framework is secure against malicious adversaries. Furthermore, experimental results on real datasets show that our verifiable federated learning has high accuracy and feasible efficiency.

Relay-Assisted Cooperative Federated Learning

Federated learning (FL) has recently emerged as a promising technology to enable artificial intelligence (AI) at the network edge, where distributed mobile devices collaboratively train a shared AI model under the coordination of an edge server. To significantly improve the communication efficiency of FL, over-the-air computation allows a large number of mobile devices to concurrently upload their local models by exploiting the superposition property of wireless multi-access channels. Due to wireless channel fading, the model aggregation error at the edge server is dominated by the weakest channel among all devices, causing severe straggler issues. In this paper, we propose a relay-assisted cooperative FL scheme to effectively address the straggler issue. In particular, we deploy multiple half-duplex relays to cooperatively assist the devices in uploading the local model updates to the edge server. The nature of the over-the-air computation poses system objectives and constraints that are distinct from those in traditional relay communication systems. Moreover, the strong coupling between the design variables renders the optimization of such a system challenging. To tackle the issue, we propose an alternating-optimization-based algorithm to optimize the transceiver and relay operation with low complexity. Then, we analyze the model aggregation error in a single-relay case and show that our relay-assisted scheme achieves a smaller error than the one without relays provided that the relay transmit power and the relay channel gains are sufficiently large. The analysis provides critical insights on relay deployment in the implementation of cooperative FL. Extensive numerical results show that our design achieves faster convergence compared with state-of-the-art schemes.

Secure Partial Aggregation: Making Federated Learning More Robust for Industry 4.0 Applications

Big data, due to its promotion for industrial intelligence, has become the cornerstone of the Industry 4.0 era. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Federated learning</i> , proposed by Google, can effectively integrate data from different devices and different domains to train models under the premise of privacy preservation. Unfortunately, this new training paradigm faces security risks both on the client side and server side. This article proposes a new federated learning scheme to defend from client-side malicious uploads (e.g., backdoor attacks). In addition, we use cryptography techniques to prevent server-side privacy attacks (e.g., membership inference). The <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">secure partial aggregation</i> protocol we designed improves the privacy and robustness of federated learning. The experiments show that models can achieve high accuracy of over 90% with a proper upload proportion, while the accuracy of the backdoor attack decreased from 99.5% to 0% with the best result. Meanwhile, we prove that our protocol can disable privacy attacks.

A Communication-Efficient Federated Learning Scheme for IoT-Based Traffic Forecasting

Federated learning (FL) is widely adopted in traffic forecasting tasks involving large-scale IoT-enabled sensor data since its decentralization nature enables data providers’ privacy to be preserved. When employing <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">state-of-the-art</i> deep learning-based traffic predictors in FL systems, the existing FL frameworks confront overlarge communication overhead when transmitting these models’ parameter updates since the modeling depth and breadth renders them incorporating an enormous number of parameters. In this article, we propose a practical FL scheme, namely, Clustering-based hierarchical and Two-step-optimized FL (CTFed), to tackle this issue. The proposed scheme follows a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">divide et impera</i> strategy that clusters the clients into multiple groups based on the similarity between their local models’ parameters. We integrate the particle swarm optimization algorithm and devises a two-step approach for local model optimization. This scheme enables only one but representative local model update from each cluster to be uploaded to the central server, thus reduces the communication overhead of the model updates transmission in FL. CTFed is orthogonal to the gradient compression- or sparsification-based approaches so that they can orchestrate to optimize the communication overhead. Extensive case studies on three real-world data sets and three <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">state-of-the-art</i> models demonstrate the outstanding training efficiency, accurate prediction performance, and robustness to unstable network environments of the proposed scheme.

Toward Crowdsourced Transportation Mode Identification: A Semisupervised Federated Learning Approach

Privacy-preserving transportation mode identification (TMI) is among the key challenges toward future intelligent transportation systems. With recent developments in federated learning (FL), crowdsourcing has emerged as a promising cost-effective data source for training powerful TMI classifiers without compromising users’ data privacy. However, existing TMI approaches have relied heavily on the availability of transportation mode labels, which is often limited in real-world applications. While recent semisupervised studies have partially addressed this issue by assigning pseudolabels to unlabeled data, such practice often degrades classification performance as more unlabeled data are incorporated. In response to this issue, we present a semisupervised FL scheme for TMI termed mean teacher semisupervised FL (MTSSFL). MTSSFL trains a deep neural network ensemble under a novel semisupervised FL framework, achieving highly accurate and privacy-protected crowdsourced TMI without depending on the availability of massive labeled data. MTSSFL introduces <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">consistency updating</i> to insert the global model in the gradient updates of the local models that only have unlabeled data to improve their training. We also devise <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">mean-teacher-averaging</i> , a secure parameter aggregation mechanism that further boosts the global model’s TMI performance without requiring additional training. Our extensive case studies on a real-world data set demonstrate that MTSSFL’s classification accuracy is merely 1.1% lower than the state-of-the-art semisupervised TMI approach while being the only one to satisfy FL’s privacy-preserving constraints. In addition, MTSSFL can achieve high accuracy with less training overhead due to the proposed semisupervised learning design.

Are You Left Out?

Wearable computers engage in percutaneous interactions with human users and revolutionize the way of learning human activities. Due to rising privacy concerns, federated learning has been recently proposed to train wearable data with privacy preservation collaboratively. However, under the state-of-the-art (SOTA) schemes, user profiles on wearable devices of inferior networking conditions are regarded as 'left out'. Such schemes suffer from three fundamental limitations: (1) the widely adopted network-capacity-based client selection leads to biased training; (2) the aggregation has low communication efficiency; (3) users lack convenient channels for providing feedback on wearable devices. Therefore, this paper proposes a Fair and Communication-efficient Federated Learning scheme, namely FCFL. FCFL is a full-stack learning system specifically designed for wearable computers, improving the SOTA performance in terms of communication efficiency, fairness, personalization, and user experience. To this end, we design a technique named ThrowRightAway (TRA) to loose the network capacity constraints. Clients with poor networks are allowed to be selected as participators to improve the representation and guarantee the model's fairness. Remarkably, we propose Movement Aware Federated Learning (MAFL) to aggregate only the model updates with top contributions to the global model for the sake of communication efficiency. Accordingly, we implemented an FCFL-supported prototype as a sports application on smartwatches. Our comprehensive evaluation demonstrated that FCFL is a communication efficient scheme significantly reducing uploaded data by up to 29.77%, with a prominent feature of guaranteeing enhanced fairness up to 65.07%. Also, FCFL achieves robust personalization performance (i.e., 20% improvements of global model accuracy) in the face of packet loss below a certain fraction (10%-30%). A follow-up user survey shows that our FCFL-supported prototypical system on wearable devices significantly reduces users' workload.

SFedChain: blockchain-based federated learning scheme for secure data sharing in distributed energy storage networks.

The intelligence of energy storage devices has led to a sharp increase in the amount of detection data generated. Data sharing among distributed energy storage networks can realize collaborative control and comprehensive analysis, which effectively improves the clustering and intelligence. However, data security problems have become the main obstacle for energy storage devices to share data for joint modeling and analysis. The security issues caused by information leakage far outweigh property losses. In this article, we first proposed a blockchain-based machine learning scheme for secure data sharing in distributed energy storage networks. Then, we formulated the data sharing problem into a machine-learning problem by incorporating secure federated learning. Innovative verification methods and consensus mechanisms were used to encourage participants to act honestly, and to use well-designed incentive mechanisms to ensure the sustainable and stable operation of the system. We implemented the scheme of SFedChain and experimented on real datasets with different settings. The numerical results show that SFedChain is promising.

Federated Learning for COVID-19 Detection With Generative Adversarial Networks in Edge Cloud Computing

COVID-19 has spread rapidly across the globe and become a deadly pandemic. Recently, many artificial intelligence-based approaches have been used for COVID-19 detection, but they often require public data sharing with cloud datacentres and thus remain privacy concerns. This paper proposes a new federated learning scheme, called FedGAN, to generate realistic COVID-19 images for facilitating privacy-enhanced COVID-19 detection with generative adversarial networks (GANs) in edge cloud computing. Particularly, we first propose a GAN where a discriminator and a generator based on convolutional neural networks (CNNs) at each edge-based medical institution alternatively are trained to mimic the real COVID-19 data distribution. Then, we propose a new federated learning solution which allows local GANs to collaborate and exchange learned parameters with a cloud server, aiming to enrich the global GAN model for generating realistic COVID-19 images without the need for sharing actual data. To enhance the privacy in federated COVID-19 data analytics, we integrate a differential privacy solution at each hospital institution. Moreover, we propose a new blockchain-based FedGAN framework for secure COVID-19 data analytics, by decentralizing the FL process with a new mining solution for low running latency. Simulations results demonstrate the superiority of our approach for COVID-19 detection over the state-of-the-art schemes.

CRFL: A novel federated learning scheme of client reputation assessment via local model inversion

Federated learning (FL) is gradually becoming a key learning paradigm in Privacy-preserving Machine Learning (ML) systems. In FL, a large number of clients cooperate with a central server to learn a shared model without sharing their own data sets. However, since there is a great disparity between the client data sets, standard FL is often hard to tune and suffers from performance degradation due to the inharmony among local models. To this end, in this paper we propose a novel FL scheme, termed client reputation federated learning (CRFL), which dynamically assesses the reputation of the clients participating in FL. Our method leverages techniques from model explanation, and aims at precisely measure each client's impact to the global model. To be specific, we first calculate the saliency-weighted variance on pixelwise relevance scores as the quality factor of a single sample. Then we extract activation function values at the last hidden layer to compute the divergence factor of individual data set. Finally, the server integrates these two factors as an assessment of the client reputation. By leveraging such assessment, CRFL can dynamically adjust the weights of the clients in each aggregation round, thus leading to a significant improvement over the baseline method in terms of model accuracy and convergence rate. Intensive experiments are conducted on the MNIST and CIFAR-10 data sets, and experimental results demonstrate the efficacy of the proposed method.

Federated Transfer Learning Based Cross-Domain Prediction for Smart Manufacturing

Smart manufacturing aims to support highly customizable production processes. Therefore, the associated machine intelligence needs to be quickly adaptable to new products, processes, and applications with limited training data while preserving data privacy. In this article, a new federated transfer learning framework, federated transfer learning for cross-domain prediction, is proposed to address the challenges of data scarcity and data privacy faced by most machine learning approaches in modern smart manufacturing with cross-domain applications. The framework architecture consists of a central server and several groups of smart devices, where each group handles a different application. The existing applications can share their knowledge through the central server as base models, while new applications can convert a base model to their target-domain models with limited application-specific data using a transfer learning technique. Meanwhile, the federated learning scheme is deployed within a group to further enhance the accuracy of the application-specific model. The integrated framework allows model sharing across the central server and different smart devices without exposing any raw data and, hence, protects the data privacy. Two public datasets, COCO and PETS2009, which represent the source and target applications, are employed for evaluations. The simulation results show that the proposed method outperforms two state-of-the-art machine learning approaches by achieving better learning efficiency and accuracy.