Aggregation Protocol Research Articles

Long-Term Privacy-Preserving Aggregation With User-Dynamics for Federated Learning

Privacy-preserving aggregation protocol is an essential building block in privacy-enhanced federated learning (FL), which enables the server to obtain the sum of users’ locally trained models while keeping local training data private. However, most of the work on privacy-preserving aggregation provides privacy guarantees for only one communication round in FL. In fact, as FL usually involves long-term training, i.e., multiple rounds, it may lead to more information leakages due to the dynamic user participation over rounds. In this connection, we propose a long-term privacy-preserving aggregation (LTPA) protocol providing both single-round and multi-round privacy guarantees. Specifically, we first introduce our batch-partitioning-dropping-updating (BPDU) strategy that enables any user-dynamic FL system to provide multi-round privacy guarantees. Then we present our LTPA construction which integrates our proposed BPDU strategy with the state-of-the-art privacy-preserving aggregation protocol. Furthermore, we investigate the impact of LTPA parameter settings on the trade-off between privacy guarantee, protocol efficiency, and FL convergence performance from both theoretical and experimental perspectives. Experimental results show that LTPA provides similar complexity to that of the state-of-the-art, i.e., an additional cost of around only 1.04X for a 100,000-user FL system, with an additional long-term privacy guarantee.

Secure Aggregation is Insecure: Category Inference Attack on Federated Learning

Federated learning allows a large number of resource-constrained clients to train a globally-shared model together without sharing local data. These clients usually have only a few classes (categories) of data for training, where the data distribution is non-iid (not independent identically distributed). In this article, we put forward the concept of <i>category privacy</i> for the first time to indicate <i>which classes of data a client has</i> , which is an important but ignored privacy goal in the federated learning with non-iid data. Although secure aggregation protocols are designed for federated learning to protect the input privacy of clients, we perform the first systematic study on <i>category inference attack</i> and demonstrate that these protocols cannot fully protect category privacy. We design a differential selection strategy and two de-noising approaches to achieve the attack goal successfully. In our evaluation, we apply the attack to non-iid federated learning settings with various datasets. On MNIST, CIFAR-10, AG_news, and DBPedia dataset, our attack achieves <inline-formula><tex-math notation="LaTeX">$&gt;90\%$</tex-math></inline-formula> accuracy measured in F1-score in most cases. We further consider a possible detection method and propose two strategies to make the attack more inconspicuous.

VerSA: Verifiable Secure Aggregation for Cross-Device Federated Learning

In privacy-preserving cross-device federated learning, users train a global model on their local data and submit encrypted local models, while an untrusted central server aggregates the encrypted models to obtain an updated global model. Prior work has demonstrated how to verify the correctness of aggregation in such a setting. However, such verification relies on strong assumptions, such as a trusted setup among all users under unreliable network conditions, or it suffers from expensive cryptographic operations, such as bilinear pairing. In this paper, we scrutinize the verification mechanism of prior work and propose a model recovery attack, demonstrating that most local models can be leaked within a reasonable time (e.g., <inline-formula><tex-math notation="LaTeX">$98\%$</tex-math></inline-formula> of encrypted local models are recovered within 21 h). Then, we propose <small>VerSA</small> , a verifiable secure aggregation protocol for cross-device federated learning. <small>VerSA</small> does not require any trusted setup for verification between users while minimizing the verification cost by enabling both the central server and users to utilize only a lightweight pseudorandom generator to prove and verify the correctness of model aggregation. We experimentally confirm the efficiency of <small>VerSA</small> under diverse datasets, demonstrating that <small>VerSA</small> is orders of magnitude faster than verification in prior work.

SAFELearning: Secure Aggregation in Federated Learning With Backdoor Detectability

For model privacy, local model parameters in federated learning shall be obfuscated before sent to the remote aggregator. This technique is referred to as <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">secure aggregation</i> . However, secure aggregation makes model poisoning attacks such as backdooring more convenient given that existing anomaly detection methods mostly require access to plaintext local models. This paper proposes a new federated learning technique SAFE-Learning to support backdoor detection for secure aggregation. We achieve this through two new primitives - <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">oblivious random grouping (ORG)</i> and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">partial parameter disclosure (PPD)</i> . ORG partitions participants into one-time random subgroups with group configurations oblivious to participants; PPD allows secure partial disclosure of aggregated subgroup models for anomaly detection without leaking individual model privacy. ORG is based on our construction of several new primitives including tree-based random subgroup generation, oblivious secure aggregation, and randomized Diffie-Hellman key exchange. ORG can thwart colluding attackers from knowing each other’s group membership assignment with non-negligible advantage than random guess. Backdoor attacks are detected based on statistical distributions of the subgroup aggregated parameters of the learning iterations. SAFELearning can significantly reduce backdoor model accuracy without jeopardizing the main task accuracy under common backdoor strategies. Extensive experiments show SAFELearning is robust against malicious and faulty participants, whilst being more efficient than the state-of-art secure aggregation protocol in terms of both communication and computation costs.

Cluster Head Selection and Multipath Routing Based Energy Efficient Wireless Sensor Network

The Wireless Sensor Network (WSN) is a network of Sensor Nodes (SN) which adopt radio signals for communication amongst themselves. There is an increase in the prominence of WSN adaptability to emerging applications like the Internet of Things (IoT) and Cyber-Physical Systems (CPS). Data security, detection of faults, management of energy, collection and distribution of data, network protocol, network coverage, mobility of nodes, and network heterogeneity are some of the issues confronted by WSNs. There is not much published information on issues related to node mobility and management of energy at the time of aggregation of data. Towards the goal of boosting the mobility-based WSNs’ network performance and energy, data aggregation protocols such as the presently-used Mobility Low-Energy Adaptive Clustering Hierarchy (LEACH-M) and Energy Efficient Heterogeneous Clustered (EEHC) scheme have been examined in this work. A novel Artificial Bee Colony (ABC) algorithm is proposed in this work for effective election of CHs and multipath routing in WSNs so as to enable effective data transfer to the Base Station (BS) with least energy utilization. There is avoidance of the local optima problem at the time of solution space search in this proposed technique. Experimentations have been conducted on a large WSN network that has issues with mobility of nodes.

LSFL: A Lightweight and Secure Federated Learning Scheme for Edge Computing

Nowadays, many edge computing service providers expect to leverage the computational power and data of edge nodes to improve their models without transmitting data. Federated learning facilitates collaborative training of global models among distributed edge nodes without sharing their training data. Unfortunately, existing privacy-preserving federated learning applied to this scenario still faces three challenges: 1) It typically employs complex cryptographic algorithms, which results in excessive training overhead; 2) It cannot guarantee Byzantine robustness while preserving data privacy; and 3) Edge nodes have limited computing power and may drop out frequently. As a result, the privacy-preserving federated learning cannot be effectively applied to edge computing scenarios. Therefore, we propose a lightweight and secure federated learning scheme LSFL, which combines the features of privacy-preserving and Byzantine-Robustness. Specifically, we design the Lightweight Two-Server Secure Aggregation protocol, which utilizes two servers to enable secure Byzantine robustness and model aggregation. This scheme protects data privacy and prevents Byzantine nodes from influencing model aggregation. We implement and evaluate LSFL in a LAN environment, and the experiment results show that LSFL meets fidelity, security, and efficiency design goals, and maintains model accuracy compared to the popular FedAvg scheme.

Privacy-Enhanced and Verification-Traceable Aggregation for Federated Learning

Federated learning (FL) is a distributed machine learning framework, which allows multiple users to collaboratively train and obtain a global model with high accuracy. Currently, FL is paid more attention by researchers and a growing number of protocols are proposed. This article first analyzes the security vulnerabilities of the VerifyNet and VeriFL protocols, and proposes a new aggregation protocol for FL. We use additive homomorphic encryption and double masking to simultaneously protect the user’s local model and the aggregated global model while most of the existing protocols only consider the privacy of the local model. Also, linear homomorphic hash and digital signature are used to achieve traceable verification, which means the users can not only verify the aggregation results, but also be able to identify the wrong epoch if the results are wrong. In summary, our protocol can realize the privacy of the local model and global model and achieve verification traceability even if the cloud server colludes with malicious users. The experimental results show that the proposed protocol improves the security of FL without decreasing the efficiency of the users and classification accuracy of the training model.

HAMIL: Hierarchical aggregation-based multi-instance learning for microscopy image classification

Multi-instance learning is common for computer vision tasks, especially in biomedical image processing. Traditional methods for multi-instance learning focus on designing feature aggregation methods and multi-instance classifiers, where the aggregation operation is performed either in the feature extraction or learning phase. As deep neural networks (DNNs) achieve great success in image processing via automatic feature learning, certain feature aggregation mechanisms need to be incorporated into common DNN architecture for multi-instance learning. Moreover, flexibility and reliability are crucial considerations to deal with varying quality and number of instances. In this study, we propose a hierarchical aggregation network for multi-instance learning, called HAMIL. The hierarchical aggregation protocol enables feature fusion in a defined order, and the simple convolutional aggregation units lead to an efficient and flexible architecture. We assess the model performance on two microscopy image classification tasks, namely protein subcellular localization using immunofluorescence images and gene annotation using spatial gene expression images. The experimental results show that HAMIL outperforms the state-of-the-art feature aggregation methods and the existing models for addressing these two tasks. The visualization analyses also demonstrate the ability of HAMIL to focus on high-quality instances.

SDA-RDOS: A New Secure Data Aggregation Protocol for Wireless Sensor Networks in IoT Resistant to DOS Attacks

In a typical Wireless Sensor Network (WSN), thousands of sensor nodes can be distributed in the environment. Then, each sensor node transmits its detected data to the base station with the help of cooperation. In this type of network, data aggregation protocols are used to increase the network’s lifetime and reduce each sensor node’s communication load and energy consumption. With Data Clustering, the density of data circulating in the network is reduced, thus increasing the network’s life. Energy, delay, and efficiency are essential criteria in Data Clustering; however, security is another crucial aspect to be considered. A comprehensive solution for secure data clustering has yet to be seen when the literature is examined. In the solutions developed, data availability, which means that the WSN is resistant to Denial of Service (DOS) attacks, has been neglected too much, even though confidentiality, integrity, and authentication are met with different algorithms. This study developed a comprehensive, secure clustering protocol by considering all security requirements, especially data availability. The developed protocol uses the blowfish encryption algorithm, EAX mode, and RSA algorithm. The proposed protocol was theoretically analyzed, empirically evaluated, and simulated from many perspectives. Comparisons were made with LSDAR, SUCID, and OOP-MDCRP protocols. As a result of the study, a comprehensive security solution is provided and more successful results were obtained according to Energy Efficiency, Network Lifetime, Average Delay, and Packet delivery ratio criteria.

A pairing-free data authentication and aggregation mechanism for Intelligent Healthcare System

The Intelligent Healthcare System (IHS), a prominent medical area underlying the Internet of Things (IoT), might be deployed to gather and analyze health information across a range of sources to deliver improved as well as more economical medical treatment. Managing all the health-related information over the internet becomes a challenging chore as numerous people rely on it implicitly or explicitly. The significant challenges of IHS comprise the privacy and security along with protection of aggregation of patients’ data as sensors obtain and transmit patient’s health information. The few of surveyed existing works based on security and privacy goals for IHS are inefficient due to cumbersome of huge computational cost and non-resilience of different attacks. Therefore, to resolve such challenges: to provide the robustness, preserve security goals and improve the efficiency, we introduced an efficient data authentication and aggregation protocol for IHS without pairing and ID-based cryptography. Provable security has been provided with Random Oracle model (ROM), which proofs the robustness of the presented scheme. Thus, the defined security requirements of our proposed work have been achieved. The performance analysis through MIRACL shows that the scheme presented outperforms the similar prior mechanisms in terms of computational-communicational cost and energy overheads.

CluRMA: A cluster-based RSU-enabled message aggregation scheme for vehicular ad hoc networks

The fundamental goal of Vehicular Ad hoc NETworks (VANETs) is to facilitate safe driving by sending safety/warning messages to vehicles as well as to exchange non-safety/infotainment messages among vehicles. Flooding is a common broadcasting technique and it is mainly used for disseminating safety/non-safety messages. However, flooding causes broadcast storm problem which is the most challenging issue in VANETs. In a dense vehicular network, high message generation rate demanding more transmissions could further increases the effects of the broadcast storm problem. In this context, message aggregation schemes could offer a great relief. In this paper, we propose a cluster based RSU enabled message aggregation protocol (CluRMA) which applies two levels of message aggregation — local aggregation at the cluster heads and global aggregation at the Road Side Unit (RSU). Before message aggregation process is performed, we make use of cosine distance for similarity checking between the messages for eliminating the duplicates. Then the syntactic aggregation is accomplished by employing message compression techniques — Adaptive Huffman Compression technique for safety messages, and Arithmetic Coding Technique for non-safety messages. The competence of the proposed scheme has been rigorously analyzed and compared with the existing ones regarding several metrics including delay, packet delivery ratio, collision rate, redundancy rate and overload.

Differentially Private Byzantine-Robust Federated Learning

Federated learning is a collaborative machine learning framework where a global model is trained by different organizations under the privacy restrictions. Promising as it is, privacy and robustness issues emerge when an adversary attempts to infer the private information from the exchanged parameters or compromise the global model. Various protocols have been proposed to counter the security risks, however, it becomes challenging when one wants to make federated learning protocols robust against Byzantine adversaries while preserving the privacy of the individual participant. In this article, we propose a differentially private Byzantine-robust federated learning scheme (DPBFL) with high computation and communication efficiency. The proposed scheme is effective in preventing adversarial attacks launched by the Byzantine participants and achieves differential privacy through a novel aggregation protocol in the shuffle model. The theoretical analysis indicates that the proposed scheme converges to the approximate optimal solution with the learning error dependent on the differential privacy budget and the number of Byzantine participants. Experimental results on MNIST, FashionMNIST and CIFAR10 demonstrate that the proposed scheme is effective and efficient.

G-VCFL: Grouped Verifiable Chained Privacy-Preserving Federated Learning

Federated learning, as a typical distributed learning paradigm, shows great potential in Industrial Internet of Things, Smart Home, Smart City, etc. It enables collaborative learning without data leaving local users. Despite the huge benefits, it still faces the risk of privacy breaches and a single point of failure for aggregation server. Adversaries can use intermediate models to infer user privacy, or even return incorrect global model by manipulating the aggregation server. To address these issues, several federated learning solutions focusing on privacy-preserving and security have been proposed. However, theses solutions still faces challenges in resource-limited scenarios. In this paper, we propose G-VCFL, a grouped verifiable chained privacy-preserving federated learning scheme. Specifically, we first use the grouped chain learning mechanism to guarantee the privacy of users, and then propose a verifiable secure aggregation protocol to guarantee the verifiability of the global model. G-VCFL does not require any complex cryptographic primitives and does not introduce noise, but enables verifiable privacy-preserving federated learning by utilizing lightweight pseudorandom generators. We conduct extensive experiments on real-world datasets by comparing G-VCFL with other state-of-the-art approaches. The experimental results and functional evaluation indicate that G-VCFL is efficient in the six experimental cases and satisfies all the intended design goals.

Distributed Fog Computing and Federated-Learning-Enabled Secure Aggregation for IoT Devices

Federated learning (FL), as a prospective way to process and analyze the massive data from the Internet of Things (IoT) devices, has attracted increasing attention from academia and industry. However, considering the unreliable nature of IoT devices, ensuring the efficiency of FL while protecting the privacy of devices’ input data is a challenging task. To address these issues, we propose a secure aggregation protocol based on efficient additive secret sharing in the fog-computing (FC) setting. As the secure aggregation is performed frequently in the training process of FL, the protocol should have low communication and computation overhead. First, we use a fog node (FN) as an intermediate processing unit to provide local services which can assist the cloud server aggregated the sum during the training process. Second, we design a light Request-then-Broadcast method to ensure our protocol has the robustness to dropped-out clients. Our protocol also provides two simple new client selection methods. The security and performance of our protocol are analyzed and compared with existed schemes. We conduct experiments on high-dimensional inputs, and our experimental results demonstrate about 24– <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$168\times $ </tex-math></inline-formula> improvement in computation overhead and 87– <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$287\times $ </tex-math></inline-formula> improvement in communication overhead compared to Google’s secure aggregation protocol (Bonwatiwz et al. CCS17).

A Survey of Data Aggregation Protocols for Energy Conservation in WSN and IoT

Conservation of energy has been a major concern for Wireless Sensor Networks (WSNs) and IoT applications. Several strategies were devised, aimed at optimizing energy consumption in these applications, based on: (a) use of low-powered hardware devices, (b) deploying mobile/relay agents for data collection, (c) clustering, and (d) data aggregation. Amongst these, data aggregation is widely acknowledged as an important tool to conserve energy in WSN and IoTs. The paper provides a comprehensive survey of various data aggregation strategies, discusses the efficacy of these strategies in handling issues that are typical to WSN and IoT applications. These issues severely impact the performance metrics such as: energy efficiency, latency, fault-tolerance, network throughput, and network lifetime. Therefore, to optimize the data aggregation approach, an application developer needs to arrive at optimal tradeoffs between these parameters. A major contribution of the paper is to present a holistic review of data aggregation approaches emphasizing the effect of topology, security, mobility, interference, and fault-tolerance in WSN and IoTs. Based on gap areas in literature, we throw open few challenges and present them as “posers”, and put-forth suggestions for further research.

Top-k sparsification with secure aggregation for privacy-preserving federated learning

The proposal of federated learning solves problems of data silos and privacy protection in the field of artificial intelligence. However, privacy attacks can infer or reconstruct sensitive information from the submitted gradient, which causes users’ privacy leakage in federated learning. Secure aggregation (SecAgg) protocol can protect users’ privacy while completing federated learning tasks, but it incurs significant communication overhead and wall clock training time on large-scale model training task. Thus, it is difficult to apply SecAgg in bandwidth-limited federated applications. Recently, Rand-k sparsification with secure aggregation (Rand-k SparseSecAgg) was proposed to optimize SecAgg protocol, while its optimization of communication overhead and training time is limited. In this paper, we replace Rand-k sparsification with Top-k sparsification, and design a Top-k sparsification with secure aggregation (Top-k SparseSecAgg) protocol for privacy-preserving federated learning to further reduce communication overhead and wall clock training time. In addition, we optimize the proposed protocol by assigning clients to different groups in the logical layer, which reduces the upper limit of compression ratio and practical communication overhead in Top-k SparseSecAgg. Experiments demonstrate that Top-k SparseSecAgg can reduce communication overhead by 6.25× as compared to SecAgg, 3.78× as compared to Rand-k SparseSecAgg, and reduce wall clock training time 1.43× as compared to SecAgg and 1.13× as compared to Rand-k SparseSecAgg. Thus, our protocol is more suitable in bandwidth-limited federated applications to protect privacy and complete learning task.

Integrity Protection for Data Aggregation in Smart Grid

To evaluate the working state of the smart grid, users need to count the status of devices collected by multiple smart meters. To avoid network congestion during data collection processes in the smart grid, an aggregation protocol is required to aggregate messages from multiple smart meters into one short message. However, since these messages may contain sensitive information of smart meters, privacy is a fundamental requirement for such a data aggregation protocol. At the same time, to avoid tampering of transmitted messages, data integrity is another important requirement during data aggregation. Currently, the privacy issue has been well addressed by using homomorphic encryption algorithms such as the Paillier cryptographic system, where multiple smart meters use the same encryption key for encrypting different messages. However, the integrity issue is much harder than the privacy issue since multiple smart meters use different private keys for signing different messages. To address the integrity issue, we propose a novel data aggregation protocol for the smart grid, called DASG. In DASG, we shall show how to aggregate signatures from multiple smart meters using the Chinese Remainder theorem and the Lagrange interpolation techniques, respectively. Since these two techniques can aggregate multiple messages into one, DASG is quite light-weight. Therefore, our newly designed data aggregation protocol for the smart grid can achieve both security and efficiency goals in the smart grid environment. Experimental results show that DASG is feasible for real world applications.

An Energy Aware Grid-Based Clustering Power Efficient Data Aggregation Protocol for Wireless Sensor Networks

A wireless sensor network (WSN) is made up of multiple sensors deployed in a specific sensing area to identify the occurrence of events and quickly transmit useful information back to the base station (BS). In WSNs, schemes to reduce energy consumption are an important topic of research. A well-designed data transmission scheme can effectively extend the lifetime of a network. In this paper, we propose an energy aware grid-based clustering power efficient data aggregation protocol (GB-PEDAP) for WSNs. The proposed scheme has a two-layer architecture: the inner layer and the outer layer. The inner layer uses direct transmission to collect the data of the cluster (cell), and the outer layer uses a tree structure transmission to collect the data of the cluster head (cell head). In our simulations, the number of rounds executed by GB-PEDAP was approximately 1.2 rounds of TBEEP, 1.3 rounds of GSTEB, and 1.5 rounds of PEDAP. With the initial energy, 0.25 J, the execution rounds of the first node death for GB-PEDAP, TBEEP, GSTEB, and PEDAP were 751, 572, 486, and 339, respectively. The proposed GB-PEDAP can evenly disperse the energy consumption of sensors to avoid the rapid death of sensors, extending the lifetime of a WSN.

DHSA: efficient doubly homomorphic secure aggregation for cross-silo federated learning

Secure aggregation is widely used in horizontal federated learning (FL), to prevent the leakage of training data when model updates from data owners are aggregated. Secure aggregation protocols based on homomorphic encryption (HE) have been utilized in industrial cross-silo FL systems, one of the settings involved with privacy-sensitive organizations such as financial or medical, presenting more stringent requirements on privacy security. However, existing HE-based solutions have limitations in efficiency and security guarantees against colluding adversaries without a Trust Third Party. This paper proposes an efficient Doubly Homomorphic Secure Aggregation (DHSA) scheme for cross-silo FL, which utilizes multi-key homomorphic encryption (MKHE) and seed homomorphic pseudorandom generator (SHPRG) as cryptographic primitives. The application of MKHE provides strong security guarantees against up to \(N-2\) participates colluding with the aggregator, with no TTP required. To mitigate the large computation and communication cost of MKHE, we leverage the homomorphic property of SHPRG to replace the majority of MKHE computation by computationally friendly mask generation from SHPRG, while preserving the security. Overall, the resulting scheme satisfies the stringent security requirements of typical cross-silo FL scenarios, at the same time providing high computation and communication efficiency for practical usage. We experimentally demonstrate that our scheme brings a speedup to 20\(\times \) over the state-of-the-art HE-based secure aggregation and reduces the traffic volume to approximately 1.5\(\times \) inflation over the plain learning setting.

Delay-aware relay node selection for cluster-based wireless sensor networks

Maximizing the network lifetime through balancing energy and lowering the delay is a primary design goal of wireless sensor networks (WSNs). In WSNs, the sensor nodes task is to acquire the data from the area or point of interest and transmit it to the sink node via relay nodes timely. Relay nodes must dissipate some energy to perform this task. So, the relay nodes and cluster heads face the challenge of energy balancing. Hence, choosing the optimal cluster heads and relay nodes is necessary to balance the energy among the sensor nodes. In this context, this paper presents a novel clustering algorithm to balance energy by considering different metrics of WSNs, including distance, energy, link lifetime, and delay characteristics—using a relay node, etc.In this paper, a Hybrid Heuristic Data Aggregation (HHDAP) Protocol is used by combining particle swarm optimization (PSO) and improved Ant colony optimization (ACO).Through this HHDAP algorithm, the relay nodes are chosen based on a sensor node's distance, energy, link lifetime, and delay metrics. This work is simulated using NS2, and the performance of proposed and existing works are evaluated. The results show the performance improvements of the proposed work over the existing algorithms in terms of different quality of service metrics.