Privacy-preserving Data Research Articles

Energy Efficient and Differentially Private Federated Learning via a Piggyback Approach

This paper aims to develop a differential private federated learning (FL) scheme with the least artificial noises added while minimizing the energy consumption of participating mobile devices. By observing that some communication efficient FL approaches and even the nature of wireless communications contribute to the differential privacy (DP) preservation of training data on mobile devices, in this paper, we propose to jointly leverage gradient compression techniques (i.e., gradient quantization and sparsification) and additive white Gaussian noises (AWGN) in wireless channels to develop a piggyback DP approach for FL over mobile devices. Even with the piggyback DP approach, information distortion caused by gradient compression and noise perturbation may slow down FL convergence, which in turn consumes more energy of mobile devices for local computing and model update communications. Thus, we theoretically analyze FL convergence and formulate an energy efficient FL optimization under piggyback DP, transmission power, and FL convergence constraints. Furthermore, we propose an efficient iterative algorithm where closed-form solutions for artificial DP noise and power control are derived. Extensive simulation and experimental results demonstrate the effectiveness of the proposed scheme in terms of energy efficiency and privacy preservation.

Privacy Amplification via Shuffling: Unified, Simplified, and Tightened

The shuffle model of differential privacy provides promising privacy-utility balances in decentralized, privacy-preserving data analysis. However, the current analyses of privacy amplification via shuffling lack both tightness and generality. To address this issue, we propose the variation-ratio reduction as a comprehensive framework for privacy amplification in both single-message and multi-message shuffle protocols. It leverages two new parameterizations: the total variation bounds of local messages and the probability ratio bounds of blanket messages, to determine indistinguishability levels. Our theoretical results demonstrate that our framework provides tighter bounds, especially for local randomizers with extremal probability design, where our bounds are exactly tight. Additionally, variation-ratio reduction complements parallel composition in the shuffle model, yielding enhanced privacy accounting for popular sampling-based randomizers employed in statistical queries (e.g., range queries, marginal queries, and frequent itemset mining). Empirical findings demonstrate that our numerical amplification bounds surpass existing ones, conserving up to 30% of the budget for single-message protocols, 75% for multi-message ones, and a striking 75%-95% for parallel composition. Our bounds also result in a remarkably efficient Õ ( n ) algorithm that numerically amplifies privacy in less than 10 seconds for n = 10 8 users.

Adaptive personalized privacy-preserving data collection scheme with local differential privacy

Local differential privacy (LDP) is a state-of-the-art privacy notion that enables terminal participants to share their private data safely while controlling the privacy disclosure at the source. In most recent works, it is assumed that the privacy parameter is determined solely by collectors and then dispatched to all participants. However, it is inelegant and unpromising for all participants to accept the same level of privacy preservation due to their personalized preferences. Here, an adaptive data collection scheme is proposed to realize personalized privacy preservation while achieving higher data utility, in which two different LDP perturbation methods are adaptively chosen by data participants according to their personalized privacy preferences. The adaptive boundary based on the minimum mean square error (MSE) is theoretically and accurately derived to allow participants to adaptively choose the best perturbation method. Then, a weighted combination method is demonstrated to do effective data aggregation from multiple privacy groups. Moreover, an expanded data strategy (EDS) with multiple privacy perturbations is presented to equivalently increase the sample size without harming others privacy, thereby further improving the accuracy of statistics. Finally, the experiments show that the proposed scheme performs better than the previous proposal in terms of MSE and average error rate (AER), especially using the EDS method.

Advancements in Privacy-Preserving Techniques for Federated Learning: A Machine Learning Perspective

Federated learning has emerged as a promising paradigm for collaborative machine learning while preserving data privacy. However, concerns about data privacy remain significant, particularly in scenarios where sensitive information is involved. This paper reviews recent advancements in privacy-preserving techniques for federated learning from a machine learning perspective. It categorizes and analyses state-of-the-art approaches within a unified framework, highlighting their strengths, limitations, and potential applications. By providing insights into the landscape of privacy-preserving federated learning, this review aims to guide researchers and practitioners in developing robust and privacy-conscious machine learning solutions for collaborative environments. The paper concludes with future research directions to address ongoing challenges and further enhance the effectiveness and scalability of privacy-preserving federated learning.

AI Consciousness and Technological Advancement in Bangladesh's Higher Education: AI Awareness among the Learners

This comprehensive research article explores the transformative potential and multifaceted challenges of integrating Artificial Intelligence (AI) consciousness and technological advancements into the higher education system in Bangladesh. The study delves into various dimensions of this integration, including its implications for pedagogy, curriculum, institutional policy, and technological innovation. A particular emphasis is placed on addressing disparities in AI awareness among future students in Bangladesh's higher education system. The research adopts a balanced approach, juxtaposing the promise of technological advancements with critical concerns related to data privacy, educational equity, and cultural preservation. Through interdisciplinary analysis and a robust set of policy recommendations, the study posits that Bangladesh stands at a pivotal moment. It can either seize this technological epoch to elevate its higher education system to global standards while mitigating AI awareness disparities or risk exacerbating existing educational inequities.

Rethinking the defense against free-rider attack from the perspective of model weight evolving frequency

Federated learning (FL) is a distributed machine learning approach where multiple clients collaboratively train a joint model without exchanging their own data. Despite FL's unprecedented success in data privacy-preserving, its vulnerability towards free-rider attacks. Numerous defense methods have been proposed, however, they fail to resist highly camouflaged free-riders. To address these challenges, we reconsider the defense from a novel perspective, i.e., model weight evolving frequency. Empirically, we gain a novel insight that during the FL's training, the model weight evolving frequency of free-riders and that of benign clients are significantly different. Inspired by this insight, we propose a novel defense method based on the model Weight Evolving Frequency, referred to as WEF-Defense. Specifically, we first collect the weight evolving frequency (defined as WEF-Matrix) during local training. Each client uploads the WEF-Matrix of the local model as well as the model weights to the server. The server then separates free-riders from benign clients based on the difference in the WEF-Matrix. At last, the server provides different global models for the corresponding clients using a personalization algorithm, which prevents free-riders from gaining high-quality models. Comprehensive experiments conducted on five datasets and five models demonstrate that WEF-Defense achieves better defense effectiveness (∼×1.4) than the state-of-the-art baselines and identifies free-riders at an earlier stage of training. Besides, we verify the effectiveness of WEF-Defense against an adaptive attack and visualize the WEF-Matrix during the training to interpret its effectiveness. The data and code of WEF-Defense are available at: https://github.com/research-limingjun/WEF-Defense.git.

FedCRMW: Federated model ownership verification with compression-resistant model watermarking

Federated Learning is a collaborative machine learning paradigm that allows training models on decentralized data while preserving data privacy. It has gained significant attention due to its potential applications in various domains. However, the issue of protecting model copyright in the Federated Learning setting has become a critical concern. In this paper, we propose a novel watermarking framework called FedCRMW (Federal Learning Compression-Resistance Model Watermark) to address the challenge of model copyright protection in Federated Learning. FedCRMW embeds unique watermarks into client-contributed models, ensuring ownership, integrity, and authenticity. The framework leverages client-specific identifiers and exclusive logos to construct trigger sets for watermark embedding, enhancing security and traceability. One of the key advantages of FedCRMW is its optimization for the common data compression challenge in the Federated Learning scenario. By utilizing compressed data inputs for copyright verification, we achieve an efficient watermark validation process and reduce communication and storage overheads. Experimental results demonstrate the effectiveness of FedCRMW in terms of watermark success rate, imperceptibility, robustness against attacks, and resistance to model compression and pruning. Compared to existing watermarking methods, FedCRMW exhibits superior performance in the Federated Learning context.

Application analysis of data encryption technology

Data encryption technology is a key method to secure sensitive data. The privacy and confidentiality of data is protected by encrypting the data, i.e., transforming the original data into a form that cannot be understood without authorization. However, the security of traditional encryption algorithms is gradually challenged with the increase in computing power. Therefore, studying how to apply more advanced encryption techniques has become a current research hotspot. This study aims to analyze the application of data encryption techniques and explore the advantages and applicability of emerging encryption algorithms. This paper presents a variety of methods for preventing users' privacy breaches, with a primary focus on the principles of homomorphic encryption and how data can be accessed by users without decryption. Additionally, the working principles of secure multiparty computation are discussed, allowing multiple users to perform calculations on shared data while preserving data privacy. Furthermore, the paper explores data encryption techniques that employ specific algorithms to convert plaintext into ciphertext, ensuring both data consistency and privacy. Finally, a summary and future prospects are provided.

Empowering safe and secure autonomy: Federated learning in the era of autonomous driving

Artificial Intelligence (AI) has a significant impact on empowering autonomous driving systems to perceive and interpret the environment effectively. However, ensuring data privacy and security in autonomous driving systems is a critical challenge. To surmount these hurdles, federated learning has emerged as an effective strategy. Federated learning is a decentralized machine learning approach that facilitates the cooperative training of models across a diverse set of connected devices, enabling them to collectively learn and improve their performance, while preserving data privacy. This approach eliminates the necessity of sharing raw data and only involves sharing model updates with a central aggregator, thereby ensuring privacy and minimizing data exposure. This paper examines the implementation of federated learning in autonomous driving. It explores the principles of federated learning, including decentralized training, local model updates, model aggregation, privacy preservation, iterative learning, and heterogeneity handling. Two specific approaches, Deep Federated Learning (DFL) and End-to-End Federated Learning, are discussed, highlighting their benefits in enhancing privacy and maintaining prediction accuracy. The paper also discusses the applications of federated learning in communication and control aspects of autonomous driving. It emphasizes the scalability, adaptability, edge computing, real-time learning, federated transfer learning, and privacy-preserving data sharing as potential future prospects for federated learning in autonomous driving. Overall, federated learning offers a unique opportunity to address privacy concerns in autonomous driving systems while harnessing the collective intelligence of a fleet of vehicles. It has the potential to revolutionize the field and contribute to the development of safe and secure autonomous driving technologies.

Privacy preservation of medical data in blockchain with secret key generation using RideNN

Privacy preservation of medical data in blockchain with secret key generation using RideNN

FedCSL: A Scalable and Accurate Approach to Federated Causal Structure Learning

As an emerging research direction, federated causal structure learning (CSL) aims at learning causal relationships from decentralized data across multiple clients while preserving data privacy. Existing federated CSL algorithms suffer from scalability and accuracy issues, since they require computationally expensive CSL algorithms to be executed at each client. Furthermore, in real-world scenarios, the number of samples held by each client varies significantly, and existing methods still assign equal weights to the learned structural information from each client, which severely harms the learning accuracy of those methods. To address these two limitations, we propose FedCSL, a scalable and accurate method for federated CSL. Specifically, FedCSL consists of two novel strategies: (1) a federated local-to-global learning strategy that enables FedCSL to scale to high-dimensional data for tackling the scalability issue, and (2) a novel weighted aggregation strategy that does not rely on any complex encryption techniques while preserving data privacy for tackling the accuracy issue. Extensive experiments on benchmark datasets, high-dimensional synthetic datasets and a real-world dataset verify the efficacy of the proposed FedCSL method. The source code is available at https://github.com/Xianjie-Guo/FedCSL.

LR-XFL: Logical Reasoning-Based Explainable Federated Learning

Federated learning (FL) is an emerging approach for training machine learning models collaboratively while preserving data privacy. The need for privacy protection makes it difficult for FL models to achieve global transparency and explainability. To address this limitation, we incorporate logic-based explanations into FL by proposing the Logical Reasoning-based eXplainable Federated Learning (LR-XFL) approach. Under LR-XFL, FL clients create local logic rules based on their local data and send them, along with model updates, to the FL server. The FL server connects the local logic rules through a proper logical connector that is derived based on properties of client data, without requiring access to the raw data. In addition, the server also aggregates the local model updates with weight values determined by the quality of the clients’ local data as reflected by their uploaded logic rules. The results show that LR-XFL outperforms the most relevant baseline by 1.19%, 5.81% and 5.41% in terms of classification accuracy, rule accuracy and rule fidelity, respectively. The explicit rule evaluation and expression under LR-XFL enable human experts to validate and correct the rules on the server side, hence improving the global FL model’s robustness to errors. It has the potential to enhance the transparency of FL models for areas like healthcare and finance where both data privacy and explainability are important.

Data Disparity and Temporal Unavailability Aware Asynchronous Federated Learning for Predictive Maintenance on Transportation Fleets

Predictive maintenance has emerged as a critical application in modern transportation, leveraging sensor data to forecast potential damages proactively using machine learning. However, privacy concerns limit data sharing, making Federated learning an appealing approach to preserve data privacy. Nevertheless, challenges arise due to disparities in data distribution and temporal unavailability caused by individual usage patterns in transportation. In this paper, we present a novel asynchronous federated learning approach to address system heterogeneity and facilitate machine learning for predictive maintenance on transportation fleets. The approach introduces a novel data disparity aware aggregation scheme and a federated early stopping method for training. To validate the effectiveness of our approach, we evaluate it on two independent real-world datasets from the transportation domain: 1) oil dilution prediction of car combustion engines and 2) remaining lifetime prediction of plane turbofan engines. Our experiments show that we reliably outperform five state-of-the-art baselines, including federated and classical machine learning models. Moreover, we show that our approach generalises to various prediction model architectures.

Privacy-preserving data integration and sharing in multi-party IoT environments: An entity embedding perspective

The increasing prevalence of IoT applications highlights the urgency for insightful data fusion and information acquisition, boosting data integration and sharing needs. However, challenges arise in multi-party data sharing due to inherent data heterogeneity and privacy concerns. To address these issues, this paper discusses the feasibility of using embedding vectors as the semantic representation, aiming to enhance interoperability across diverse data sources and lay the foundation for natural language-based data querying. At the specific method level, this paper proposes an improved entity tree embedding algorithm to reduce information loss and ameliorate the representation of entity semantics. Additionally, a privacy preservation mechanism based on the entity embedding approach is introduced to provide privacy protection for text-based data. Experimental results on address data demonstrate the mechanism’s efficacy in achieving privacy protection comparable to the widely adopted 2D Laplace plane noise method. Furthermore, incorporating the entity tree embedding into the privacy mechanism could yield more robust and reasonable results regarding location privacy and service quality, signifying the validity of the entity embedding results.

A secure paillier cryptosystem based privacy-preserving data aggregation and query processing models for smart grid

A secure paillier cryptosystem based privacy-preserving data aggregation and query processing models for smart grid

Federated Learning-based neural networks for autonomous driving

An emerging technology with the capacity to revolutionize the transportation sector is autonomous driving, offering the promise of heightened safety, efficiency, and convenience. However, the widescale deployment of autonomous vehicles presents a multitude of challenges, notably the necessity for robust and adaptable machine learning (ML) models capable of handling a wide array of dynamic real-world scenarios. Enter Federated Learning (FL), a decentralized ML approach that has gained recognition as a potential solution to these challenges. This paper delves into the primary advantages of FL within the context of autonomous driving. It highlights FL's capacity to seamlessly adapt to edge devices, respond to localized changes, and continually enhance safety and performance. The document substantiates these advantages through numerous case studies and empirical evidence, demonstrating how FL can potentially elevate the vision, decision-making, control systems, data transmission, and learning model capabilities of autonomous vehicles. By harnessing the collective intelligence of autonomous vehicles while preserving data privacy and security, FL holds the potential to propel us closer to a future where safe, efficient, and autonomous transportation becomes an attainable reality.

Federated Learning-based neural networks for airline passenger satisfaction prediction

Preserving user data privacy is paramount for airlines. However, achieving highly accurate predictions of passenger satisfaction while safeguarding the privacy of individual companys data remains a considerable challenge. To achieve this objective, we utilized a privacy-preserving machine learning technique known as Federated Learning (FL). FL allows model training on decentralized devices while ensuring data security and privacy. The FL process comprises client training processes, communication rounds, and weight aggregation. We simulate FL principles using multiple processes to enable distributed learning. Clients preprocess data and train local models, ensuring data privacy. Communication rounds involve clients downloading the global model, local training, and transmitting updates to a central server. Weight aggregation methods like Federated Averaging merge these updates, preserving data privacy. Additionally, we leverage Artificial Neural Networks (ANNs) as foundational techniques. ANNs consist of input, hidden, and output layers, with weight adjustments based on real value differences to achieve accuracy. Our approach combines FL with ANNs to demonstrate FLs potential in privacy-preserving predictive analytics. We use the Airline Passenger Satisfaction dataset for modeling and evaluate the impact of neural network depth and submodel quantity on prediction accuracy. Our experimental results reveal that neither the depth of neural networks nor the number of submodels significantly affects prediction accuracy. FL emerges as a promising approach to balance data privacy and prediction accuracy effectively.

Federated Learning for Cybersecurity in Edge and Cloud Computing

Purpose: The article explores the integration of federated learning within edge and cloud computing frameworks to address complex cybersecurity challenges. It aims to illustrate how federated learning, by enabling collaborative model training across decentralized devices without data exchange, can serve as an effective mechanism for enhancing cybersecurity defenses. This study investigates the potential of federated learning to improve privacy-preserving data analysis and augment real-time threat detection capabilities in the context of the growing Internet of Things (IoT) ecosystem. Methodology: The research delves into the conceptual framework of federated learning, examining its application in cybersecurity contexts through a detailed literature review and theoretical analysis. It evaluates the benefits and limitations of federated learning in enhancing data privacy and reducing latency in threat detection. Furthermore, the article assesses the technical and security challenges of implementing federated learning, including communication overhead, model aggregation complexities, and vulnerability to model poisoning, through qualitative analysis. Findings: The study finds that federated learning significantly improves privacy-preserving data analysis and enhances real-time threat detection capabilities by keeping data localized while enabling collaborative learning. However, it also identifies key challenges in deploying federated learning strategies, such as the risk of model poisoning and the complexities involved in model aggregation and communication overhead. The research highlights the need for robust mechanisms to address these challenges to fully leverage federated learning in cybersecurity. Unique Contribution to Theory, Policy, and Practice: This article contributes uniquely to the theoretical understanding of federated learning as a cybersecurity measure, offering a comprehensive analysis of its applications, benefits, and limitations within edge and cloud computing environments. Practically, it provides insights for cybersecurity professionals and researchers on integrating federated learning into existing cybersecurity frameworks to enhance data privacy and threat detection. The article recommends further exploration into combining federated learning with other cutting-edge technologies to develop resilient cybersecurity measures. Additionally, it suggests that policymakers should consider the implications of federated learning on data privacy regulations and cybersecurity standards. Through its thorough examination of federated learning's potential and challenges, the article offers valuable recommendations for fortifying cybersecurity frameworks in an increasingly interconnected world.

Predicting ride-hailing passenger demand: A POI-based adaptive clustering federated learning approach

Passenger demand prediction is a key task for online ride-hailing platforms to optimize their resource allocation and service quality. However, centralized data collection and mining of massive passengers’ travel data expose serious privacy and security risks. To address this challenge, we propose a POI-based Adaptive Clustering Federated Learning with Spatio-Temporal Graph Attention Gate Recurrent Unit Networks (CFL-STG) algorithm that achieves accurate passenger demand prediction while protecting passenger privacy. Firstly, we propose a localized data storage mechanism for preserving data privacy and facilitating federated learning among different regions. Then, to tackle the statistical heterogeneity of data in federated learning, we introduce a clustering strategy that groups the regions based on their Point of Interest (POI) features, which reflect their demand patterns. For each group, we design a Spatio-Temporal Graph Attention Gate Recurrent Unit Network (STG) as a local model that can capture the spatio-temporal dynamics and inter-regional influence of POI features on passenger demand. Experiments on real data show that the proposed framework outperforms the existing federated learning models and common centralized data mining methods in terms of demand prediction performance and learning efficiency.

F1 Score Based Weighted Asynchronous Federated Learning

Abstract: The domain of federated learning has observed remarkable developments in recent years, enabling collaborative model training while preserving data privacy. This paper discusses several recent advancements in the field of federated learning, particularly in asynchronous and weighted federated learning. A novel approach within the federated learning paradigm titled "F1 Score Based Weighted Asynchronous Federated Learning" is introduced. The approach addresses issues of biased aggregation and device heterogeneity by assigning weights to devices based on their F1 scores, prioritizing those with superior performance in classification. Leveraging the asynchronous nature of federated learning, this approach enhances both resource efficiency and convergence speed. By incorporating F1 scores into the weighting mechanism, a balanced emphasis on precision and recall is achieved. It can optimize resource utilization, expediting the learning process by focusing on updates from devices with more accurate and valuable information. This approach can enhance collaborative model improvement while preserving data privacy in federated learning.