Mitigating Unfairness in Differentially-Private Federated Learning

Differential privacy mechanisms can offer a trade-off between privacy and utility by using privacy metrics and utility metrics. The trade-off of differential privacy shows that one thing increases and another decreases in terms of privacy metrics and utility metrics. However, there is no unified trade-off measurement of differential privacy mechanisms. To this end, we proposed the definition of privacy-preserving monotonicity of differential privacy, which measured the trade-off between privacy and utility. First, to formulate the trade-off, we presented the definition of privacy-preserving monotonicity based on computational indistinguishability. Second, building on privacy metrics of the expected estimation error and entropy, we theoretically and numerically showed privacy-preserving monotonicity of Laplace mechanism, Gaussian mechanism, exponential mechanism, and randomized response mechanism. In addition, we also theoretically and numerically analyzed the utility monotonicity of these several differential privacy mechanisms based on utility metrics of modulus of characteristic function and variant of normalized entropy. Third, according to the privacy-preserving monotonicity of differential privacy, we presented a method to seek trade-off under a semi-honest model and analyzed a unilateral trade-off under a rational model. Therefore, privacy-preserving monotonicity can be used as a criterion to evaluate the trade-off between privacy and utility in differential privacy mechanisms under the semi-honest model. However, privacy-preserving monotonicity results in a unilateral trade-off of the rational model, which can lead to severe consequences.

Artificial Intelligence (AI) has attracted a large amount of attention in recent years. However, several new problems, such as privacy violations, security issues, or effectiveness, have been emerging. Differential privacy has several attractive properties that make it quite valuable for AI, such as privacy preservation, security, randomization, composition, and stability. Therefore, this paper presents differential privacy mechanisms for multi-agent systems, reinforcement learning, and knowledge transfer based on those properties, which proves that current AI can benefit from differential privacy mechanisms. In addition, the previous usage of differential privacy mechanisms in private machine learning, distributed machine learning, and fairness in models is discussed, bringing several possible avenues to use differential privacy mechanisms in AI. The purpose of this paper is to deliver the initial idea of how to integrate AI with differential privacy mechanisms and to explore more possibilities to improve AIs performance.

Bounded privacy-utility monotonicity indicating bounded tradeoff of differential privacy mechanisms

To preserve confidential information for numeric and character data, there are corresponding to differential privacy mechanisms. However, current work without uniform evaluation criterion for these differential privacy mechanisms, because the data types are different. In this paper, we proposed privacy-preserving monotonicity principle as an evaluation criterion of differential privacy mechanisms. Firstly, this paper summarized three perturbation paradigms of existing work, including the linear perturbation, non-linear perturbation, and randomized perturbation. Secondly, for numeric and character data, we proposed privacy-preserving monotonicity principle of differential privacy based on computational indistinguishability, respectively. Finally, through analysis privacy-preserving monotonicity of existing perturbation methods for each perturbation paradigm, we presented constrained perturbation paradigms for numeric and character data that can achieve privacy-preserving monotonicity. Therefore, our privacy-preserving monotonicity principle shows the tradeoff between privacy and utility, and it can be regarded as an evaluation criterion of differential privacy mechanisms. Furthermore, we show that constrained perturbation paradigms of maintaining privacy-preserving monotonicity provide a useful guideline for differential privacy development.

Federated learning (FL) has emerged as a pivotal paradigm for training machine learning models across decentralized devices while maintaining data privacy. In the healthcare domain, FL enables collaborative training among diverse medical devices and institutions, enhancing model robustness and generalizability without compromising patient privacy. In this paper, we propose DPS-GAT, a novel approach integrating graph attention networks (GATs) with differentially private client selection and resource allocation strategies in FL. Our methodology addresses the challenges of data heterogeneity and limited communication resources inherent in medical applications. By employing graph neural networks (GNNs), we effectively capture the relational structures among clients, optimizing the selection process and ensuring efficient resource distribution. Differential privacy mechanisms are incorporated, to safeguard sensitive information throughout the training process. Our extensive experiments, based on the Regensburg pediatric appendicitis open dataset, demonstrated the superiority of our approach, in terms of model accuracy, privacy preservation, and resource efficiency, compared to traditional FL methods. The ability of DPS-GAT to maintain a high and stable number of client selections across various rounds and differential privacy budgets has significant practical implications, indicating that FL systems can achieve strong privacy guarantees without compromising client engagement and model performance. This balance is essential for real-world applications where both privacy and performance are paramount. This study suggests a promising direction for more secure and efficient FL medical applications, which could improve patient care through enhanced predictive models and collaborative data utilization.

Differential privacy is often used in location privacy protection because of its strict reasoning and proof privacy guarantee. When users make continuous location query, it will cause noise superposition, which leads to the decline of query accuracy. At present, although differential privacy based on rule tree structure can reduce the query error, it will generate a lot of invalid zero nodes. The data structure is too large, and more improvement in the query accuracy can be further investigated. In this paper, we proposed a differential privacy location privacy protection method based on generative adversary network. Firstly, the definition of location data privacy protection under differential privacy mechanism is given, and then resume density aware network under differential privacy mechanism. Based on density aware network, the privacy protection problem of location data can be transformed into the distribution of fitting trajectory length. Finally, we use Markov chain to generate a new trajectory, and introduce the generative adversarial network to construct a set differential privacy protection method. Compared with other methods to improve the accuracy of differential privacy query, this method can effectively reduce the problem of query accuracy decline caused by noise superposition in continuous query, and can adapt to lbs location query service in different density environments.KeywordsGenerative adversarial networkLocation differencePrivacy protectionMarkov chainDensity aware network

Bitcoin is one of the best-known cryptocurrencies, which captivated researchers with its innovative blockchain structure. Examinations of this public blockchain resulted in many proposals for improvement in terms of anonymity and privacy. Generally used methods for improvement include mixing protocols, ring signatures, zero-knowledge proofs, homomorphic commitments, and off-chain storage systems. To the best of our knowledge, in the literature, there is no study examining Bitcoin in terms of differential privacy, which is a privacy notion coming up with some mechanisms that enable running useful statistical queries without identifying any personal information. In this paper, we provide a theoretical examination of differential privacy in Bitcoin. Our motivation arises from the idea that the Bitcoin public blockchain structure can benefit from differential privacy mechanisms for improved privacy, both making anonymization and privacy breaches by direct queries impossible, and preserving the checkability of the integrity of the blockchain. We first examine the current Bitcoin implementation for four query functions using the differential privacy formulation. Then, we present the feasibility of the utilization of two differential privacy mechanisms in Bitcoin; the noise addition to the transaction amounts and the user graph perturbation. We show that these mechanisms decrease the fraction of the cases violating differential privacy, therefore they can be used for improving anonymity and privacy in Bitcoin. Moreover, we showcase the noise addition to transaction amounts by using IBM Differential Privacy Library. We compare four differential privacy mechanisms for varying privacy parameter values and determine the feasible mechanisms and the parameters.

In the Bigdata era, healthcare informatics need exploration of health records to identify hidden patterns. Machine learning and Deep learning techniques provide classification, clustering and prediction tasks. Healthcare data are processed in a centralized architecture pose a single point of failure and difficult to collaborate with different distributions of data to design a robust system. The sensitive data of the healthcare system are private and fragmented, difficult to collaborate for efficient learning models. Federated learning (FL) is a distributed preservation of privacy learning paradigm to address the data sensitiveness and silos. The model is trained with different distributions of data with distributed models to provide the global model. The sensitive data present in the local device model are not shared with the global model but only the gradients are transmitted till the convergence of the model. The privacy-preserving mechanism is essential to protect the model from privacy attacks. Differential privacy preserving is immune to privacy attacks on aggregated data of FL. The striking features of blockchain like decentralization, provenance, immutability, and finality enables a single shared ledger of the patient data and its distribution among the stakeholders with the mitigation of privacy threats. Blockchain provides the secure transaction between the local health model and the global health model during its gradient updation. Differential privacy mechanism with blockchain provides secure E-health data maintenance and data analytics in the distributed healthcare 4.0 industry. This article identifies the current challenges in healthcare informatics and addresses those issues with enabling technologies like FL, blockchain and differential privacy preserving security mechanisms. Communication efficient FL, and fusion learning is identified for E-health data management. The extraction of knowledge structure in healthcare data provides insight into recent trends in the domain and its opportunities in the associated industries. This article in wholesome explores healthcare informatics with the application of artificial learning and security mechanisms.

Differential privacy mechanisms have been proposed to guarantee the privacy of individuals in various types of statistical information. When constructing a probabilistic mechanism to satisfy differential privacy, it is necessary to consider the impact of an arbitrary record on its statistics, i.e., sensitivity, but there are situations where sensitivity is difficult to derive. In this paper, we first summarize the situations in which it is difficult to derive sensitivity in general, and then propose a definition equivalent to the conventional definition of differential privacy to deal with them. This definition considers neighboring datasets as in the conventional definition. Therefore, known differential privacy mechanisms can be applied. Next, as an example of the difficulty in deriving sensitivity, we focus on the t-test, a basic tool in statistical analysis, and show that a concrete differential privacy mechanism can be constructed in practice. Our proposed definition can be treated in the same way as the conventional differential privacy definition, and can be applied to cases where it is difficult to derive sensitivity.

Data security and user privacy concerns are receiving increasing attention. Federated learning models based on differential privacy offer a distributed machine learning framework that protects data privacy. However, the noise introduced by the differential privacy mechanism may affect the model’s usability, especially when reasonable gradient clipping is absent. Fluctuations in the gradients can lead to issues like gradient explosion, compromising training stability and potentially leaking privacy. Therefore, gradient clipping has become a crucial method for protecting both model performance and data privacy. To balance privacy protection and model performance, we propose the Adaptive Weight-Based Differential Privacy Federated Learning (AWDP-FL) framework, which processes model gradient parameters at the neural network layer level. First, by designing and recording the change trends of two-layer historical gradient sequences, we analyze and predict gradient variations in the current iteration and calculate the corresponding weight values. Then, based on these weights, we perform adaptive gradient clipping for each data point in each training batch, which is followed by gradient momentum updates based on the third moment. Before uploading the parameters, Gaussian noise is added to protect privacy while maintaining model accuracy. Theoretical analysis and experimental results validate the effectiveness of this framework under strong privacy constraints.

The deployment of deep learning applications has to address the increasing privacy concerns when using private and sensitive data for training. A conventional deep learning model is prone to privacy attacks that can recover the sensitive information from either model parameters or accesses to the inference model. Recently, differential privacy (DP) has been proposed to offer provable privacy guarantees by randomizing the training process of neural networks. However, many approaches tend to provide the worst case privacy protection for model publishing, inevitably impairing the accuracy of the trained models. Thus, we present a novel private knowledge transfer strategy, where the private teacher trained on sensitive data is not publicly accessible but the student models can be released with privacy guarantees. In this paper, a three-player (teacher-student-discriminator) learning framework, Private Knowledge Distillation with Generative Adversarial Networks (PKDGAN), is proposed, where the student acquires the distilled knowledge from the teacher and is trained with the discriminator to generate similar outputs as the teacher. Moreover, a cooperative learning strategy is also suggested to support the collective training of multiple students against the discriminator when each student is with insufficient unlabelled training data. To enforce rigorous privacy guarantees, PKDGAN applies a Rényi differential privacy mechanism throughout the training process, and use it with a moment accountant technique to track the privacy cost. PKDGAN allows students to be trained with unlabelled public data and very few epochs, which avoids the exposure of training data while ensuring model performance. In the experiments, PKDGAN is found to have consistently good performance on various datasets (MNIST, SVHN, CIFAR-10, and Market-1501). When compared to prior works [1], [2], PKDGAN exhibits 5-82% accuracy loss improvement without compromising any privacy guarantee.

Differential privacy (DP) is an influential privacy measure and has been studied to protect private data. DP has been often studied in classical probability theory, but few researchers studied quantum versions of DP. In this paper, we consider classical-quantum DP mechanisms which (i) convert binary private data to quantum states and (ii) satisfy a quantum version of the DP constraint. The class of classical-quantum DP mechanisms contains classical DP mechanisms. As a main result, we show that some classical DP mechanism optimizes any information quantity satisfying the information processing inequality. Therefore, the performance of classical DP mechanisms attains that of classical-quantum DP mechanisms.

New generation head-mounted displays, such as VR and AR glasses, are coming into the market with already integrated eye tracking and are expected to enable novel ways of human-computer interaction in numerous applications. However, since eye movement properties contain biometric information, privacy concerns have to be handled properly. Privacy-preservation techniques such as differential privacy mechanisms have recently been applied to eye movement data obtained from such displays. Standard differential privacy mechanisms; however, are vulnerable due to temporal correlations between the eye movement observations. In this work, we propose a novel transform-coding based differential privacy mechanism to further adapt it to the statistics of eye movement feature data and compare various low-complexity methods. We extend the Fourier perturbation algorithm, which is a differential privacy mechanism, and correct a scaling mistake in its proof. Furthermore, we illustrate significant reductions in sample correlations in addition to query sensitivities, which provide the best utility-privacy trade-off in the eye tracking literature. Our results provide significantly high privacy without any essential loss in classification accuracies while hiding personal identifiers.

New generation head-mounted displays, such as VR and AR glasses, are coming into the market with already integrated eye tracking and are expected to enable novel ways of human-computer interaction in numerous applications. However, since eye movement properties contain biometric information, privacy concerns have to be handled properly. Privacy-preservation techniques such as differential privacy mechanisms have recently been applied to eye movement data obtained from such displays. Standard differential privacy mechanisms; however, are vulnerable due to temporal correlations between the eye movement observations. In this work, we propose a novel transform-coding based differential privacy mechanism to further adapt it to the statistics of eye movement feature data and compare various low-complexity methods. We extend the Fourier perturbation algorithm, which is a differential privacy mechanism, and correct a scaling mistake in its proof. Furthermore, we illustrate significant reductions in sample correlations in addition to query sensitivities, which provide the best utility-privacy trade-off in the eye tracking literature. Our results provide significantly high privacy without any essential loss in classification accuracies while hiding personal identifiers.

Differential privacy mechanism can maintain privacy-utility monotonicity. Thus, differential privacy mechanism does not obtain privacy-utility balance for numerical data. To this end, we provide privacy-utility balance of differential privacy mechanism with the collaborative perspective in this paper. First, we constructed the collaborative model achieving privacy-utility balance of differential privacy mechanism. Second, we presented the collaborative algorithm of differential privacy mechanism under our collaborative model. Third, our theoretical analysis showed that the collaborative algorithm of differential privacy mechanism could keep privacy-utility balance. Finally, our experimental results demonstrated that the collaborative differential privacy mechanism can maintain privacy-utility balance. Thus, we provide a new collaborative model to solve the privacy-utility balance problem of differential privacy mechanism. Our collaborative algorithm is easy to apply to query processing of numerical data.