Privacy-utility Tradeoff Research Articles

Barycentric and pairwise Rényi quantum leakage with application to privacy-utility trade-off

Barycentric and pairwise quantum Rényi leakages are proposed as two measures of information leakage for privacy and security analysis in quantum computing and communication systems. These quantities both require minimal assumptions on the eavesdropper, i.e. they do not make any assumptions on the eavesdropper’s attack strategy or the statistical prior on the secret or private classical data encoded in the quantum system. They also satisfy important properties of positivity, independence, post-processing inequality and unitary invariance. The barycentric quantum Rényi leakage can be computed by solving a semi-definite program; the pairwise quantum Rényi leakage possesses an explicit formula. The barycentric and pairwise quantum Rényi leakages form upper bounds on the maximal quantum leakage, the sandwiched quantum α -mutual information, the accessible information and the Holevo’s information. Furthermore, differentially private quantum channels are shown to bound these measures of information leakage. Global and local depolarizing channels, that are common models of noise in quantum computing and communication, restrict private or secure information leakage. Finally, a privacy-utility trade-off formula in quantum machine learning using variational circuits is developed. The privacy guarantees can only be strengthened, i.e. information leakage can only be reduced, if the performance degradation grows larger and vice versa .

Privacy-Preserving Synthetic Data Generation Method for IoT-Sensor Network IDS Using CTGAN.

The increased usage of IoT networks brings about new privacy risks, especially when intrusion detection systems (IDSs) rely on large datasets for machine learning (ML) tasks and depend on third parties for storing and training the ML-based IDS. This study proposes a privacy-preserving synthetic data generation method using a conditional tabular generative adversarial network (CTGAN) aimed at maintaining the utility of IoT sensor network data for IDS while safeguarding privacy. We integrate differential privacy (DP) with CTGAN by employing controlled noise injection to mitigate privacy risks. The technique involves dynamic distribution adjustment and quantile matching to balance the utility-privacy tradeoff. The results indicate a significant improvement in data utility compared to the standard DP method, achieving a KS test score of 0.80 while minimizing privacy risks such as singling out, linkability, and inference attacks. This approach ensures that synthetic datasets can support intrusion detection without exposing sensitive information.

Privacy-Preserving Analytics in HR Tech- Federated Learning and Differential Privacy Techniques for Sensitive Data

This paper explores the application of privacy-preserving analytics in human resources (HR), focusing on the synergistic use of federated learning and differential privacy. As HR departments increasingly leverage data-driven insights, the protection of sensitive employee information becomes paramount. Federated learning enables collaborative model training without centralizing raw data, while differential privacy adds calibrated noise to ensure individual data remains indiscernible. Together, these techniques form a robust framework for safeguarding HR data while enabling advanced analytics. The paper discusses the challenges of handling sensitive HR information, examines the implementation of federated learning and differential privacy, and demonstrates their combined effectiveness in maintaining data utility while ensuring privacy. By adopting these approaches, organizations can derive valuable workforce insights, comply with data protection regulations, and foster employee trust. This research contributes to the growing field of ethical data use in HR, offering a blueprint for balancing analytical capabilities with privacy imperatives in the modern workplace. Keywords—Privacy-preserving analytics, Federated learning, Differential privacy, HR analytics, Data protection, Employee privacy, Decentralized learning, GDPR compliance, CCPA compliance, Sensitive data handling, Data-driven HR, Privacy-utility trade-off.

Privacy protection against user profiling through optimal data generalization

Personalized information systems are information-filtering systems that endeavor to tailor information-exchange functionality to the specific interests of their users. The ability of these systems to profile users based on their search queries at Google, disclosed locations at Twitter or rated movies at Netflix, is on the one hand what enables such intelligent functionality, but on the other, the source of serious privacy concerns. Leveraging on the principle of data minimization, we propose a data-generalization mechanism that aims to protect users’ privacy against non-fully trusted personalized information systems. In our approach, a user may like to disclose personal data to such systems when they feel comfortable. But when they do not, they may wish to replace specific and sensitive data with more general and thus less sensitive data, before sharing this information with the personalized system in question. Generalization therefore may protect user privacy to a certain extent, but clearly at the cost of some information loss. In this work, we model mathematically an optimized version of this mechanism and investigate theoretically some key properties of the privacy-utility trade-off posed by this mechanism. Experimental results on two real-world datasets demonstrate how our approach may contribute to privacy protection and show it can outperform state-of-the-art perturbation techniques like data forgery and suppression by providing higher utility for a same privacy level. On a practical level, the implications of our work are diverse in the field of personalized online services. We emphasize that our mechanism allows each user individually to take charge of their own privacy, without the need to go to third parties or share resources with other users. And on the other hand, it provides privacy designers/engineers with a new data-perturbative mechanism with which to evaluate their systems in the presence of data that is likely to be generalizable according to a certain hierarchy, highlighting spatial generalization, with practical application in popular location based services. Overall, a data-perturbation mechanism for privacy protection against user profiling, which is optimal, deterministic, and local, based on a untrusted model towards third parties.

Robust Privatization With Multiple Tasks and the Optimal Privacy-Utility Tradeoff

Robust Privatization With Multiple Tasks and the Optimal Privacy-Utility Tradeoff

Achieving the Exactly Optimal Privacy-Utility Trade-Off With Low Communication Cost via Shared Randomness

Achieving the Exactly Optimal Privacy-Utility Trade-Off With Low Communication Cost via Shared Randomness

Edge Private Graph Neural Networks with Singular Value Perturbation

Graph neural networks (GNNs) play a key role in learning representations from graph-structured data and are demonstrated to be useful in many applications. However, the GNN training pipeline has been shown to be vulnerable to node feature leakage and edge extraction attacks. This paper investigates a scenario where an attacker aims to recover private edge information from a trained GNN model. Previous studies have employed differential privacy (DP) to add noise directly to the adjacency matrix or a compact graph representation. The added perturbations cause the graph structure to be substantially morphed, reducing the model utility. We propose a new privacy-preserving GNN training algorithm, Eclipse, that maintains good model utility while providing strong privacy protection on edges. Eclipse is based on two key observations. First, adjacency matrices in graph structures exhibit low-rank behavior. Thus, Eclipse trains GNNs with a low-rank format of the graph via singular values decomposition (SVD), rather than the original graph. Using the low-rank format, Eclipse preserves the primary graph topology and removes the remaining residual edges. Eclipse adds noise to the low-rank singular values instead of the entire graph, thereby preserving the graph privacy while still maintaining enough of the graph structure to maintain model utility. We theoretically show Eclipse provide formal DP guarantee on edges. Experiments on benchmark graph datasets show that Eclipse achieves significantly better privacy-utility tradeoff compared to existing privacy-preserving GNN training methods. In particular, under strong privacy constraints (𝜖 < 4), Eclipse shows significant gains in the model utility by up to 46%. We further demonstrate that Eclipse also has better resilience against common edge attacks (e.g., LPA), lowering the attack AUC by up to 5% compared to other state-of-the-art baselines.

SoK: Can Trajectory Generation Combine Privacy and Utility?

While location trajectories represent a valuable data source for analyses and location-based services, they can reveal sensitive information, such as political and religious preferences. Differentially private publication mechanisms have been proposed to allow for analyses under rigorous privacy guarantees. However, the traditional protection schemes suffer from a limiting privacy-utility trade-off and are vulnerable to correlation and reconstruction attacks. Synthetic trajectory data generation and release represent a promising alternative to protection algorithms. While initial proposals achieve remarkable utility, they fail to provide rigorous privacy guarantees. This paper proposes a framework for designing a privacy-preserving trajectory publication approach by defining five design goals, particularly stressing the importance of choosing an appropriate Unit of Privacy. Based on this framework, we briefly discuss the existing trajectory protection approaches, emphasising their shortcomings. This work focuses on the systematisation of the state-of-the-art generative models for trajectories in the context of the proposed framework. We find that no existing solution satisfies all requirements. Thus, we perform an experimental study evaluating the applicability of six sequential generative models to the trajectory domain. Finally, we conclude that a generative trajectory model providing semantic guarantees remains an open research question and propose concrete next steps for future research.

Dual defense: Combining preemptive exclusion of members and knowledge distillation to mitigate membership inference attacks

Membership inference (MI) attacks threaten user privacy through determining if a given data example has been used to train a target model. Existing MI defenses protect the membership privacy through preemptive exclusion of members techniques and knowledge distillation. Unfortunately, using either of these two defenses alone, the defense effect can still offers an unsatisfactory trade-off between membership privacy and utility.Given that the defense method that directly combines these two defenses is still very limited (e.g., the test accuracy of the target model is decreased by about 40% (in our experiments)), in this work, we propose a dual defense (DD) method that includes the preemptive exclusion of high-risk member samples module and the knowledge distillation module, which thwarts the access of the resulting models to the private training data twice to mitigate MI attacks. Our defense method can be divided into two steps: the preemptive exclusion of high-risk member samples (Step 1) and the knowledge distillation to obtain the protected student model (Step 2). We propose three types of exclusions: existing MI attacks-based exclusions, sample distances of members and nonmembers-based exclusions, and mutual information value-based exclusions, to preemptively exclude the high-risk member samples. During the knowledge distillation phase, we add ground-truth labeled data to the reference dataset to decrease the protected student model's dependency on soft labels, aiming to maintain or improve its test accuracy. Extensive evaluation shows that DD significantly outperforms state-of-the-art defenses and offers a better privacy-utility trade-off. For example, DD achieves ∼100% test accuracy improvement over the distillation for membership privacy (DMP) defense for ResNet50 trained on CIFAR100. DD simultaneously achieves the reductions in the attack effectiveness (e.g., the TPR@0.01%FPR of enhanced MI attacks decreased by 2.10% on the ImageNet dataset, the membership advantage (MA) of risk score-based attacks decreased by 56.30%) and improvements of the target models' test accuracies (e.g., by 42.80% on CIFAR100).

Local differential privacy for data security in key value pair data

Data collection using local differential privacy (LDP) has mainly been studied for homogeneous data. Several data categories, including key-value pairs, must be estimated simultaneously in real-world applications, including the frequency of keys and the mean values within each key. It is challenging to achieve an acceptable utility-privacy tradeoff using LDP for key-value data collection since the data has two aspects, and a client could have multiple key-value pairs. Current LDP approaches are not scalable enough to handle large and small datasets. When the dataset is small, there is insufficient data to calculate statistical parameters; when the dataset is enormous, such as in streaming data, there is a risk of data leakage due to the high availability of too much information. The result is unsuitable for examination due to the substantial amount of randomization used in some methods. Existing LDP approaches are mostly restricted to basic data categories like category and numerical values. To address these difficulties, this research developed the DKVALP (Differentially private key-value pairs) algorithm, which ensures differential privacy in key-value pair data. This DKVALP is a lightweight, differentially private data algorithm that generates random noise using an updated Laplace algorithm to ensure differential privacy for the data. According to execution outputs on synthetic and real-world datasets, the proposed DKVALP framework offers improved usefulness for both frequency and mean predictions over the similar LDP security as conventional approaches.

On the rate–distortion–perception–semantics tradeoff in low-rate regime for lossy compression

This paper formulates a multi-letter information theoretical optimization problem to characterize the tradeoff between rate, distortion, perception, and semantics. It turns out that our formulation is related to existing works, such as the rate–distortion theory, utility-privacy tradeoff, and rate–distortion–perception tradeoff. In this paper, we solve this problem in the low-rate regime by applying a local geometric approach, which allows us to express the optimal solutions analytically. By using such an approach, we obtain the analytical solution of the multi-letter optimization problem, which demonstrates the optimal rate–distortion–perception–semantics tradeoff, and characterizes the gain from increasing the number of letters as well as the limiting performance. Moreover, our result also validates the single-letter optimality of the scenarios above, which reveals the information theoretical insights of system and algorithm designs.

Generating Synthetic Health Sensor Data for Privacy-Preserving Wearable Stress Detection.

Smartwatch health sensor data are increasingly utilized in smart health applications and patient monitoring, including stress detection. However, such medical data often comprise sensitive personal information and are resource-intensive to acquire for research purposes. In response to this challenge, we introduce the privacy-aware synthetization of multi-sensor smartwatch health readings related to moments of stress, employing Generative Adversarial Networks (GANs) and Differential Privacy (DP) safeguards. Our method not only protects patient information but also enhances data availability for research. To ensure its usefulness, we test synthetic data from multiple GANs and employ different data enhancement strategies on an actual stress detection task. Our GAN-based augmentation methods demonstrate significant improvements in model performance, with private DP training scenarios observing an 11.90-15.48% increase in F1-score, while non-private training scenarios still see a 0.45% boost. These results underline the potential of differentially private synthetic data in optimizing utility-privacy trade-offs, especially with the limited availability of real training samples. Through rigorous quality assessments, we confirm the integrity and plausibility of our synthetic data, which, however, are significantly impacted when increasing privacy requirements.

Privacy-Preserving Gaze Data Streaming in Immersive Interactive Virtual Reality: Robustness and User Experience.

Eye tracking is routinely being incorporated into virtual reality (VR) systems. Prior research has shown that eye tracking data, if exposed, can be used for re-identification attacks [14]. The state of our knowledge about currently existing privacy mechanisms is limited to privacy-utility trade-off curves based on data-centric metrics of utility, such as prediction error, and black-box threat models. We propose that for interactive VR applications, it is essential to consider user-centric notions of utility and a variety of threat models. We develop a methodology to evaluate real-time privacy mechanisms for interactive VR applications that incorporate subjective user experience and task performance metrics. We evaluate selected privacy mechanisms using this methodology and find that re-identification accuracy can be decreased to as low as 14% while maintaining a high usability score and reasonable task performance. Finally, we elucidate three threat scenarios (black-box, black-box with exemplars, and white-box) and assess how well the different privacy mechanisms hold up to these adversarial scenarios. This work advances the state of the art in VR privacy by providing a methodology for end-to-end assessment of the risk of re-identification attacks and potential mitigating solutions. f.

Where’s Waldo? A framework for quantifying the privacy-utility trade-off in marketing applications

Firms can rely on various data protection methods to comply with the General Data Protection Regulation’s (GDPR) anonymization directive. We develop a privacy attack to estimate customers’ privacy risk and find that data protection methods commonly used in practice do not offer a reliable guarantee of privacy protection.We therefore develop a framework that describes the use of deep learning to generate synthetic data that are both (differentially) private, and useful for marketing analysts. Empirically, we apply our framework to two privacy-sensitive marketing applications in which an analyst is faced with everyday managerial practices. In contrast to GDPR’s directive to minimize data collection, we show that customers’ privacy risk can be reduced by blending into a large crowd: a “Where’s Waldo” effect. Our framework provides a data protection method with a formal privacy guarantee and allows analysts to quantify, control, and communicate privacy risk levels with stakeholders, draw meaningful insights, and share data even under privacy regulations.

The Costs of Anonymization: Case Study Using Clinical Data.

Sharing data from clinical studies can accelerate scientific progress, improve transparency, and increase the potential for innovation and collaboration. However, privacy concerns remain a barrier to data sharing. Certain concerns, such as reidentification risk, can be addressed through the application of anonymization algorithms, whereby data are altered so that it is no longer reasonably related to a person. Yet, such alterations have the potential to influence the data set's statistical properties, such that the privacy-utility trade-off must be considered. This has been studied in theory, but evidence based on real-world individual-level clinical data is rare, and anonymization has not broadly been adopted in clinical practice. The goal of this study is to contribute to a better understanding of anonymization in the real world by comprehensively evaluating the privacy-utility trade-off of differently anonymized data using data and scientific results from the German Chronic Kidney Disease (GCKD) study. The GCKD data set extracted for this study consists of 5217 records and 70 variables. A 2-step procedure was followed to determine which variables constituted reidentification risks. To capture a large portion of the risk-utility space, we decided on risk thresholds ranging from 0.02 to 1. The data were then transformed via generalization and suppression, and the anonymization process was varied using a generic and a use case-specific configuration. To assess the utility of the anonymized GCKD data, general-purpose metrics (ie, data granularity and entropy), as well as use case-specific metrics (ie, reproducibility), were applied. Reproducibility was assessed by measuring the overlap of the 95% CI lengths between anonymized and original results. Reproducibility measured by 95% CI overlap was higher than utility obtained from general-purpose metrics. For example, granularity varied between 68.2% and 87.6%, and entropy varied between 25.5% and 46.2%, whereas the average 95% CI overlap was above 90% for all risk thresholds applied. A nonoverlapping 95% CI was detected in 6 estimates across all analyses, but the overwhelming majority of estimates exhibited an overlap over 50%. The use case-specific configuration outperformed the generic one in terms of actual utility (ie, reproducibility) at the same level of privacy. Our results illustrate the challenges that anonymization faces when aiming to support multiple likely and possibly competing uses, while use case-specific anonymization can provide greater utility. This aspect should be taken into account when evaluating the associated costs of anonymized data and attempting to maintain sufficiently high levels of privacy for anonymized data. German Clinical Trials Register DRKS00003971; https://drks.de/search/en/trial/DRKS00003971. RR2-10.1093/ndt/gfr456.

Communicating the Privacy-Utility Trade-off: Supporting Informed Data Donation with Privacy Decision Interfaces for Differential Privacy

Data collections, such as those from citizen science projects, can provide valuable scientific insights or help the public to make decisions based on real demand. At the same time, the collected data might cause privacy risks for their volunteers, for example, by revealing sensitive information. Similar but less apparent trade-offs exist for data collected while using social media or other internet-based services. One approach to addressing these privacy risks might be to anonymize the data, for example, by using Differential Privacy (DP). DP allows for tuning and, consequently, communicating the trade-off between the data contributors' privacy and the resulting data utility for insights. However, there is little research that explores how to communicate the existing trade-off to users. % We contribute to closing this research gap by designing interactive elements and visualizations that specifically support people's understanding of this privacy-utility trade-off. We evaluated our user interfaces in a user study (N=378). Our results show that a combination of graphical risk visualization and interactive risk exploration best supports the informed decision, \ie the privacy decision is consistent with users' privacy concerns. Additionally, we found that personal attributes, such as numeracy, and the need for cognition, significantly influence the decision behavior and the privacy usability of privacy decision interfaces. In our recommendations, we encourage data collectors, such as citizen science project coordinators, to communicate existing privacy risks to their volunteers since such communication does not impact donation rates. %Understanding such privacy risks can also be part of typical training efforts in citizen science projects. %DP allows volunteers to balance their privacy concerns with their wish to contribute to the project. From a design perspective, we emphasize the complexity of the decision situation and the resulting need to design with usability for all population groups in mind. % We hope that our study will inspire further research from the human-computer interaction community that will unlock the full potential of DP for a broad audience and ultimately contribute to a societal understanding of acceptable privacy losses in specific data contexts.

Traceable mixnets

We introduce the notion of traceable mixnets. In a traditional mixnet, multiple mix-servers jointly permute and decrypt a list of ciphertexts to produce a list of plaintexts, along with a proof of correctness, such that the association between individual ciphertexts and plaintexts remains completely hidden. However, in many applications, the privacy-utility tradeoff requires answering some specific queries about this association, without revealing any information beyond the query result. We consider queries of the following types: a) given a ciphertext in the mixnet input list, whether it encrypts one of a given subset of plaintexts in the output list, and b) given a plaintext in the mixnet output list, whether it is a decryption of one of a given subset of ciphertexts in the input list. Traceable mixnets allow the mix-servers to jointly prove answers to the above queries to a querier such that neither the querier nor a threshold number of mix-servers learn any information beyond the query result. Further, if the querier is not corrupted, the corrupted mix-servers do not even learn the query result. We first comprehensively formalise these security properties of traceable mixnets and then propose a construction of traceable mixnets using novel distributed zero-knowledge proofs (ZKPs) of set membership and of a statement we call reverse set membership. Although set membership has been studied in the single-prover setting, the main challenge in our distributed setting lies in making sure that none of the mix-servers learn the association between ciphertexts and plaintexts during the proof. We implement our distributed ZKPs and show that they are faster than state-of-the-art by at least one order of magnitude.

A Generalized Shuffle Framework for Privacy Amplification: Strengthening Privacy Guarantees and Enhancing Utility

The shuffle model of local differential privacy is an advanced method of privacy amplification designed to enhance privacy protection with high utility. It achieves this by randomly shuffling sensitive data, making linking individual data points to specific individuals more challenging. However, most existing studies have focused on the shuffle model based on (ε0,0)-Locally Differentially Private (LDP) randomizers, with limited consideration for complex scenarios such as (ε0,δ0)-LDP or personalized LDP (PLDP). This hinders a comprehensive understanding of the shuffle model's potential and limits its application in various settings. To bridge this research gap, we propose a generalized shuffle framework that can be applied to PLDP setting. This generalization allows for a broader exploration of the privacy-utility trade-off and facilitates the design of privacy-preserving analyses in diverse contexts. We prove that the shuffled PLDP process approximately preserves μ-Gaussian Differential Privacy with μ = O(1/√n). This approach allows us to avoid the limitations and potential inaccuracies associated with inequality estimations. To strengthen the privacy guarantee, we improve the lower bound by utilizing hypothesis testing instead of relying on rough estimations like the Chernoff bound or Hoeffding's inequality. Furthermore, extensive comparative evaluations clearly show that our approach outperforms existing methods in achieving strong central privacy guarantees while preserving the utility of the global model. We have also carefully designed corresponding algorithms for average function, frequency estimation, and stochastic gradient descent.

Federated Learning via Input-Output Collaborative Distillation

Federated learning (FL) is a machine learning paradigm in which distributed local nodes collaboratively train a central model without sharing individually held private data. Existing FL methods either iteratively share local model parameters or deploy co-distillation. However, the former is highly susceptible to private data leakage, and the latter design relies on the prerequisites of task-relevant real data. Instead, we propose a data-free FL framework based on local-to-central collaborative distillation with direct input and output space exploitation. Our design eliminates any requirement of recursive local parameter exchange or auxiliary task-relevant data to transfer knowledge, thereby giving direct privacy control to local users. In particular, to cope with the inherent data heterogeneity across locals, our technique learns to distill input on which each local model produces consensual yet unique results to represent each expertise. Our proposed FL framework achieves notable privacy-utility trade-offs with extensive experiments on image classification and segmentation tasks under various real-world heterogeneous federated learning settings on both natural and medical images. Code is available at https://github.com/lsl001006/FedIOD.

Preserving fairness and diagnostic accuracy in private large-scale AI models for medical imaging

BackgroundArtificial intelligence (AI) models are increasingly used in the medical domain. However, as medical data is highly sensitive, special precautions to ensure its protection are required. The gold standard for privacy preservation is the introduction of differential privacy (DP) to model training. Prior work indicates that DP has negative implications on model accuracy and fairness, which are unacceptable in medicine and represent a main barrier to the widespread use of privacy-preserving techniques. In this work, we evaluated the effect of privacy-preserving training of AI models regarding accuracy and fairness compared to non-private training.MethodsWe used two datasets: (1) A large dataset (N = 193,311) of high quality clinical chest radiographs, and (2) a dataset (N = 1625) of 3D abdominal computed tomography (CT) images, with the task of classifying the presence of pancreatic ductal adenocarcinoma (PDAC). Both were retrospectively collected and manually labeled by experienced radiologists. We then compared non-private deep convolutional neural networks (CNNs) and privacy-preserving (DP) models with respect to privacy-utility trade-offs measured as area under the receiver operating characteristic curve (AUROC), and privacy-fairness trade-offs, measured as Pearson’s r or Statistical Parity Difference.ResultsWe find that, while the privacy-preserving training yields lower accuracy, it largely does not amplify discrimination against age, sex or co-morbidity. However, we find an indication that difficult diagnoses and subgroups suffer stronger performance hits in private training.ConclusionsOur study shows that – under the challenging realistic circumstances of a real-life clinical dataset – the privacy-preserving training of diagnostic deep learning models is possible with excellent diagnostic accuracy and fairness.