Disclosure Risk Measures Research Articles

A comparison of synthetic data approaches using utility and disclosure risk measures

Seongbin An, Trang Doan, Juhee Lee, Jiwoo Kim, Yong Jae Kim, Yunji Kim, Changwon Yoon, Sungkyu Jung, Dongha Kim, Sunghoon Kwon, Hang J Kim, Jeongyoun Ahn, Cheolwoo Park. Korean J Appl Stat 2023;36:141-66. https://doi.org/10.5351/KJAS.2023.36.2.141

Flexible adversary disclosure risk measure for identity and attribute disclosure attacks

Flexible adversary disclosure risk measure for identity and attribute disclosure attacks

Statistical Data Privacy: A Song of Privacy and Utility

To quantify trade-offs between increasing demand for open data sharing and concerns about sensitive information disclosure, statistical data privacy (SDP) methodology analyzes data release mechanisms that sanitize outputs based on confidential data. Two dominant frameworks exist: statistical disclosure control (SDC) and the more recent differential privacy (DP). Despite framing differences, both SDC and DP share the same statistical problems at their core. For inference problems, either we may design optimal release mechanisms and associated estimators that satisfy bounds on disclosure risk measures, or we may adjust existing sanitized output to create new statistically valid and optimal estimators. Regardless of design or adjustment, in evaluating risk and utility, valid statistical inferences from mechanism outputs require uncertainty quantification that accounts for the effect of the sanitization mechanism that introduces bias and/or variance. In this review, we discuss the statistical foundations common to both SDC and DP, highlight major developments in SDP, and present exciting open research problems in private inference.

Balancing data privacy and usability in the federal statistical system

The federal statistical system is experiencing competing pressures for change. On the one hand, for confidentiality reasons, much socially valuable data currently held by federal agencies is either not made available to researchers at all or only made available under onerous conditions. On the other hand, agencies which release public databases face new challenges in protecting the privacy of the subjects in those databases, which leads them to consider releasing fewer data or masking the data in ways that will reduce their accuracy. In this essay, we argue that the discussion has not given proper consideration to the reduced social benefits of data availability and their usability relative to the value of increased levels of privacy protection. A more balanced benefit–cost framework should be used to assess these trade-offs. We express concerns both with synthetic data methods for disclosure limitation, which will reduce the types of research that can be reliably conducted in unknown ways, and with differential privacy criteria that use what we argue is an inappropriate measure of disclosure risk. We recommend that the measure of disclosure risk used to assess all disclosure protection methods focus on what we believe is the risk that individuals should care about, that more study of the impact of differential privacy criteria and synthetic data methods on data usability for research be conducted before either is put into widespread use, and that more research be conducted on alternative methods of disclosure risk reduction that better balance benefits and costs.

Synthesizing Geocodes to Facilitate Access to Detailed Geographical Information in Large-Scale Administrative Data

Abstract We investigate whether generating synthetic data can be a viable strategy for providing access to detailed geocoding information for external researchers, without compromising the confidentiality of the units included in the database. Our work was motivated by a recent project at the Institute for Employment Research in Germany that linked exact geocodes to the Integrated Employment Biographies, a large administrative database containing several million records. We evaluate the performance of three synthesizers regarding the trade-off between preserving analytical validity and limiting disclosure risks: one synthesizer employs Dirichlet Process mixtures of products of multinomials, while the other two use different versions of Classification and Regression Trees (CART). In terms of preserving analytical validity, our proposed synthesis strategy for geocodes based on categorical CART models outperforms the other two. If the risks of the synthetic data generated by the categorical CART synthesizer are deemed too high, we demonstrate that synthesizing additional variables is the preferred strategy to address the risk-utility trade-off in practice, compared to limiting the size of the regression trees or relying on the strategy of providing geographical information only on an aggregated level. We also propose strategies for making the synthesizers scalable for large files, present analytical validity measures and disclosure risk measures for the generated data, and provide general recommendations for statistical agencies considering the synthetic data approach for disseminating detailed geographical information.

The Vulnerability of Multiplicative Noise Protection to Correlation-Attacks on Continuous Microdata

When multiplicative noises are used to protect values of a sensitive attribute in a microdata, it is frequently assumed that data intruders use the noise-multiplied value to estimate the corresponding unobservable original value of a target record. In this paper, we show that, data intruders could easily construct another estimate instead of using the noise-multiplied value to attack an original value. The new estimate, namely “correlation-attack” estimate, is obtained by exploiting the potentially high correlation between the noise-multiplied data and the original data. We provide a detailed comparison between the two estimates (noise-multiplied value and the correlation-attack estimate) by comparing the mean squared errors of the two underlying estimators, and we propose that data providers should always assess the disclosure risks from both estimators when generating noise-multiplied data. Correspondingly, we propose a disclosure risk measure which could be used by data providers for noise generating variable selection during data masking stage. A simulation study is provided to illustrate how the disclosure risk measure could help with noise generating variable selection for masking a set of original data.

A tutorial in assessing disclosure risk in microdata.

Statistical agencies are releasing statistical data to other agencies for research purposes or to inform public policy. Prior to data release, these agencies have a legal and ethical obligation to protect the confidentiality of individuals in the data. Agencies often release altered versions of the data, but there usually remains risks of disclosure. Many well-studied risk measures are available to assess risk; however, many agencies today continue to use subjective judgement, past experience, and ad hoc rules or checklists to assess disclosure risk. More recently, there has been a recognized demand for quantitative risk measures that provide a more objective criteria for data release. This tutorial provides an overview of the statistical disclosure control framework for microdata. We focus on the risk analysis stage within this framework by defining existing disclosure risk measures and how to estimate them with available software.

Bayesian nonparametric disclosure risk estimation via mixed effects log-linear models

Statistical agencies and other institutions collect data under the promise to protect the confidentiality of respondents. When releasing microdata samples, the risk that records can be identified must be assessed. To this aim, a widely adopted approach is to isolate categorical variables key to the identification and analyze multi-way contingency tables of such variables. Common disclosure risk measures focus on sample unique cells in these tables and adopt parametric log-linear models as the standard statistical tools for the problem. Such models often have to deal with large and extremely sparse tables that pose a number of challenges to risk estimation. This paper proposes to overcome these problems by studying nonparametric alternatives based on Dirichlet process random effects. The main finding is that the inclusion of such random effects allows us to reduce considerably the number of fixed effects required to achieve reliable risk estimates. This is studied on applications to real data, suggesting, in particular, that our mixed models with main effects only produce roughly equivalent estimates compared to the all two-way interactions models, and are effective in defusing potential shortcomings of traditional log-linear models. This paper adopts a fully Bayesian approach that accounts for all sources of uncertainty, including that about the population frequencies, and supplies unconditional (posterior) variances and credible intervals.

An evolutionary algorithm to enhance multivariate Post-Randomization Method (PRAM) protections

The amount of public statistical information available is growing and more accurate protection methods are needed in order to achieve data confidentiality. The Post-Randomization Method (PRAM) protection method was introduced in 1997 as a very powerful method for categorical microdata, but it is still not widely used. This method has a Markov matrix as a parameter. The main problem of the application of this method is that it is difficult to find a good Markov matrix that performs changes in the microdata file producing low loss of valuable information and low risk of disclosure of sensitive data.In this paper we present a methodology that helps us to find a matrix to perform better protections. This is achieved by using an evolutionary algorithm with integrated Information Loss and Disclosure Risk measures. Experiments using three different datasets are also presented in order to empirically evaluate the application of this technique.

Differential Privacy and Statistical Disclosure Risk Measures

We compare the disclosure risk criterion of e-differential privacy with a criterion based on probabilities that intruders uncover actual values given the released data. To do so, we generate fully ...

An evolutionary approach to enhance data privacy

Dissemination of data with sensitive information about individuals has an implicit risk of unauthorized disclosure. Perturbative masking methods propose the distortion of the original data sets before publication, tackling a difficult tradeoff between data utility (low information loss) and protection against disclosure (low disclosure risk). In this paper, we describe how information loss and disclosure risk measures can be integrated within an evolutionary algorithm to seek new and enhanced masking protections for continuous microdata. The proposed technique constitutes a hybrid approach that combines state-of-the-art protection methods with an evolutionary algorithm optimization. We also provide experimental results using three data sets in order to illustrate and empirically evaluate the application of this technique.

A Framework for Evaluating the Utility of Data Altered to Protect Confidentiality

When releasing data to the public, statistical agencies and survey organizations typically alter data values in order to protect the confidentiality of survey respondents' identities and attribute values. To select among the wide variety of data alteration methods, agencies require tools for evaluating the utility of proposed data releases. Such utility measures can be combined with disclosure risk measures to gauge risk-utility tradeoffs of competing methods. This article presents utility measures focused on differences in inferences obtained from the altered data and corresponding inferences obtained from the original data. Using both genuine and simulated data, we show how the measures can be used in a decision-theoretic formulation for evaluating disclosure limitation procedures.

Privacy in Data Mining

In this chapter we describe the main tools for privacy in data mining. We present an overview of the tools for protecting data, and then we focus on protection procedures. Information loss and disclosure risk measures are also described.

A Measure of Disclosure Risk for Microdata

SummaryProtection against disclosure is important for statistical agencies releasing microdata files from sample surveys. Simple measures of disclosure risk can provide useful evidence to support decisions about release. We propose a new measure of disclosure risk: the probability that a unique match between a microdata record and a population unit is correct. We argue that this measure has at least two advantages. First, we suggest that it may be a more realistic measure of risk than two measures that are currently used with census data. Second, we show that consistent inference (in a specified sense) may be made about this measure from sample data without strong modelling assumptions. This is a surprising finding, in its contrast with the properties of the two ‘similar’ established measures. As a result, this measure has potentially useful applications to sample surveys. In addition to obtaining a simple consistent predictor of the measure, we propose a simple variance estimator and show that it is consistent. We also consider the extension of inference to allow for certain complex sampling schemes. We present a numerical study based on 1991 census data for about 450 000 enumerated individuals in one area of Great Britain. We show that the theoretical results on the properties of the point predictor of the measure of risk and its variance estimator hold to a good approximation for these data.