Disclosure Limitation Research Articles

Product Use Information and the Limits of Voluntary Disclosure

Concern about asymmetric information in markets for consumer goods and services has focused on product attribute information. We highlight the importance of another category of information - product use information. In important markets, sellers have better information about how a consumer will use their product or service than the consumer herself. Moreover, we show that the classic unraveling results do not extend to product use information, and thus sellers are less likely to voluntarily disclose this type of information. Our findings have important policy implications: While most disclosure mandates target product attribute information, our analysis suggests that mandating disclosure of product use information may be more important. Indeed, policymakers are beginning to recognize the importance of product use disclosures.

Editorial: In This Issue

In this issue of the Journal of Privacy and Confidentiality , we continue our focus on the technical aspects of research methodologies and activities in the areas of privacy, confidentiality, and disclosure limitation through a trio of articles, but we begin by reproducing related materials from some other sources on broader aspects of the topics.

Assessing the protection provided by misclassification-based disclosure limitation methods for survey microdata

Government statistical agencies often apply statistical disclosure limitation techniques to survey microdata to protect the confidentiality of respondents. There is a need for valid and practical ways to assess the protection provided. This paper develops some simple methods for disclosure limitation techniques which perturb the values of categorical identifying variables. The methods are applied in numerical experiments based upon census data from the United Kingdom which are subject to two perturbation techniques: data swapping (random and targeted) and the post randomization method. Some simplifying approximations to the measure of risk are found to work well in capturing the impacts of these techniques. These approximations provide simple extensions of existing risk assessment methods based upon Poisson log-linear models. A numerical experiment is also undertaken to assess the impact of multivariate misclassification with an increasing number of identifying variables. It is found that the misclassification dominates the usual monotone increasing relationship between this number and risk so that the risk eventually declines, implying less sensitivity of risk to choice of identifying variables. The methods developed in this paper may also be used to obtain more realistic assessments of risk which take account of the kinds of measurement and other nonsampling errors commonly arising in surveys.

A multiple imputation approach to disclosure limitation for high‐age individuals in longitudinal studies

Disclosure limitation is an important consideration in the release of public use data sets. It is particularly challenging for longitudinal data sets, since information about an individual accumulates with repeated measures over time. Research on disclosure limitation methods for longitudinal data has been very limited. We consider here problems created by high ages in cohort studies. Because of the risk of disclosure, ages of very old respondents can often not be released; in particular, this is a specific stipulation of the Health Insurance Portability and Accountability Act (HIPAA) for the release of health data for individuals. Top-coding of individuals beyond a certain age is a standard way of dealing with this issue, and it may be adequate for cross-sectional data, when a modest number of cases are affected. However, this approach leads to serious loss of information in longitudinal studies when individuals have been followed for many years. We propose and evaluate an alternative to top-coding for this situation based on multiple imputation (MI). This MI method is applied to a survival analysis of simulated data, and data from the Charleston Heart Study (CHS), and is shown to work well in preserving the relationship between hazard and covariates.

Masking methods that preserve positivity constraints in microdata

Statistical agencies have conflicting obligations to protect confidential information provided by respondents to surveys or censuses and to make data available for research and planning activities. When the microdata themselves are to be released, in order to achieve these conflicting objectives, statistical agencies apply statistical disclosure limitation (SDL) methods to the data, such as noise addition, swapping or microaggregation. Some of these methods do not preserve important structure and constraints in the data, such as positivity of some attributes or inequality constraints between attributes. Failure to preserve constraints is not only problematic in terms of data utility, but also may increase disclosure risk. In this paper, we describe a method for SDL that preserves both positivity of attributes and the mean vector and covariance matrix of the original data. The basis of the method is to apply multiplicative noise with the proper, data-dependent covariance structure.

Letter from the Editors

“The present moment seems a very appropriate one to launch a new journal on Algebraic Statistics”Fabrizio Catanese, Editor of the Journal of Algebraic GeometryMany classical statistical models, such as Gaussian models from multivariate statistics and models for discrete random variables have algebraic structure in their parameter spaces. Algebraic statistics is a young, vibrant, quickly growing, and active discipline focused on the applications of algebraic geometry and its computational tools in statistical models. It has been applied in disclosure limitation, design of experiments, hypothesis testing for log-linear models, maximum likelihood estimation, and approximations to Bayesian integrals, and in the last decade it finds its applications in several other areas, such as computational biology, chemical networks, and finance. For example, the algebraic approach to phylogenetics is one of the subdisciplines of algebraic statistics, which has seen many applications in computational biology. Besides phylogenetics, this approach has seen biological applications in inferring the progression to drug resistance in HIV, determining the parametric behaviors of sequence alignments, and studying the geometry of fitness landscapes.Example applications include mathematical modeling, operations research, model identification, system analysis and design, system verification, and system synthesis. Application areas include systems biology, genomics, proteomics, phylogenetics and evolutionary biology, chemical networks, finance, engineering, to name a few. Areas in algebra include polynomial methods, commutative algebra and algebraic geometry, group theory, string rewriting, automated reasoning, automata theory, formal languages, combinatorics, graph theory, and artificial intelligence, among others.We, the editorial staff, believe this is an important new field, which has the potential to provide immeasurable benefits to all of mankind. However, as with all fields, a common intersection forum is needed. One in which researchers can share their visions, challenges, and victories. A forum which makes it easy for the neophytes – as well as the simply curious – to explore the field, and the experts to discuss how best to help inspire and teach them. Stemming from the mission and great success of the European Journal of Pure and Applied (EJPAM), we believe we can serve this research community by providing this forum, Journal of Algebraic Statistics (JAlgStat).It is our great pleasure to invite original research papers and critical survey articles for submission to the Journal of Algebraic Statistics. The JAlgStat is intended to foster communication and interaction between researchers who apply algebraic methods and symbolic computation to various problems in statistics and the life sciences. Algebraists, biologists, statisticians, computer scientists, and scientists who are interested in algebraic methods to solve their problems need to develop a common language to discuss problems of common interest. While several of mathematicians, statisticians, computer scientists, and life scientists involved in this research area already have close collaborations with researchers in other fields, many others do not. Journal of Algebraic Statistics provides a much needed opportunity to further these connections. We will exert every effort to handle submissions swiftly and fairly to provide rapid reviews, taking advantage of the sophisticated electronic infrastructure and facilities provided by the Article Management System.We all wish Journal of Algebraic Statistics –the first journal in the field- every success in the contemporary scientific world. Please join us in thanking those who have chosen to honor our inaugural issue with their research; with all your help, we can make this a truly valuable research forum.We are also especially grateful to our all Advisory and Editorial Board members, editorial message senders and all authors who have contributed to this inaugural volume.

Vulnerability of Complementary Cell Suppression to Intruder Attack

Complementary cell suppression was the first and remains a popular method for disclosure limitation of magnitude data such as economic censuses data. We show that, when not solved in a rigorous mathematical way, suppression can fail to protect data, sometimes fatally. When solved properly as a mathematical programming problem, suppression is guaranteed to meet certain conditions related to protecting individual data, but we demonstrate that other vulnerabilities exist. Suppression sacrifices both confidential and nonconfidential data, forcing potentially significant degradation in data quality and usability. These effects are often compounded because mathematical relationships induced by suppression tend to produce "over-protected" solutions. To mitigate these effects, it has been suggested that the data releaser provide exact interval estimates of suppressed cell values. We demonstrate for two standard data sensitivity measures that, even when safe, exact intervals further threaten data security, in some situations completely.

The Relevance or Irrelevance of Weights for Confidentiality and Statistical Analyses

Sample survey weights represent a standard part of the survey statistician's repertoire, but they remain a mystery for many of those who work with survey data. They also pose a potential challenge for confidentiality protection. Much has been written about how to use weights in statistical analyses since the basic idea of weighted estimates|weighting units inversely proportional to their probability of selection|emerged from the classic paper by Horvitz and Thompson. Very little has been written about the effect of releasing survey weights on the confidentiality of survey data. The two topics are inextricably intertwined. This paper discusses both, largely from the model-based perspective, and explains what is claimed and/or known about the issue of confidentiality protection. The paper also provides a prescription for ways to deal with weights both for analysis and for disclosure limitation.

Preserving data utility via BART

When preparing data for public release, information organizations face the challenge of preserving the quality of data while protecting the confidentiality of both data subjects and sensitive data attributes. Without knowing what type of analyses will be conducted by data users, it is often hard to alter data without sacrificing data utility. In this paper, we propose a new approach to mitigate this difficulty, which entails using Bayesian additive regression trees (BART), in connection with existing methods for statistical disclosure limitation, to help preserve data utility while meeting confidentiality requirements. We illustrate the performance of our method through both simulation and a data example. The method works well when the targeted relationship underlying the original data is not weak, and the performance appears to be robust to the intensity of alteration.

Disclosure of Financial Relationships to Participants in Clinical Research

The authors review the potential goals of disclosing investigators' financial relationships to research participants and discuss the limitations of disclosure in achieving these goals. They note that equity interests are especially problematic and should be limited or avoided rather than simply disclosed.

Using Multiple Imputation to Integrate and Disseminate Confidential Microdata

Summary In data integration contexts, two statistical agencies seek to merge their separate databases into one file. The agencies also may seek to disseminate data to the public based on the integrated file. These goals may be complicated by the agencies' need to protect the confidentiality of database subjects, which could be at risk during the integration or dissemination stage. This article proposes several approaches based on multiple imputation for disclosure limitation, usually called synthetic data, that could be used to facilitate data integration and dissemination while protecting data confidentiality. It reviews existing methods for obtaining inferences from synthetic data and points out where new methods are needed to implement the data integration proposals.

Data masking for disclosure limitation

AbstractProtecting confidentiality is essential to the functioning of systems for collecting and disseminating data on individuals and enterprises that are necessary for evidence‐based public policy formulation. Deidentification of records, defined as removing obvious identifiers such as name and address, is not sufficient to protect confidentiality. Microdata have characteristics that lead to increased disclosure risk, such as existence of identification files, geographical detail, outliers, many/detailed attribute variables, or longitudinal or panel structure in the data. Data stewardship organizations can lower disclosure risk through disclosure limitation methods and through the construction of synthetic data. Both record and attribute suppression can be represented by matrix masks, as can perturbation through noise addition, and data swapping. Also sampling and aggregation have matrix mask representations. Distinct from masking methods, synthetic data construction considers the microdata to be a realization of some statistical model. It then replaces the true microdata with samples generated according to the model. The released data consist of records of individual synthetic units rather than records for the actual units. The organization must recognize uncertainty in both model form and values of model parameters. This argues for the relevance of hierarchical and mixture models to generate the synthetic data. Synthetic data has an advantage as a disclosure limitation method over masked data because it is easier for the user to analyze. Copyright © 2009 Wiley Periodicals, Inc., A Wiley CompanyThis article is categorized under: Data: Types and Structure > Data Preparation and Processing

Distribution-preserving statistical disclosure limitation

One approach to limiting disclosure risk in public-use microdata is to release multiply-imputed, partially synthetic data sets. These are data on actual respondents, but with confidential data replaced by multiply-imputed synthetic values. A mis-specified imputation model can invalidate inferences based on the partially synthetic data, because the imputation model determines the distribution of synthetic values. We present a practical method to generate synthetic values when the imputer has only limited information about the true data generating process. We combine a simple imputation model (such as regression) with density-based transformations that preserve the distribution of the confidential data, up to sampling error, on specified subdomains. We demonstrate through simulations and a large scale application that our approach preserves important statistical properties of the confidential data, including higher moments, with low disclosure risk.

Integrated exact, hybrid and metaheuristic learning methods for confidentiality protection

A vital task facing government agencies and commercial organizations that report data is to represent the data in a meaningful way and simultaneously to protect the confidentiality of critical components of this data. The challenge is to organize and disseminate data in a form that prevents such critical components from being inferred by groups bent on corporate espionage, to gain competitive advantages, or having a desire to penetrate the security of the information underlying the data. Controlled tabular adjustment is a recently developed approach for protecting sensitive information by imposing a special form of statistical disclosure limitation on tabular data. The underlying model gives rise to a mixed integer linear programming problem involving both continuous and discrete (zero-one) variables. We develop stratified ordered (s-ordered) heuristics and a new meta-heuristic learning approach for solving this model, and compare their performance to previous heuristics and to an exact algorithm embodied in the state-of-the-art ILOG- CPLEX software. Our new approaches are based on partitioning the problem into its discrete and continuous components, first creating an s-ordered heuristic that reduces the number of binary variables through a grouping procedure that combines an exact mathematical programming model with constructive heuristics. To gain further advantages we then replace the mathematical programming model with an evolutionary scatter search approach that makes it possible to extend the method to large problems with over 9000 entries. Finally, we introduce a new metaheuristic learning method that significantly improves the quality of solutions obtained.

First Issue Editorial

When the founders of this Journal -- Cynthia Dwork, Stephen Fienberg and Alan Karr -- made its initial call for papers, they and we identified many constituencies that participate in the scientific analysis of privacy and confidentiality. Statisticians, particularly those working within national statistical offices, have developed the field of statistical disclosure limitation. Com- puter scientists contribute work in privacy-preserving data-mining and cryptographic analyses of privacy. Lawyers and social scientists study the role of government and regulation in the creation and protection of individual and business privacy. Health researchers struggle with the trade-off between a patient’s privacy and the contribution to science that access to inte- grated medical records might allow. Survey designers in all fields of human endeavor wrestle with methods of enticing survey cooperation under a variety of ethical and privacy guarantees. Gargantuan online services gather petabytes of data on search queries, online purchases, e-mail exchanges, and other social network interactions while pushing their computer scientists to exploit the corporate asset these data represent without damaging the companies’ ability to do future business by breaching the confidence of their client/users. And many, many data users from all of the fields listed above perform analyses that are conditioned on the privacy and confidentiality protections imposed on their work without all the means to assess the consequences of those measures on the inferences they have made.
 We are certainly not the first journal to venture into this domain. But we are the first journal to solicit actively contributions from the entire community that are aimed at multiple constituencies within that community. We think that a brief illustration of how the research questions share a common theme would provide a useful introduction to this first volume.

Global Measures of Data Utility for Microdata Masked for Disclosure Limitation

When releasing microdata to the public, data disseminators typically alter the original data to protect the confidentiality of database subjects' identities and sensitive attributes. However, such alteration negatively impacts the utility (quality) of the released data. In this paper, we present quantitative measures of data utility for masked microdata, with the aim of improving disseminators' evaluations of competing masking strategies. The measures, which are global in that they reflect similarities between the entire distributions of the original and released data, utilize empirical distribution estimation, cluster analysis, and propensity scores. We evaluate the measures using both simulated and genuine data. The results suggest that measures based on propensity score methods are the most promising for general use.

What Future for Disclosure as a Regulatory Technique? Lessons from the Global Financial Crisis and Beyond

Inadequate disclosure has been at the heart of most policy analysis of the global financial crisis. According to the inadequate disclosure critique, investors had insufficient information regarding the risks involved in structured securities, the flaws of credit ratings, and the impact of excessive executive compensation, all among the main causes of the recent financial market collapse. However, the global financial crisis has also exposed the many limits of disclosure as an effective regulatory tool in the context of financial markets. For instance, many of the risks that led to the creation of the 2008 catastrophe were often fully disclosed but the markets failed to understand what was disclosed and appreciate the implications. The reasons for this failure were product complexity and the impact of socio-psychological factors such as bounded rationality, strategic trade behaviour (herding), and cognitive biases. These findings pose a great challenge to the prevailing rational choice view of disclosure as a regulatory remedy of most market failures. At the same time, the issue of transparent financial markets dominates the global regulatory reform agenda. Accordingly, there is a clear need to devise strategies that make disclosure work under actual (not hypothetical) market conditions and redress the adverse impact of socio-psychological factors. The chapter argues that in specific contexts, such as the field of prudential regulation of banks, disclosure will only work if it is supplemented by protective regulation, e.g., business activity barriers and position limits. It also argues that only through the use of experiments, as a complement to empirical studies, policy-makers and regulators will be able to measure the actual contribution of disclosure to investor protection and devise effective disclosure strategies.

The Global Financial Crisis and the Disclosure Paradigm in European Financial Regulation: The Case for Reform

The global financial crisis has exposed the many limits of disclosure as an effective regulatory tool in financial markets. First, the famed disciplining power of the market failed to constrain disastrous risk taking by banks. Second, most of the risks that led to the creation of the 2008 catastrophe were often fully disclosed but the markets failed to understand them. In the case of banks, disclosure-based market discipline failed mainly because of the implicit government guarantee. In the case of capital markets, the reasons for disclosure's failure were product complexity and the impact of socio-psychological factors. Yet much of European Financial Regulation is based on the disclosure paradigm to remedy market failure, discipline market actors, improve investor/consumer choice, and prevent abuse. The EU needs to reexamine the role of disclosure in two contexts: prudential regulation of banks and retail investor protection. EU policy-makers should use empirical and experimental studies before any reform of the investor protection framework. Insertion of default options in a variety of financial contracts may be a necessary supplement to disclosure for retail investors. Furthermore, an independent EU financial products committee would be a better regulatory protection strategy than reliance on investor choice assisted by enhanced disclosure.

Make assurance double sure: combination of two disclosure limitation methods and estimation of general regression models

In order to guarantee confidentiality and privacy of firm-level data, statistical offices apply various disclosure limitation techniques. However, each anonymization technique has its protection limits such that the probability of disclosing the individual information for some observations is not minimized. To overcome this problem, we propose combining two separate disclosure limitation techniques, blanking and multiplication of independent noise, in order to protect the original dataset. The proposed approach yields a decrease in the probability of reidentifying/disclosing individual information and can be applied to linear and nonlinear regression models.

Verification servers: Enabling analysts to assess the quality of inferences from public use data

To protect confidentiality, statistical agencies typically alter data before releasing them to the public. Ideally, although generally not done, the agency also provides a way for secondary data analysts to assess the quality of inferences obtained with the released data. Quality measures can help secondary data analysts to identify inaccurate conclusions resulting from the disclosure limitation procedures, as well as have confidence in accurate conclusions. We propose a framework for an interactive, web-based system that analysts can query for measures of inferential quality. As we illustrate, agencies seeking to build such systems must consider the additional disclosure risks from releasing quality measures. We suggest some avenues of research on limiting these risks.