Privacy-preserving Record Linkage Research Articles

Proposed Framework for Adopting Privacy-Preserving Record Linkage for Public Health Action.

To propose a framework for adoption of privacy-preserving record linkage (PPRL) for public health applications. Twelve interviews with subject matter experts (SMEs) were conducted virtually and coded using an inductive approach. A collaborative session was conducted with SMEs to identify key steps in the PPRL project lifecycle which informed development of a PPRL implementation checklist. This framework has 2 decision-making levels: the organization level and the project or program level. Organization-level considerations include PPRL governance, the optimal choice among approved PPRL solutions, the need for longitudinal linkages, the potential issue of vendor lock-in, and costs. Program-level considerations include characteristics of the PPRL use case, linkage quality and accuracy, data privacy and use, security thresholds, compatibility with data owners' data architecture, and trade-offs between open-source and commercial PPRL solutions. A PPRL implementation checklist was developed to guide public health practitioners considering PPRL for data linkage. The framework may be considered by public health entities to guide adoption and implementation of PPRL in public health research and surveillance. Public health experts may refer to this framework and the PPRL implementation checklist when determining the appropriateness of PPRL for specific use cases and implementation planning.

Bloom Encodings in DGA Detection: Improving Machine Learning Privacy by Building on Privacy-Preserving Record Linkage

Bloom Encodings in DGA Detection: Improving Machine Learning Privacy by Building on Privacy-Preserving Record Linkage

Improving Cardiac Insights: Harnessing Privacy-Preserving Record Linkage (PPRL) to Obtain, Link, and Enhance Data for a Healthier Australia

ObjectiveGeneral Practices (GPs) collect vital information on health and disease in Australia, however much of this sensitive data is difficult to share for linkage purposes due to confidentiality and privacy concerns. PPRL techniques have presented an opportunity to work with GP’s, and better understand health conditions such as negative cardiac outcomes in Australia. ApproachWe utilised two algorithms, GHRANITE and Bloom Filters, to hash 2.5 million records from over 100 GPs. Both approaches utilised the same basic principles: data custodians pass identifying data through an irreversible hashing algorithm, and generate a unique key for each record. The hashing is done at the practice level ensuring that original identifying patient information never leaves GP's premises. Hashed data can be sent elsewhere, and if the same algorithm is used on a different set of data, linkage can be performed on both sets to identify individuals without exposing identifying information. ResultsThe PPRL process involved encoding over 150 million records from diverse datasets, encompassing hospitals, housing, and other sources. Initial results indicated 70.90% linkage utilising GHRANITE, and 84.10% linkage utilising Bloom Filters. Moving forward, the goal is to provide the linked data back to GPs, enabling them to identify high-risk patients and implement targeted interventions or additional measures. ConclusionThrough the successful implementation of PPRL, the project has made strides in overcoming data sharing barriers while safeguarding confidentiality. By harnessing innovative algorithms, the initiative has paved the way for more insights into cardiac outcomes and the potential for proactive healthcare interventions.

Linking Data in a Shared Service Environment

Many federal agencies in the U.S. have begun exploring the use of privacy enhancing technologies, such as privacy preserving record linkage (PPRL), to integrate data from disparate sources to support evidence building and high-quality research. Recent legislation, including the Foundations for Evidence-Based Policymaking Act of 2018 and the CHIPS and Science Act of 2022, and recommendations from the Advisory Committee on Data for Evidence Building support the establishment of a shared service environment, namely the National Secure Data Service (NSDS), to facilitate this work. This talk will provide an overview of two NSDS Demonstration projects that utilize PPRL to integrate data covered by different legal provisions. The first project aims to link data between two federal statistical agencies and the objective of the second is to link data from a federal statistical agency and its parent agency. The talk will focus on the development of the required data sharing agreements, legal provisions that needed to be considered, security and privacy measures that had to be addressed, the software and IT infrastructure required, and the distinct PPRL tools utilized for each project. The talk will conclude with a presentation of preliminary linkage results, lessons learned, and a summary of how these projects will inform potential future data sharing efforts between and within agencies through the establishment of a future NSDS.

A single parameter method for secure privacy preserving record linkage

ObjectivesData linkage is the process of matching records that refer to the same entities (often people) across databases. In applications such as health research or government services, the databases to be linked are often sensitive and cannot be shared between organisations. Privacy-preserving record linkage (PPRL) aims to overcome this challenge by facilitating the comparison of encoded or encrypted records without having to share sensitive data. Most existing PPRL techniques are based on heuristics and they have limitations in the privacy protection they offer, such as being vulnerable to certain cryptanalysis attacks. Furthermore, existing PPRL methods have multiple parameters, which, if not set properly by the user, can result in sub-optimal linkage quality and reduced privacy protection. ApproachWe present a novel PPRL method that uses random reference q-gram sets to generate bit-arrays that represent sensitive values. Our method has a single parameter to be set by the user that trades scalability with linkage quality and privacy protection. All other parameters are either data-driven or have strong bounds based on this user parameter. ResultsWe conceptually analyse our method and conduct experiments on multiple databases. The results demonstrate that our method provides high linkage quality and strong privacy protection while being scalable to link very large databases. ConclusionOur novel PPRL method provides high linkage quality, scalability, and improved privacy protection compared to existing PPRL methods such as Bloom filter encoding. A major advantage of our method is that it requires a single parameter to be set by the user.

Reporting on the establishment of a Privacy Preserving Record Linkage to Facilitate an Ongoing Crosswalk Between Research and Health Administrative Databases

IntroductionThe Ontario Brain Institute (OBI) is a provincially funded, not-for-profit organization that accelerates discovery and innovation, benefiting both patients and the economy (Stuss, 2014). OBI has established a large-scale neuroinformatics platform - Brain-CODE - to support the collection, storage, federation, sharing, and analysis of different data types across several brain disorders (Behan et al., 2023; Vaccarino et al., 2018). A privacy preserving record linkage protocol was developed to allow for the linkage of research data at Brain-CODE with health administrative data holdings at the Institute for Clinical Evaluative Sciences (ICES) (Gee et al., 2018). Objective and ApproachThe methodology related to an ongoing crosswalk linkage between OBI and ICES, to allow for more seamless integration between the respective data holdings, has been previously presented (Behan et al., 2020). This methodology has since been operationalized leading to the establishment of an ongoing crosswalk linkage that is updated on an annual basis. ResultsSince the initial development of this ongoing crosswalk, two updates have been successfully completed leading to the linkage of over 7,000 study participants between the two platforms. This has led to a more efficient utilization of human and computational resources, compared to earlier data linkage projects completed on a project-by-project basis. Analysis projects in the areas of neurodegeneration, concussion, and neurodevelopmental disorders have already leveraged this crosswalk linkage process. Conclusions/ImplicationsThe establishment of this ongoing crosswalk linkage has supported a more streamlined approach of data linkage activities between OBI and ICES allowing for enhanced neuroscience-focused research activities.

What’s in a name field?

The National Center for Health Statistics (NCHS) has a data linkage program that combines national survey data with key sources of health outcomes and health care utilization. The overall accuracy and quality of a data linkage depends on the quality of the data fields. This applies in a variety of data linkage methods, including clear text and privacy preserving record linkage. Data pre-processing and cleaning are essential to address data quality issues in most linkage tasks. Automating data cleaning prevents time-consuming manual reviews particularly when linkages involve a large number of records. For some data fields, these activities are relatively straightforward. For example, dates typically have a limited number of plausible values that make checking and cleaning relatively easy and straightforward. Unique identifiers (e.g., social security number) often conform to some set format or have restrictions on the values that would be expected. Other data fields such as first name and last name present greater challenges with respect to automating the cleaning process. For some data sources, first name and last name fields may contain non-name text that needs to be identified and removed. In this project, we examined the use of an artificial intelligence (AI) based large language model (LLM) and a simple rule-based algorithm to identify non-name text in name fields that should be removed prior to linkage. We also investigated the impact of automated name clean-up algorithms on the quality of an example linkage.

Knowledge-Driven Data Sharing Governance: Developing a data provenance model for constructing context-aware data sharing frameworks for linkage data within the All of Us Research Program.

Health data access and sharing in the U.S. have been hindered by complex privacy and security conditions, limiting research and improvement of health outcomes. This challenge is even greater in minority and sensitive populations - who would benefit most from integrative research. The All of Us Research Program is a National Institutes of Health funded initiative aiming to enhance health research through the analysis of lifestyle, environment, and genetics data from over one million participants, with a focus on communities historically underrepresented in biomedical research. The program is enhancing the utility of data by incorporating passive data streams, such as claims, electronic health records (EHRs), environmental, geocodes, mortality, and residential history. Further, through the use of national health information exchange standards, All of Us also aims to address technical challenges in linking EHRs from health information networks to deliver a more complete longitudinal participant record. However, such an initiative necessitates a thoughtful approach through data awareness, advancing data access in ways that preserve individual privacy while increasing scientific utility. Here, we describe our privacy-preserving record linkage (PPRL) and data privacy risk disclosure control strategy, emphasizing fit-for-purpose data provisioning. Our knowledge-driven data ecosystem architecture leverages a combination of PPRL, data provenance, and geocoding at multiple levels to support context-aware data sharing granular access control.

Implementing Linkage Quality Assessments in Large-Scale Privacy Preserving Infrastructure: Did we kill them a second time and why are there zombies?

The National Clinical Cohort Collaborative (N3C) uses privacy-preserving record linkage (PPRL) to formulate longitudinal patient histories and for de-duplication across disparate data including electronic health records, administrative claims, and mortality data. The N3C employs a linkage honest broker implementation model which separates the entity holding de-identified PPRL tokens, from the entity holding the substantive data payloads. As a result of this data governance structure, linkage quality assessments are necessarily distributed into 2 distinct processes: 1) Linkage Honest Broker receipt and processing of linkages, 2) Data Aggregation entity assembling data based on received linkage crosswalks. Data is released on a weekly basis. This presentation will focus on Step 2 for the Mortality data linkages where the downstream data linkage pipelines must assess linkage quality across several dimensions of quality, focused on plausibility. There are 4 types of mortality data spanning the electronic health record, government Social Security Administration death master files, third-party obituary data, and administrative claims. Each of these sources has different levels of authoritativeness and data latency, compounding the challenges of reconciling fact of death including accurate death reporting dates. To address these challenges, we implemented an automated linkage quality pipeline which generates reports that assess linkage plausibility across the various data types, classifying concordance and discordance between data sources including age, dates of death, surfacing plausible and non-plausible activities post-death. Sharing our real-world implementation will aid conference attendees in understanding how to plan for such activities when considering specific implementation models for privacy-preserving linkages.

Designing a Data Dashboard for Health Departments and Overdose Fatality Review Teams

IntroductionOverdose Fatality Review (OFR) teams within health departments rely on data from multiple isolated sources to study the opioid crisis. Recently, we linked electronic health record data with state agency data to form a regional Substance Misuse Data Commons (SMDC) using privacy-preserving record linkage. Our goal in this study was to use human factors design principles to design a data dashboard for OFR teams using the linked datasets that will overcome current barriers within OFR workflows. MethodsWe utilized NASA task load surveys, semi-structured interviews, and design sessions with end users to identify data needs for an optimal dashboard design. We assessed current workloads, data collection processes, and desired future state. We subsequently performed iterative design sessions for the generation and evaluation of low-fidelity prototypes. To overcome issues with privacy and security, we used synthetic data in a cloud-based platform to represent the SMDC for simulation. ResultsEleven OFR organizers participated. Pre-dashboard surveys on existing workflow showed high mental workload associated with data aggregation and case review, identifying a need for more accessible, comprehensive data. In our low-fidelity dashboard demo with synthetic data, iterative design adjustments were made in data visualization, storyline organizations, and theme-based data aggregation across pre-hospital and hospital data. ConclusionsWe refined the data dashboard prototype into a high-fidelity version, set for further usability and human factors evaluation. We addressed privacy and security concerns through synthetic data use while the real-world data is maintained in a HIPAA-secure Azure cloud environment with access for approved users.

Linkja: Open Source Privacy-Preserving Record Linkage Tool

ObjectiveTo introduce Linkja, an open-source, certified, Privacy-Preserving Record Linkage (PPRL) tool with the primary objective of facilitating secure and efficient record linkage across diverse datasets, while preserving individual privacy, providing a comprehensive and privacy-centric approach to program assessments. ApproachLinkja has been used to evaluate Chicago’s Flexible Housing Pool program by securely linking various datasets, including Homeless Management Information Systems (HMIS), Electronic Health Records, Jail records, Medical Examiner reports, and Public Health data. Data contributors used separate honest brokers for record linkage, and generation of salt and cryptography keys. ResultsLinkja enabled this multi-sector linkage by facilitating legal and data sharing agreements, avoided licensing fees and procurement processes for all parties, and provided a de-identified for analysis. The insights gleaned from the analysis motivated the FHP program to plan a follow-up evaluation that will add emergency serviced records from a separate City of Chicago department. The successful linkage provided a nuanced understanding of the association between stable housing and reduced healthcare utilization. ConclusionsLinkja's cryptographic certification ensures the highest standards of security during data linkage, safeguarding sensitive information. The matching rules, validated on hard-to-match populations, prove essential in maintaining accuracy and reliability in the linkage process. ImplicationsThe open-source nature of Linkja promotes transparency and collaboration in the development and enhancement of privacy-preserving record linkage techniques. This tool has broad implications for sectors dealing with sensitive data, fostering responsible data-sharing practices, and enabling secure collaborative research initiatives without compromising individual privacy.

Linkage for risk-sensitive environments: Australian examples of privacy-preserving record linkage using Bloom filters

Introduction Organisations are increasingly aware of the risks and responsibilities of handling personally identifying information (PII). These factors not only influence internal data management practices but also impact on data linkage arrangements with other parties. Some linkage environments are particularly sensitive to the release or use of PII. Advances in privacy-preserving record linkage methods such as PPRL-using-Bloom make it possible to undertake highly accurate data linkage without release or disclosure of PII. Such methods play a role in enabling data linkage in risk-sensitive environments. Objectives and ApproachWe present and describe several Australian case studies where the PPRL-using-Bloom method has been used to enable data linkage between organisations. We report on the defining elements of each case, the associated risks and solutions, as well as quality and performance issues. We also reflect on challenges and opportunities for future improvement. ResultsAustralian use cases utilising privacy preserving linkage (PPRL-using-Bloom) include projects linking state-based datasets to Commonwealth datasets, some linking primary care data to state-based secondary healthcare data, and others linking healthcare data to non-health datasets such as police and criminal justice datasets. Conclusion / ImplicationsMethods such as PPRL-using-Bloom play a critical role in enabling data linkage in highly risk-sensitive environments. However, in an ever evolving world where risks and requirements are constantly changing, linkage methodologies and technologies must remain adaptable to meet evolving demands.

Efficiency and Privacy in Record Linkage: Evaluating a Novel Blocking Technique Implemented on Cryptographic Longterm Keys

Efficient privacy-preserving record linkage (PPRL) is essential for integrating data from different providers without exposing personally identifiable information (PII). This study investigates the effectiveness of a novel blocking technique, implemented on Bloom filters (a space-efficient probabilistic data structure), to enhance efficiency and maintain privacy in a real-world evaluation. This methodology involves the utilization of Anonlink, an open-source Python-based PPRL system which generates a type of Bloom filter, Cryptographic Longterm Keys (CLK), for secure record linkage. Initially, the relevant PII fields of the two datasets undergo anonymization into CLKs. Utilizing the CLK’s property of similarity preservation, we create manageable blocks of records based on bits in common within the Bloom filters. This allows us to reduce the number of comparisons of non-matching records to improve linkage efficiency. Finally, record linkage is performed to identify potential matches within the blocked datasets. This blocking technique for CLKs is evaluated in terms of efficiency and record matching precision, aiming to determine the optimal balance between the two factors. Preliminary results indicate a significant reduction in computational burden, with recall minimally affected. Moreover, the implemented blocking technique poses no additional risks of privacy breaches. Preliminary evaluation of the blocking technique shows a promising avenue for secure and efficient data integration, especially in datasets with PII, warranting further investigation for validation and wider application.

A Hybrid-Approach for Privacy Preserving Record Linkage - A Case Study from Germany.

Over the last decade, the exponential growth in patient data volume and velocity has transformed it into a valuable resource for researchers. Yet, accessing comprehensive, unique patient data sets remains a challenge, particularly when individuals have received treatments across various practices and hospitals. Traditional record linkage methods fall short in adequately protecting patient privacy in these scenarios. Privacy Preserving Record Linkage (PPRL) offers a solution, employing techniques such as data cryptographic methods to identify common patients occurring in multiple datasets, while maintaining the privacy of other patients. This paper proposes an investigation into combined approaches of two common German PPRL tools, namely E-PIX and MainSEL. Each tool, while aiming for 'privacy preservation', employs distinct methods that offer unique advantages and drawbacks. Our research aims to explore these in a combined approach to leverage their respective strengths and mitigate their limitations. We anticipate that this synergistic approach will not only enhance data privacy but also allow for easier synchronisation of research data. This study is particularly pertinent in light of evolving privacy regulations and the increasing complexity of healthcare data management. By advancing PPRL methodologies, we aim to contribute to more robust, privacy-compliant data analysis practices in healthcare research.

Privacy-Preserving Linkage of Distributed Pseudonymised Datasets in a Virtual European Rare Disease Platform.

Secondary use of data for research purposes is especially important in rare diseases (RD), since, per definition, data are sparse. The European Joint Programme on Rare Diseases (EJP RD) aims at developing an RD infrastructure which supports the secondary use of data. Significant amounts of RD data are a) distributed and b) available only in pseudonymised format. Privacy-Preserving Record Linkage (PPRL) concerns the linking of such distributed datasets without disclosing the participant's identities. We present a concept for linking a PPRL Service to the EJP RD Virtual Platform (VP). Level 1 (resource discovery) connection is provided by running an FDP within the PPRL Service. On Level 2 (data discoverability), the PPRL Service can represent both, an individual and a catalog endpoint. Our solution can count patients in PPRL-supporting resources, count duplicates only once, and count only patients registered to multiple resources. Currently, we are preparing the deployment within the EJP RD VP.

Use Cases Requiring Privacy-Preserving Record Linkage in Paediatric Oncology.

Large datasets in paediatric oncology are inherently rare. Therefore, it is paramount to fully exploit all available data, which are distributed over several resources, including biomaterials, images, clinical trials, and registries. With privacy-preserving record linkage (PPRL), personalised or pseudonymised datasets can be merged, without disclosing the patients' identities. Although PPRL is implemented in various settings, use case descriptions are currently fragmented and incomplete. The present paper provides a comprehensive overview of current and future use cases for PPRL in paediatric oncology. We analysed the literature, projects, and trial protocols, identified use cases along a hypothetical patient journey, and discussed use cases with paediatric oncology experts. To structure PPRL use cases, we defined six key dimensions: distributed personalised records, pseudonymisation, distributed pseudonymised records, record linkage, linked data, and data analysis. Selected use cases were described (a) per dimension and (b) on a multi-dimensional level. While focusing on paediatric oncology, most aspects are also applicable to other (particularly rare) diseases. We conclude that PPRL is a key concept in paediatric oncology. Therefore, PPRL strategies should already be considered when starting research projects, to avoid distributed data silos, to maximise the knowledge derived from collected data, and, ultimately, to improve outcomes for children with cancer.

Enhanced Multi-Party Privacy-Preserving Record Linkage Using Trusted Execution Environments

Enhanced Multi-Party Privacy-Preserving Record Linkage Using Trusted Execution Environments

Privacy preserving record linkage for public health action: opportunities and challenges.

To understand the landscape of privacy preserving record linkage (PPRL) applications in public health, assess estimates of PPRL accuracy and privacy, and evaluate factors for PPRL adoption. A literature scan examined the accuracy, data privacy, and scalability of PPRL in public health. Twelve interviews with subject matter experts were conducted and coded using an inductive approach to identify factors related to PPRL adoption. PPRL has a high level of linkage quality and accuracy. PPRL linkage quality was comparable to that of clear text linkage methods (requiring direct personally identifiable information [PII]) for linkage across various settings and research questions. Accuracy of PPRL depended on several components, such as PPRL technique, and the proportion of missingness and errors in underlying data. Strategies to increase adoption include increasing understanding of PPRL, improving data owner buy-in, establishing governance structure and oversight, and developing a public health implementation strategy for PPRL. PPRL protects privacy by eliminating the need to share PII for linkage, but the accuracy and linkage quality depend on factors including the choice of PPRL technique and specific PII used to create encrypted identifiers. Large-scale implementations of PPRL linking millions of observations-including PCORnet, National Institutes for Health N3C, and the Centers for Disease Control and Prevention COVID-19 project have demonstrated the scalability of PPRL for public health applications. Applications of PPRL in public health have demonstrated their value for the public health community. Although gaps must be addressed before wide implementation, PPRL is a promising solution to data linkage challenges faced by the public health ecosystem.

A Parallel Multi-Party Privacy-Preserving Record Linkage Method Based on a Consortium Blockchain

Privacy-preserving record linkage (PPRL) is the process of linking records from various data sources, ensuring that matching records for the same entity are shared among parties while not disclosing other sensitive data. However, most existing PPRL approaches currently rely on third parties for linking, posing risks of malicious tampering and privacy breaches, making it difficult to ensure the security of the linkage. Therefore, we propose a parallel multi-party PPRL method based on consortium blockchain technology which can effectively address the issue of semi-trusted third-party validation, auditing all parties involved in the PPRL process for potential malicious tampering or attacks. To improve the efficiency and security of consensus within a consortium blockchain, we propose a practical Byzantine fault tolerance consensus algorithm based on matching efficiency. Additionally, we have incorporated homomorphic encryption into Bloom filter encoding to enhance its security. To optimize computational efficiency, we have adopted the MapReduce model for parallel encryption and utilized a binary storage tree as the data structure for similarity computation. The experimental results show that our method can effectively ensure data security while also exhibiting relatively high linkage quality and scalability.

A Multi-Party Privacy-Preserving Record Linkage Method Based on Secondary Encoding

With the advent of the big data era, data security and sharing have become the core elements of new-era data processing. Privacy-preserving record linkage (PPRL), as a method capable of accurately and securely matching and sharing the same entity across multiple data sources, is receiving increasing attention. Among the existing research methods, although PPRL methods based on Bloom Filter encoding excel in computational efficiency, they are susceptible to privacy attacks, and the security risks they face cannot be ignored. To balance the contradiction between security and computational efficiency, we propose a multi-party PPRL method based on secondary encoding. This method, based on Bloom Filter encoding, generates secondary encoding according to well-designed encoding rules and utilizes the proposed linking rules for secure matching. Owing to its excellent encoding and linking rules, this method successfully addresses the balance between security and computational efficiency. The experimental results clearly show that, in comparison to the original Bloom Filter encoding, this method has nearly equivalent computational efficiency and linkage quality. The proposed rules can effectively prevent the re-identification problem in Bloom Filter encoding (proven). Compared to existing privacy-preserving record linkage methods, this method shows higher security, making it more suitable for various practical application scenarios. The introduction of this method is of great significance for promoting the widespread application of privacy-preserving record linkage technology.