Weighted Itemsets Error (WIE) Approach for Evaluating Generated Synthetic Patient Data

Abstract

Patient data are regarded as highly sensitive and protected information by federal, state and local policies that make it available to only those who have been given access to Protected Health Information (PHI). In many applications, the access to PHI and real patient data can be substituted with generated realistic synthetic data used instead of real patient data. While methods exist that can generate synthetic data, it is unclear how to evaluate synthetic data quality. The objective of this paper is to present investigation of a new method for statistically testing the quality of synthetic patient data. Weighted Itemsets Error (WIE) measure compares frequent itemsets in the synthetic data with expected itemsets in real data, thus allowing for evaluating cooccurrence of data items. The derived measure is tested in the context of synthetic data comprising of medical diagnoses. The results demonstrate the effects of parameters that control WIE measure, and indicate that WIE is a simple yet powerful approach for evaluating synthetic datasets.