New world of big data - new challenges for evidence synthesis: Impact of data duplication on estimates generated by meta-analyses, and the development of a framework for its identification and management.

Abstract

Listen

Translate article

The aim of this study was to highlight the effects of entering duplicated or overlapping data from published studies using the same data registries into a meta-analysis, including its identification and management using a novel structured framework. Secondary analysis of data from a proportional meta-analysis of 30-day cumulative incidence of venous thromboembolic events (VTE) after metabolic and bariatric surgery was performed. Sensitivity analysis was conducted a) including all studies regardless of duplication (uncorrected sample) and b) comparing it to a corrected sample of studies. We developed a decision tree framework to identify duplicated data from prospective studies and data registries. We demonstrated that biasing from duplicated data, primarily from data registries, underestimated the incidence of VTE in the literature by 0.15% of the patient population (an erroneous difference equivalent to 22.06% of total VTE). This error persisted at 8.16% of total VTE when limiting to studies using a primarily laparoscopic approach. The decision tree framework used a comparison of the data source (country and hospital or registry), sampling timeframe (dates/years of included data) and inclusion characteristics (included procedures/diagnoses or inclusion criteria) to identify potentially duplicated data. Inter-rater reliability was excellent (κ=1.00, p<0.001), although only 17.86% of studies coded as containing data duplication were be verified by the authors while the remaining studies could not be verified. Lastly, we identified a strong lack of diversity in the geographical origins of the data from the included studies. We demonstrated that including duplicated data in a meta-analysis can result in substantially inaccurate pooled estimates. We outlined a comprehensive decision tree framework that future researchers can apply to assist with decision making when identifying and managing duplicated data, including that from data registries or other publicly accessible datasets.