PP254 Double-Counting In Evidence Synthesis Including Routinely Collected Data: Methodological and Practical Considerations

Abstract

IntroductionThe use of real-world data, as an alternative to randomized controlled trials, is becoming increasingly common in the evaluation of new health technologies. With this rise in real-world literature, such data will also enter evidence synthesis models. While it can be beneficial to utilize data from all available sources, this can introduce the problem of double-counting of participants.MethodsUsing a number of case-studies, we discuss and illustrate various issues around double-counting. These include synthesis of studies using the same database or the same subset of participants, overlapping use of intervention arms across studies and the use of registry data from the participants overlapping with those in randomized controlled trials. The implications in research are considered along with common methods used currently to overcome these issues.ResultsDouble-counting of participants in evidence synthesis can artificially inflate precision, potentially leading to inappropriate conclusions. Common methods currently used to help mitigate the impact of double-counting includes stratifying analysis to different timelines, using the most comprehensive study in the evidence synthesis model or using the study that has the largest sample size. However, in all of these cases, sensitivity analyses would need to be considered to ensure robust results.ConclusionsCurrently, there are no published guidelines on how to address the issue of double-counting. With the increased use of real-world data in evidence synthesis, double-counting has the potential to become a significant issue. Therefore, it is of significant importance that methodologies and guidelines are developed to address this.