Completeness of Reporting of Patient-Relevant Clinical Trial Outcomes: Comparison of Unpublished Clinical Study Reports with Publicly Available Data

Ulrich Grouven,Beate Wieseler,Davina Ghersi,Natalia Wolfram,Natalie McGauran,Michaela F. Kerekes,Marloes Kamphuis,Volker Vervölgyi,Petra Kohlepp

doi:10.1371/journal.pmed.1001526

Ulrich Grouven, Beate Wieseler + Show 7 more

Open Access

https://doi.org/10.1371/journal.pmed.1001526

Copy DOI

Abstract

Access to unpublished clinical study reports (CSRs) is currently being discussed as a means to allow unbiased evaluation of clinical research. The Institute for Quality and Efficiency in Health Care (IQWiG) routinely requests CSRs from manufacturers for its drug assessments. Our objective was to determine the information gain from CSRs compared to publicly available sources (journal publications and registry reports) for patient-relevant outcomes included in IQWiG health technology assessments (HTAs) of drugs. We used a sample of 101 trials with full CSRs received for 16 HTAs of drugs completed by IQWiG between 15 January 2006 and 14 February 2011, and analyzed the CSRs and the publicly available sources of these trials. For each document type we assessed the completeness of information on all patient-relevant outcomes included in the HTAs (benefit outcomes, e.g., mortality, symptoms, and health-related quality of life; harm outcomes, e.g., adverse events). We dichotomized the outcomes as "completely reported" or "incompletely reported." For each document type, we calculated the proportion of outcomes with complete information per outcome category and overall. We analyzed 101 trials with CSRs; 86 had at least one publicly available source, 65 at least one journal publication, and 50 a registry report. The trials included 1,080 patient-relevant outcomes. The CSRs provided complete information on a considerably higher proportion of outcomes (86%) than the combined publicly available sources (39%). With the exception of health-related quality of life (57%), CSRs provided complete information on 78% to 100% of the various benefit outcomes (combined publicly available sources: 20% to 53%). CSRs also provided considerably more information on harms. The differences in completeness of information for patient-relevant outcomes between CSRs and journal publications or registry reports (or a combination of both) were statistically significant for all types of outcomes. The main limitation of our study is that our sample is not representative because only CSRs provided voluntarily by pharmaceutical companies upon request could be assessed. In addition, the sample covered only a limited number of therapeutic areas and was restricted to randomized controlled trials investigating drugs. In contrast to CSRs, publicly available sources provide insufficient information on patient-relevant outcomes of clinical trials. CSRs should therefore be made publicly available. Please see later in the article for the Editors' Summary.

Highlights

Publication bias and outcome reporting bias pose a substantial threat to the validity of clinical research findings and to informed decision-making in health care [1,2]
The value of additional information from clinical study report (CSR) in drug assessment has been shown in the cases of the antiviral oseltamivir (Tamiflu) and the antidepressant reboxetine, in which conclusions on these drugs based on published evidence alone were challenged and in part even reversed by unpublished information from CSRs [12,13]
We included all health technology assessment (HTA) of drugs finalized by IQWiG between 15 January 2006 and 14 February 2011, which—besides a systematic search for journal publications—contained a systematic search for registry reports as part of the information retrieval process

Summary

Introduction

Publication bias and outcome reporting bias pose a substantial threat to the validity of clinical research findings and to informed decision-making in health care [1,2]. Various types of formats exist for reporting clinical trials of drugs: journal publications and reports from trial registries and results databases—hereafter referred to as ‘‘registry reports’’— make summaries of trials publicly available (e.g., to clinicians and authors of systematic reviews). These publicly available formats currently represent the main information source for clinical and health policy decision-making. Trial outcomes that support the use of a new treatment are more likely to be published than those that do not support its use (outcome reporting bias) Both types of bias pose a substantial threat to informed medical decision-making

Objectives

Methods

Results

Discussion

Conclusion