Finishing the Picture: Problems With Public Reporting of Clinical Trials

Abstract

When the Food and Drug Administration Amendments Act (FDAAA) was signed into law in September 2007, the legislation mandated registration and reporting of primary and secondary outcomes of all non–phase I trials of US Food and Drug Administration-regulated drugs to the ClinicalTrials.gov Web site for public review regardless of publication status. This mandate was in response to ethical and scientific concerns regarding socalled incomplete pictures of drugs that follow from selective reporting of study results. The consequences of selective reporting are significant; only after litigation was filed against Merck in 2004 were phase III clinical trial data disseminated that revealed an association with rofecoxib (Vioxx, Merck, Whitehouse Station, NJ) and increased cardiovascular risk. This is far from the exceptional case of underreporting (references 9 and 15-18 in the article by Nguyen et al). Nguyen et al in the article that accompanies this editorial describe an incomplete picture of cancer drug reporting that has potentially profound effects on patient management and counseling, as timely and complete dissemination of results can refine patient treatments, outcomes, and safety. They found that almost one half of trials had no publically available results at 3 years after completion. In addition, compliance with the FDAAA requirement of outcome reporting within 12 months of study completion was even poorer, with only 13% of trials posting results within the legislated timeframe. Contrary to the notion that the pharmaceutical industry might have the most to lose from clinical studies which did report negative results, compliance with the FDAAA legislation was highest in trials with industry primary funding. This parallels the unexpected finding reported previously; namely, that pharmacoeconomic findings reported in pharmaceutical industry–funded reports were of higher quality than those findings reported by researchers who lacked pharmaceutical industry support. While issues around publication review and revision may delay availability of study results, no such barrier exists around publication of data on ClinicalTrials.gov. It is not known if the selective incomplete reporting reflected differentials in studies with statistically nonsignificant primary or secondary outcomes or if the delayed studies were those where US Food and Drug Administration registration was being pursued and corporate interests mitigated against early dissemination. The consequences of nonreporting study results—in particular, those from clinical trials with statistically nonsignificant findings or where significant toxicities are identified which are more likely to remain unpublished—have far reaching effects. Meta-analyses of both safety and efficacy outcomes may produce biased results due to publication bias, as nonpublication of studies with nonsignificant findings typically leads to overestimation of treatment effect by the inclusion of published (and therefore more likely positive) studies and underestimation of safety effects by omission of nonpublished safety findings. As over two thirds of cancer trials included in this study were not randomized controlled trials, their inclusion or exclusion in metaanalyses likely make up the highest-quality evidence provided on that particular agent. Limitations of the study are inherent in its design. Nguyen et al chose to use publication in PubMed as an indication of journal publication; while this index is an extremely comprehensive biomedical bibliographic database, it does not index 1,800 journals found in Embase, which generally focuses more on the pharmacologic literature. In addition, the authors elected to search PubMed by the Number Clinical Trial identification to link study to publication. Although Medline began to attempt indexing publications to Number Clinical Trial numbers in 2005, the process was not uniformly adopted for several years. For the older studies registered before the passage of the FDAAA, this may potentially have created difficulty in identifying associated publications. In addition, using MEDLINE search methodology, as the authors did, has been demonstrated to miss many randomized clinical trials due to errors in indexing. How do we improve the likelihood that the complete picture of a drug will be disseminated in the future? First, while nonreporting of trial results is currently punishable by penalty under the FDAAA, individuals who conduct studies supported by federal funds should be required to report trial results to ClinicalTrials.gov as required by FDAAA as a condition of receiving continued grant support. Similarly, the US Food and Drug Administration should monitor reporting rates of studies supported by pharmaceutical manufacturers to ClinicalTrials.gov and consider regulatory actions if these trial results are not submitted to the ClinicalTrials.gov Web site. Thirdly, the proposed creation of an independent, external organization that provides independent and impartial data analysis may be helpful here. Finally, study investigators must JOURNAL OF CLINICAL ONCOLOGY E D I T O R I A L VOLUME 31 NUMBER 24 AUGUST 2