New Study Finds Publication Bias Among Trials Submitted to the US Food and Drug Administration

Abstract

One fourth of all drug trials submitted in support of new drug applications (NDAs) to the US Food and Drug Administration (FDA) remain unpublished 5 years after the fact, according to new research published in the open access journal PLoS Medicine. Among published trials, unexplained discrepancies between the FDA submissions and their corresponding publications—for example, the addition or deletion of outcomes, changes in the statistical significance of reported outcomes, and changes in overall trial conclusions—tended toward more favorable presentations of the drugs in the medical literature available to health care professionals. Lisa A. Bero, PhD, and colleagues from the University of California, San Francisco, reviewed the publication status of all 164 efficacy trials carried out in support of the 33 NDAs for new molecular entities approved by the FDA in 2001–2002, and compared information from the FDA reviews with published journal articles. Seventy-eight percent of the trials were published. In a multivariate model, trials with favorable primary outcomes (odds ratio [OR], 4.7; 95% confidence interval [CI], 1.33–17.1; P = .018) and active controls (OR, 3.4; 95% CI, 1.02–11.2; P = .047) were more likely to be published. Forty-one primary outcomes from the NDAs were omitted from the papers. Papers included 155 outcomes that were in the NDAs, 15 additional outcomes that favored the test drug, and 2 other neutral or unknown additional outcomes. Excluding outcomes with unknown significance, 43 outcomes in the NDAs did not favor the NDA drug. Of these, 20 (47%) were not included in the papers. The statistical significance of 5 (22%) of the remaining 23 outcomes changed between the NDA and the publication, with 4 changing to favor the test drug in the paper (P = .38). Excluding unknowns, 99 conclusions were provided in both NDAs and publications, 9 conclusions (9%) changed from the FDA review of the NDA to the paper, and all 9 did so in favor of the test drug (100%; 95% CI, 72%–100%; P = .0039). Given their findings, the authors conclude that “The information that is readily available in the scientific literature to health care professionals is incomplete and potentially biased.” In a commentary on the research, Dr An–Wen Chan of the Mayo Clinic in Rochester, Minnesota (uninvolved in the study), says this new research makes an important contribution to the growing body of evidence that clinical trial literature is skewed toward reporting favorable results. “Biased reporting of results from NDA trials is particularly concerning because these journal articles are the only peer reviewed source of information on recently approved drugs for health care providers, who will have had limited clinical experience with these new treatments,” Chan says. “There are also substantial cost implications if the efficacy is overestimated and the drugs overused.” See: “Reporting bias in drug trials submitted to the Food and Drug Administration: A review of publication and presentation. PLoS Med 5:e217 or http://medicine.plosjournals.org/perlserv/?request=get-document&doi=10.1371/journal.pmed.0050217.