Publication Bias In Clinical Trials Research Articles

Publication bias in clinical trials in cataract therapies: implications for evidence-based decision-making.

Evidence-based decision-making is generally based on published evidence. Therefore, if the published evidence is biased, so will the decision-making. One possible bias is the "positive-results" publication bias. This study attempts to characterize this phenomenon in cataract therapy trials. Studies were categorized as "positive" if their results were congruent with the hypothesis and "negative" if not. Secondary outcomes included the influence of funding source and differences in publication metrics between "positive" and "negative" publications. The U.S. National Library of Medicine Clinical Trials database was reviewed for cataract trials, yielding 248 trials. Trials with less than 2 treatment arms, less than 5 participants, or insufficient reporting were excluded. Data were collected on intervention, treatment arms, funding type, publication rates, citation rate, and the impact factor/H-index of journals. Of the 132 trials included, there were 69 positive and 63 negative results. Publication rate for positive results (71%) was significantly greater than negative results (17%) ( P < .01), with no significant difference in the other publication metrics. In conclusion, "negative" result trials are published less frequently, but are equally valued, if published. There are implications for evidence-based medicine with these findings.

Factors relating to nonpublication and publication bias in clinical trials in Canada: A qualitative interview study.

This study aims to understand factors contributing to nonpublication and publication bias in clinical trials in Canada. Qualitative interviews were conducted between March 2019 and April 2021 with 34 participants from the Canadian provinces of Alberta, British Columbia and Ontario, including 17 clinical trial investigators, 1 clinical research coordinator, 3 research administrators, 3 research ethics board members and 10 clinical trial participants. We conducted a thematic analysis involving coding of interview transcripts and memo-writing to identify key themes. Several factors contribute to nonpublication and publication bias in clinical trial research. A core theme was that reporting practices are shaped by incentives within the research system taht favour publication of positive over negative trials. Investigators are discouraged from reporting by experiences or perceptions of difficulty in publishing negative findings but rewarded for publishing positive findings in various ways. Trial investigators more strongly associated positive clinical trials than negative trials with opportunities for industry and nonindustry funding and with academic promotion, bonuses and recognition. Research institutions and ethics boards tended to lack well-resourced, proactive policies and practices to ensure trial findings are reported in registries or journals. Clinical trial reporting practices in Canada are shaped by incentives favouring reporting of positive over negative trials, such as funding opportunities and academic promotion, bonuses and recognition. Research institutions could help change incentives by adopting performance metrics that emphasize full reporting of results in journals or registries.

Adoption of World Health Organization Best Practices in Clinical Trial Transparency Among European Medical Research Funder Policies

Research funders can reduce research waste and publication bias by requiring their grantees to register and report clinical trials. To determine the extent to which 21 major European research funders' efforts to reduce research waste and publication bias in clinical trials meet World Health Organization (WHO) best practice benchmarks and to investigate areas for improvement. This cross-sectional study was based on 2 to 3 independent assessments of each funder's publicly available documentation and validation of results with funders during 2021. Included funders were the 21 largest nonmultilateral public and philanthropic medical research funders in Europe, with a combined budget of more than US $22 billion. Scoring of funders using an 11-item assessment tool based on WHO best practice benchmarks, grouped into 4 broad categories: trial registries, academic publication, monitoring, and sanctions. Funder references to reporting standards were captured. The primary outcome was funder adoption or nonadoption of 11 policy and monitoring measures to reduce research waste and publication bias as set out by WHO best practices. The secondary outcomes were whether and how funder policies referred to reporting standards. Outcomes were preregistered after a pilot phase that used the same outcome measures. Among 21 of the largest nonmultilateral public and philanthropic funders in Europe, some best practices were more widely adopted than others, with 14 funders (66.7%) mandating prospective trial registration and 6 funders (28.6%) requiring that trial results be made public on trial registries within 12 months of trial completion. Less than half of funders actively monitored whether trials were registered (9 funders [42.9%]) or whether results were made public (8 funders [38.1%]). Funders implemented a mean of 4 of 11 best practices in clinical trial transparency (36.4%) set out by WHO. The extent to which funders adopted WHO best practice items varied widely, ranging from 0 practices for the French Centre National de la Recherche Scientifique and the ministries of health of Germany and Italy to 10 practices (90.9%) for the UK National Institute of Health Research. Overall, 9 funders referred to reporting standards in their policies. This study found that many European medical research funder policy and monitoring measures fell short of WHO best practices. These findings suggest that funders worldwide may need to identify and address gaps in policies and processes.

Nonpublication Rates and Characteristics of Registered Randomized Clinical Trials in Digital Health: Cross-Sectional Analysis.

BackgroundClinical trials are key to advancing evidence-based medical research. The medical research literature has identified the impact of publication bias in clinical trials. Selective publication for positive outcomes or nonpublication of negative results could misdirect subsequent research and result in literature reviews leaning toward positive outcomes. Digital health trials face specific challenges, including a high attrition rate, usability issues, and insufficient formative research. These challenges may contribute to nonpublication of the trial results. To our knowledge, no study has thus far reported the nonpublication rates of digital health trials.ObjectiveThe primary research objective was to evaluate the nonpublication rate of digital health randomized clinical trials registered in ClinicalTrials.gov. Our secondary research objective was to determine whether industry funding contributes to nonpublication of digital health trials.MethodsTo identify digital health trials, a list of 47 search terms was developed through an iterative process and applied to the “Title,” “Interventions,” and “Outcome Measures” fields of registered trials with completion dates between April 1, 2010, and April 1, 2013. The search was based on the full dataset exported from the ClinlicalTrials.gov database, with 265,657 trials entries downloaded on February 10, 2018, to allow publication of studies within 5 years of trial completion. We identified publications related to the results of the trials through a comprehensive approach that included an automated and manual publication-identification process.ResultsIn total, 6717 articles matched the a priori search terms, of which 803 trials matched our latest completion date criteria. After screening, 556 trials were included in this study. We found that 150 (27%) of all included trials remained unpublished 5 years after their completion date. In bivariate analyses, we observed statistically significant differences in trial characteristics between published and unpublished trials in terms of the intervention target condition, country, trial size, trial phases, recruitment, and prospective trial registration. In multivariate analyses, differences in trial characteristics between published and unpublished trials remained statistically significant for the intervention target condition, country, trial size, trial phases, and recruitment; the odds of publication for non-US–based trials were significant, and these trials were 3.3 (95% CI 1.845-5.964) times more likely to be published than US–based trials. We observed a trend of 1.5 times higher nonpublication rates for industry-funded trials. However, the trend was not statistically significant.ConclusionsIn the domain of digital health, 27% of registered clinical trials results are unpublished, which is lower than nonpublication rates in other fields. There are substantial differences in nonpublication rates between trials funded by industry and nonindustry sponsors. Further research is required to define the determinants and reasons for nonpublication and, more importantly, to articulate the impact and risk of publication bias in the field of digital health trials.

Publication bias in meta-analyses from the Cochrane Database of Systematic Reviews.

We used a Bayesian hierarchical selection model to study publication bias in 1106 meta-analyses from the Cochrane Database of Systematic Reviews comparing treatment with either placebo or no treatment. For meta-analyses of efficacy, we estimated the ratio of the probability of including statistically significant outcomes favoring treatment to the probability of including other outcomes. For meta-analyses of safety, we estimated the ratio of the probability of including results showing no evidence of adverse effects to the probability of including results demonstrating the presence of adverse effects. In the meta-analyses of efficacy, outcomes favoring treatment had on average a 27% (95% Credible Interval (CI): 18% to 36%) higher probability to be included than other outcomes. In the meta-analyses of safety, results showing no evidence of adverse effects were on average 78% (95% CI: 51% to 113%) more likely to be included than results demonstrating that adverse effects existed. In general, the amount of over-representation of findings favorable to treatment was larger in meta-analyses including older studies. In the largest study on publication bias in meta-analyses to date, we found evidence of publication bias in Cochrane systematic reviews. In general, publication bias is smaller in meta-analyses of more recent studies, indicating their better reliability and supporting the effectiveness of the measures used to reduce publication bias in clinical trials. Our results indicate the need to apply currently underutilized meta-analysis tools handling publication bias based on the statistical significance, especially when studies included in a meta-analysis are not recent.

Publication bias in clinical trials of electronic health records

ObjectiveTo measure the rate of non-publication and assess possible publication bias in clinical trials of electronic health records. MethodsWe searched ClinicalTrials.gov to identify registered clinical trials of electronic health records and searched the biomedical literature and contacted trial investigators to determine whether the results of the trials were published. Publications were judged as positive, negative, or neutral according to the primary outcome. ResultsSeventy-six percent of trials had publications describing trial results; of these, 74% were positive, 21% were neutral, and 4% were negative (harmful). Of unpublished studies for which the investigator responded, 43% were positive, 57% were neutral, and none were negative; the lower rate of positive results was significant (p<0.001). ConclusionThe rate of non-publication in electronic health record studies is similar to that in other biomedical studies. There appears to be a bias toward publication of positive trials in this domain.

Publication bias in clinical trials due to statistical significance or direction of trial results

The tendency for authors to submit, and of journals to accept, manuscripts for publication based on the direction or strength of the study findings has been termed publication bias. To assess the extent to which publication of a cohort of clinical trials is influenced by the statistical significance, perceived importance, or direction of their results. We searched the Cochrane Methodology Register (The Cochrane Library [Online] Issue 2, 2007), MEDLINE (1950 to March Week 2 2007), EMBASE (1980 to Week 11 2007) and Ovid MEDLINE In-Process & Other Non-Indexed Citations (March 21 2007). We also searched the Science Citation Index (April 2007), checked reference lists of relevant articles and contacted researchers to identify additional studies. Studies containing analyses of the association between publication and the statistical significance or direction of the results (trial findings), for a cohort of registered clinical trials. Two authors independently extracted data. We classified findings as either positive (defined as results classified by the investigators as statistically significant (P < 0.05), or perceived as striking or important, or showing a positive direction of effect) or negative (findings that were not statistically significant (P >/= 0.05), or perceived as unimportant, or showing a negative or null direction in effect). We extracted information on other potential risk factors for failure to publish, when these data were available. Five studies were included. Trials with positive findings were more likely to be published than trials with negative or null findings (odds ratio 3.90; 95% confidence interval 2.68 to 5.68). This corresponds to a risk ratio of 1.78 (95% CI 1.58 to 1.95), assuming that 41% of negative trials are published (the median among the included studies, range = 11% to 85%). In absolute terms, this means that if 41% of negative trials are published, we would expect that 73% of positive trials would be published.Two studies assessed time to publication and showed that trials with positive findings tended to be published after four to five years compared to those with negative findings, which were published after six to eight years. Three studies found no statistically significant association between sample size and publication. One study found no significant association between either funding mechanism, investigator rank, or sex and publication. Trials with positive findings are published more often, and more quickly, than trials with negative findings.

Evidence of publication bias in clinical trials of biotherapies for solid tumors

We recently performed what to our knowledge is the largest survey published to date regarding clinical trials of biotherapies for solid tumors to review their content and quality. Data concerning general study characteristics, patient and disease factors, study methodology, and factors related to the completeness of reporting in 334 prospective clinical trials were collected from 5 distinguished journals between 1990 and 2002. We found a wide diversity with regard to the modalities of conducting and reporting such trials as well as the presence of some methodologic pitfalls, suggesting that the methodologic standards for the conduction and publishing of clinical trials of biotherapies should be improved.1 In an additional analysis, we found a statistically significant association between the high impact factor of the journal (> 5) and a positive conclusion for the study (P = 0.0003). It is interesting to note that, in a multiple regression model performed to analyze the correlation between impact factor and two explanatory variables of Phase II studies (high quality vs. poor quality and a positive study conclusion vs. a negative study conclusion), the only variable found to be predictive of publication in a high-impact factor journal was the positive conclusion of the study (P = 0.0024). These latter findings could indicate that a positive trial regarding biologic therapies is more likely to be published in a high-impact factor journal independent of its quality. Publication bias in other medical contexts already has been found and discussed previously.2, 3 We would point out that interest in biotherapies is increasing and that, in the near future, these new compounds will enrich the therapeutic possibilities of oncology. However, the considerable heterogeneity observed between trials and evidence of a publication bias might affect the reliability of the body of published data concerning biotherapies for cancer. This issue should prompt editors and reviewers to improve their strategies and methodologic tools, particularly when discussing and evaluating negative study results.

Publication bias in clinical trials and economic analyses.

Publication bias is an established notion in the clinical literature; essentially, large studies or positive results are more likely to find their way into the public domain than small studies or negative results. Meta-analysis presents a focus for the problems of publication bias, seeking to summarise the evidence in a particular therapeutic area by retrieving and analysing all available clinical studies. Economic analyses are also vulnerable to publication bias, and at 3 levels: first, in the health-outcomes data available for modelling (resulting from publication bias in the clinical literature); second, in the motivations for conducting an economic analysis; and third, in repeating the process of seeking publication.

An international survey to identify unpublished and ongoing perinatal trials: Jini Hetherington The Project on Publication Bias in Clinical Trials, National Perinatal Epidemiology Unit, Radcliffe Infirmary, Oxford, England, Johns Hopkins School of Public Health and Harvard School of Public Health(23)

An international survey to identify unpublished and ongoing perinatal trials: Jini Hetherington The Project on Publication Bias in Clinical Trials, National Perinatal Epidemiology Unit, Radcliffe Infirmary, Oxford, England, Johns Hopkins School of Public Health and Harvard School of Public Health(23)

Assessing the magnitude of publication bias in clinical trials: Jesse A. Berlin, Colin B. Begg, and Thomas A. Louis Department of Biostatistics, Harvard School of Public Health, Boston, Massachusetts(S02)

Assessing the magnitude of publication bias in clinical trials: Jesse A. Berlin, Colin B. Begg, and Thomas A. Louis Department of Biostatistics, Harvard School of Public Health, Boston, Massachusetts(S02)