Abstract

Confirmation through competent replication is a founding principle of modern science. However, biomedical researchers are rewarded for innovation, and not for confirmation, and confirmatory research is often stigmatized as unoriginal and as a consequence faces barriers to publication. As a result, the current biomedical literature is dominated by exploration, which to complicate matters further is often disguised as confirmation. Only recently scientists and the public have begun to realize that high-profile research results in biomedicine can often not be replicated. Consequently, confirmation has become central stage in the quest to safeguard the robustness of research findings. Research which is pushing the boundaries of or challenges what is currently known must necessarily result in a plethora of false positive results. Thus, since discovery, the driving force of scientific progress, is unavoidably linked to high false positive rates and cannot support confirmatory inference, dedicated confirmatory investigation is needed for pivotal results. In this chapter I will argue that the tension between the two modes of research, exploration and confirmation, can be resolved if we conceptually and practically separate them. I will discuss the idiosyncrasies of exploratory and confirmatory studies, with a focus on the specific features of their design, analysis, and interpretation.

Highlights

  • The current biomedical literature is dominated by exploration, which is often disguised as confirmation

  • Exploratory results are often garnished with far-reaching claims regarding the relevance of the observed phenomenon for the future treatment or even cure of a disease

  • If research operates with a probability that 10% of its hypotheses are true, which is a conservative estimate, and accepts type I errors at 5% and type II errors at 20% (i.e., 80% power), almost 40% of the times, it rejects the NULL hypothesis, while the NULL hypothesis is true (Colquhoun 2014)

Read more

Summary

Ulrich Dirnagl

U. Dirnagl boundaries of or challenges what is currently known must necessarily result in a plethora of false positive results. Since discovery, the driving force of scientific progress, is unavoidably linked to high false positive rates and cannot support confirmatory inference, dedicated confirmatory investigation is needed for pivotal results. I will discuss the idiosyncrasies of exploratory and confirmatory studies, with a focus on the specific features of their design, analysis, and interpretation. Keywords False negative · False positive · Preclinical randomized controlled trial · Replication · Reproducibility · Statistics. Roddenberry) Non-reproducible single occurrences are of no significance to science

Introduction
Discrimination Between Exploration and Confirmation
Exploration Must Lead to a High Rate of False Positives
The Garden of Forking Paths
Confirmation Must Weed Out the False Positives of Exploration
Exact Replication Does Not Equal Confirmation
No Publication Without Confirmation?
Findings
Team Science and Preclinical Multicenter Trials
Full Text
Published version (Free)

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call