Bad statistical practice in pharmacology (and other basic biomedical disciplines): you probably don't knowP

Abstract

Statistical analysis is universally used in the interpretation of the results of basic biomedical research, being expected by referees and readers alike. Its role in helping researchers to make reliable inference from their work and its contribution to the scientific process cannot be doubted, but can be improved. There is a widespread and pervasive misunderstanding of P-values that limits their utility as a guide to inference, and a change in the manner in which P-values are specified and interpreted will lead to improved outcomes. This paper explains the distinction between Fisher's P-values, which are local indices of evidence against the null hypothesis in the results of a particular experiment, and Neyman-Pearson α levels, which are global rates of false positive errors from unrelated experiments taken as an aggregate. The vast majority of papers published in pharmacological journals specify P-values, either as exact-values or as being less than a value (usually 0.05), but they are interpreted in a hybrid manner that detracts from their Fisherian role as indices of evidence without gaining the control of false positive and false negative error rate offered by a strict Neyman-Pearson approach. An informed choice between those approaches offers substantial advantages to the users of statistical tests over the current accidental hybrid approach.