P-value and the probability of direction of effect

Abstract

The p-value has been widely criticized in the scientific literature for its naive use in classifying results as 'significant' and 'non significant'. Much has been written about it; for example, see the American Statistical Association position statement of march 2016. To date, few alternative measures have been suggested and few changes were observed in the scientific practice regarding the use of p-value despite general agreement on the critics raised on it. In this paper, we use an alternative measure to p-value. It consists in the probability of the direction of the effect, that is the strength of empirical evidence in favour of the alternative directional hypothesis. In the context of scientific research, reporting the probability of the direction of the effect is easier to understand. Moreover, it focuses on the effect in the study rather than on the value under the null hypothesis, which sometimes has little meaning or has been used opportunistically. The proposal is not intended as an alternative to using the confidence interval, but as a probabilistic metric to be used instead of the p-value when we refer to particular hypotheses to be tested.