Erratum to: Letter to the Editor: The ongoing tyranny of statistical significance testing in biomedical research

Abstract

We thank Tyler VanderWeele [1] for his kind words about our recent paper on statistical significance [2]. We will thank him even more in the future if he stops suggesting that we are intent on ‘‘dismissing the P-value entirely.’’ We oppose some uses of P-values, not P-values themselves. If any of us does issue a call for P-values to be banished, it will require no reading between the lines to discern. Long ago, one of us [3] suggested that interpretation might be enhanced by inspecting the graph of all P-values for all hypothetical values of the measure one is estimating. Our support for the continued reporting of point estimates and confidence intervals [2] was tantamount to encouraging the reporting of three P-values: P = 1 for the point estimate and P = 0.05 for each of the interval’s limits (if the confidence level is 95%). How these recommendations can be viewed as aligning us with those who would instruct authors ‘‘to refrain from giving P-values entirely’’ is a mystery. To be fair, VanderWeele does not defend the reporting of all P-values. He has a specific P-value in mind, one so special it has become known as ‘‘the’’ P-value. It is the P-value for the null hypothesis. We have no objection to the reporting of this or any other P-value. What matters to us at present is what VanderWeele would have us do with it. As we have tried to stress, we believe it is wise for epidemiologic researchers to focus on estimates: point estimates, interval estimates or entire P-value or likelihood functions. Estimates are what systematic reviewers seek when they review a literature. When they do not find them and find null P-values instead, they are understandably disappointed, for they know their review will not reach as deep an understanding of the state of epidemiologic research on the topic as it might have. When we look at the forest plot in the Figure [4], we see 13 estimates. Apparently, VanderWeele sees something else: up to 13 ‘‘findings.’’ We are not sure how he defines this key term in his framework. It would appear that a finding cannot be a null P-value, as he urges us to use null P-values in coming to judgments about findings. It would also seem that findings, as the objects of those judgments, cannot be the judgments themselves. Our best guess is that VanderWeele’s ‘‘findings’’ are estimates, though we suspect that not every estimate qualifies. VanderWeele seems to view chance and bias as competing, alternative explanations for findings. Chance is to be considered first, for reasons that are obscure to us, with the aim of ‘‘dismissing’’ it, for reasons that seem just as obscure. Given a very low null P-value, he would have us The online version of the original article can be found under doi:10.1007/s10654-010-9507-8.