Inferential Statistics Is an Unfit Tool for Interpreting Data

Abstract

Null hypothesis significance testing is a commonly used tool for making statistical inferences in empirical studies, but its use has always been controversial. In this manuscript, I argue that even more problematic is that significance testing, and other abstract statistical benchmarks, often are used as tools for interpreting study data. This is problematic because interpreting data requires domain knowledge of the scientific topic and sensitivity to the study context, something that significance testing and other purely statistical approaches are not. By using simple examples, I demonstrate that researchers must first use their domain knowledge—professional expertise, clinical experience, practical insight—to interpret the data in their study and then use inferential statistics to provide some reasonable estimates about what can be generalized from the study data. Moving beyond the current focus on abstract statistical benchmarks will encourage researchers to measure their phenomena in more meaningful ways, transparently convey their data, and communicate their intellectual reasons for interpreting the data as they do, a shift that will better foster a scientific forum for cumulative science.