What to Do Instead of Null Hypothesis Significance Testing or Confidence Intervals

Abstract

Based on the banning of null hypothesis significance testing and confidence intervals in Basic and Applied Psychology (2015), this presentation focusses on alternative ways for researchers to think about inference. One section reviews literature on the a priori procedure. The basic idea, here, is that researchers can perform much inferential work before the experiment. Furthermore, this possibility changes the scientific philosophy in important ways. A second section moves to what researchers should do after they have collected their data, with an accent on obtaining a better understanding of the obtained variance. Researchers should try out a variety of summary statistics, instead of just one type (such as means), because seemingly conceptually similar summary statistics nevertheless can imply very different qualitative stories. Also, rather than engage in the typical bipartite distinction between variance due to the independent variable and variance not due to the independent variable; a tripartite distinction is possible that divides variance not due to the independent variable into variance due to systematic or random factors, with important positive consequences for researchers. Finally, the third major section focusses on how researchers should or should not draw causal conclusions from their data. This section features a discussion of within-participants causation versus between-participants causation, with an accent on whether the type of causation specified in the theory is matched or mismatched by the type of causation tested in the experiment. There also is a discussion of causal modeling approaches, with criticisms. The upshot is that researchers could do much more a priori work, and much more a posteriori work too, to maximize the scientific gains they obtain from their empirical research.