Chapter 10 - The “New Statistics” and Bayesian Inference

Abstract

Null hypothesis statistical testing (NHST) is a good thing—like money—yet the love of money has been accused of being the root of all evil. Similarly, a flood of ink has been spilled critiquing NHST and pointing out all the ways that its misuse by scientists has contributed to the current crisis in reproducibility. In this chapter, I will review some of the major problems with NHST and will outline good practices that can keep its benefits without leading you astray. Bayesian inference is an alternative approach to statistical testing that has some advantages over NHST. I will describe Bayesian inference briefly and contrast it to NHST. Bayesian methods are currently much less popular than NHST, but are conceptually appealing and may become more prevalent in the future. Finally, metaanalyses and systematic reviews are analyses that combine data across multiple independent studies. Metaanalyses and systematic reviews have great potential for revealing which findings are most reliable and robust. However, they face statistical challenges of their own, particularly when they are constrained to consider only published summary data rather than raw data.