Bayesian Assessment of Null Values Via Parameter Estimation and Model Comparison

Abstract

Psychologists have been trained to do data analysis by asking whether null values can be rejected. Is the difference between groups nonzero? Is choice accuracy not at chance level? These questions have been traditionally addressed by null hypothesis significance testing (NHST). NHST has deep problems that are solved by Bayesian data analysis. As psychologists transition to Bayesian data analysis, it is natural to ask how Bayesian analysis assesses null values. The article explains and evaluates two different Bayesian approaches. One method involves Bayesian model comparison (and uses Bayes factors). The second method involves Bayesian parameter estimation and assesses whether the null value falls among the most credible values. Which method to use depends on the specific question that the analyst wants to answer, but typically the estimation approach (not using Bayes factors) provides richer information than the model comparison approach.