Model-averaged Bayesian ttests.

Abstract

One of the most common statistical analyses in experimental psychology concerns the comparison of two means using the frequentist ttest. However, frequentist ttests do not quantify evidence and require various assumption tests. Recently, popularized Bayesian ttests do quantify evidence, but these were developed for scenarios where the two populations are assumed to have the same variance. As an alternative to both methods, we outline a comprehensive ttest framework based on Bayesian model averaging. This new ttest framework simultaneously takes into account models that assume equal and unequal variances, and models that use t-likelihoods to improve robustness to outliers. The resulting inference is based on a weighted average across the entire model ensemble, with higher weights assigned to models that predicted the observed data well. This new ttest framework provides an integrated approach to assumption checks and inference by applying a series of pertinent models to the data simultaneously rather than sequentially. The integrated Bayesian model-averaged ttests achieve robustness without having to commit to a single model following a series of assumption checks. To facilitate practical applications, we provide user-friendly implementations in JASP and via the package in . A tutorial video is available at https://www.youtube.com/watch?v=EcuzGTIcorQ.