Bayesian alternatives for common null-hypothesis significance tests in psychiatry: a non-technical guide using JASP

Donald R. Williams,Daniel S. Quintana

doi:10.1186/s12888-018-1761-4

Abstract

BackgroundDespite its popularity as an inferential framework, classical null hypothesis significance testing (NHST) has several restrictions. Bayesian analysis can be used to complement NHST, however, this approach has been underutilized largely due to a dearth of accessible software options. JASP is a recently developed open-source statistical package that facilitates both Bayesian and NHST analysis using a graphical interface. This article provides an applied introduction to Bayesian inference with Bayes factors using JASP.MethodsWe use JASP to compare and contrast Bayesian alternatives for several common classical null hypothesis significance tests: correlations, frequency distributions, t-tests, ANCOVAs, and ANOVAs. These examples are also used to illustrate the strengths and limitations of both NHST and Bayesian hypothesis testing.ResultsA comparison of NHST and Bayesian inferential frameworks demonstrates that Bayes factors can complement p-values by providing additional information for hypothesis testing. Namely, Bayes factors can quantify relative evidence for both alternative and null hypotheses. Moreover, the magnitude of this evidence can be presented as an easy-to-interpret odds ratio.ConclusionsWhile Bayesian analysis is by no means a new method, this type of statistical inference has been largely inaccessible for most psychiatry researchers. JASP provides a straightforward means of performing reproducible Bayesian hypothesis tests using a graphical “point and click” environment that will be familiar to researchers conversant with other graphical statistical packages, such as SPSS.

Highlights

Despite its popularity as an inferential framework, classical null hypothesis significance testing (NHST) has several restrictions
With p-values, we cannot be certain if non-significance is due to data insensitivity or to evidence supporting a lack of relationship between these two variables [4, 19, 30]
For our Bayesian analysis, we will compare two models: the null hypothesis (H0) that the data is distributed according to a bivariate normal distribution with zero covariance — and that there is no correlation between the spirituality and age (i.e., ρ = 0) — and the alternative hypothesis (H1) that age and spirituality distributed according to a bivariate normal distribution with a non-zero covariance are related

Summary

Introduction

Despite its popularity as an inferential framework, classical null hypothesis significance testing (NHST) has several restrictions. JASP is a recently developed open-source statistical package that facilitates both Bayesian and NHST analysis using a graphical interface. The prevailing inferential framework for summarizing evidence in psychiatry is null hypothesis significance testing (NHST), which is a hybrid of Fisherian and Neyman-Pearson statistics [1]. NHST generates a test-statistic, such as a t-value, and the probability (p-value) of observing this value or a more extreme result is computed, assuming that the null hypothesis is true. Even a “large” non-significant p-value does not provide evidence for the null hypothesis [5]. Unless an a priori power analysis is performed, there is no clear indication if a dataset is sensitive enough to detect a true effect when using p-values [8]

Objectives

Methods

Results

Conclusion