Reply to Mac Giolla and Ly (2019): On the reporting of Bayes factors in deception research

Abstract

Bayes factors provide a continuous measure of evidence for one hypothesis (e.g., the null, H0) relative to another (e.g., the alternative, H1). Warmelink et al. (2019, Legal Criminol Psychol, 24, 258) reported Bayes factors alongside p‐values to draw inferences about whether the order of expected versus unexpected questions influenced the amount of details interviewees provided during an interview. Mac Giolla & Ly (2019) provided several recommendations to improve the reporting of Bayesian analyses and used Warmelink et al. (2019) as a concrete example. These included (I) not to over‐rely on cut‐offs when interpreting Bayes factors; (II) to rely less on Bayes factors, and switch to ‘nominal support’; and (III) to report the posterior distribution. This paper elaborates on their recommendations and provides two further suggestions for improvement. First, we recommend deception researchers report Robustness Regions to demonstrate the sensitivity of their conclusions to the model of H1 used. Second, we demonstrate a method that deception researchers can use to estimate, a priori, the sample size likely to be required to provide conclusive evidence.