Bayesian Methods Might Solve the Problems with Magnitude-based Inference.

Abstract

Dear Editor-in-Chief, The recent and interesting article by Sainani (1) addresses issues with using magnitude-based inference (MBI) as a stand-alone statistical method, namely, type I and type II error rates. Claims of MBI boasting superior error rates compared with standard null hypothesis significance testing are appealing and of interest to many sports science researchers, who often observe small treatment effects using small sample sizes. Sainani (1), among others (2,3), has highlighted concerns with MBI, with Sainani (1) recently providing empirical and mathematical evidence against the claims of superior error rates of MBI. Attempts to adopt more robust statistical methods for the challenges encountered in sports science should be commended. Bayesian estimation has been shown to overcome the concerns raised by Sainani (1) while providing more intuitive and practically interpretable results compared with the MBI approach (3,4). In 2015, Mengersen and colleagues (3) provided a Bayesian formulation alongside MBI that showed the ability of the Bayesian statistical framework to evaluate small effects in a small participant sample. Bayesian estimation provided more directly interpretable and theoretically justifiable probabilistic outcomes compared with the MBI approach. As alluded to by Sainani (1) and addressed elsewhere (5,6), confidence intervals are often interpreted incorrectly, and this has led and will continue to lead to inappropriate conclusions drawn from data, particularly through MBI. In contrast, the corresponding Bayesian interval (credible interval) is directly interpretable; as outlined by Mengersen et al. (3), a 95% credible interval indicates that the true parameter lies within the interval with an estimated probability of 0.95. Critically, the Bayesian approach can provide further information. From the posterior distribution, the probability that an effect size exceeds a threshold of interest can be easily calculated. For example, the authors (3) calculate the probability that Cohen’s d exceeded an objectively derived threshold of sporting interest. In addition to the variety of inferences that can be drawn from the posterior distribution and its ease of interpretation, the Bayesian approach also provides a principled framework for incorporating prior information from previous studies or experts, which might be valuable when sample sizes are small to moderate. Importantly, Bayesian methods do not rely on asymptotic assumptions that are often made in classical hypothesis testing and confidence interval construction (e.g., the inherent symmetric nature of the confidence intervals considered in the mathematical derivations of Sainani (1)). Such asymptotic assumptions may be questionable in small-to-moderate sample sizes. It is worth noting that parametric bootstrap confidence intervals also do not rely on asymptotic assumptions; however, they are more computationally intensive compared with standard classical confidence intervals, and retain other drawbacks of the classical approach, such as difficult interpretation. Despite the template and worked example provided by Mengersen et al. (3), there has been an inherent lack of Bayesian methods used within sports science research. Accessibility was suggested as a potential explanation for its absence (3); however, texts with supportive code for multiple statistical platforms (e.g., R and Stan) are readily available (7,8). The need to select a prior distribution may also lead to hesitancy in implementing Bayesian estimation, although the construction of a “vague” prior is usually straightforward (3). We believe that the aforementioned barriers would be easily overcome through collaborative links between sports scientists and Bayesian-trained statisticians. Sainani (1) should be commended for the thorough mathematical reasoning showing that MBI does not exhibit superior error rates to null hypothesis significance testing. As hinted at in Sainani (1) and shown in Mengersen et al. (3), Bayesian estimation may be able to enhance statistical analytics in sports science. David N. Borg Geoffrey M. Minett Ian B. Stewart Institute of Health and Biomedical Innovation and School of Exercise and Nutrition Sciences Queensland University of Technology Brisbane, Queensland, AUSTRALIA Christopher C. Drovandi School of Mathematical Sciences and the Australian Research Council Centre of Excellence for Mathematical and Statistical Frontiers Queensland University of Technology Brisbane, Queensland, AUSTRALIA