Abstract
The inferential inadequacies of statistical significance testing are now widely recognized. There is, however, no consensus on how to move research into a ‘post p < 0.05’ era. We present a potential route forward via the Analysis of Credibility, a novel methodology that allows researchers to go beyond the simplistic dichotomy of significance testing and extract more insight from new findings. Using standard summary statistics, AnCred assesses the credibility of significant and non-significant findings on the basis of their evidential weight, and in the context of existing knowledge. The outcome is expressed in quantitative terms of direct relevance to the substantive research question, providing greater protection against misinterpretation. Worked examples are given to illustrate how AnCred extracts additional insight from the outcome of typical research study designs. Its ability to cast light on the use of p-values, the interpretation of non-significant findings and the so-called ‘replication crisis’ is also discussed.
Highlights
Statistical inference plays a key role in the scientific enterprise by providing techniques for turning data into insight
There remains no consensus on the way forward, despite a plethora of suggestions ranging from a simple tightening of the p-value threshold [32] through false discovery risk methods [33,34] and Bayes Factors [35] to sophisticated Bayesian hierarchical modelling [36]
As a response to the call to move towards the ‘post p < 0.05 era’, the Analysis of Credibility (AnCred) has been developed in the 10 spirit of evolution rather than revolution
Summary
Statistical inference plays a key role in the scientific enterprise by providing techniques for turning data into insight. The ASA Statement gives no explicit guidance on how this should be accomplished, stating only that some statisticians ‘supplement or even replace p-value’ using methods such as estimation via confidence intervals (CIs), Bayesian methods and false discovery rates This lack of specific guidance reflects long-standing debate among statisticians about the relative merits of different inferential. There is unlikely ever to be agreement on a single inferential technique to replace significance testing, not least because of the multi-faceted nature of inference. Both the ASA Statement and its associated Commentaries point to a consensus on the desirable features of any acceptable alternatives:. The result is an inferential toolkit which can be used alongside standard statistical significance testing, extracting extra insight from findings expressed using conventional summary statistics
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
