Abstract
Problemification: Over-reliance on null hypothesis significance testing (NHST) is one of the most important causes of the emerging crisis over the credibility and reproducibility of our science.Implications: Most studies in the behavioural and social sciences have low levels of statistical power. Because ‘significant’ results are often required, but often difficult to produce, the temptation to engage in questionable research practices that will produce these results is immense.Purpose: Methodologists have been trying for decades to convince researchers, reviewers and editors that significance tests are neither informative nor useful. A recent set of articles published in top journals and endorsed by hundreds of scientists around the world seem to provide a fresh impetus for overturning the practice of using NHST as the primary, and sometimes sole basis for evaluating research results.Recommendations: Authors, reviewers and journal editors are asked to change long-engrained habits and realise that ‘statistically significant’ says more about the design of one’s study than about the importance of one’s results. They are urged to embrace the ATOM principle in evaluating research results, that is, accept that there will always be uncertainty, and be thoughtful, open and modest in evaluating what the data mean.
Highlights
There are many indications that several sciences, including psychology, have a reproducibility crisis in their hands (Ioannidis, 2005; McNutt, 2014; Pashler & Wagenmakers, 2012)
Efendic and Van Zyl (2019) provide an excellent summary of the challenges this crisis poses to Industrial and Organizational Psychology, and they outline several thoughtful responses this journal might make to increase the robustness and credibility of the research published in the South African Journal of Industrial Psychology
A series of recent articles (Amrhein, Greenland, & McShane, 2019; Wasserstein & Lazar, 2016; Wasserstein, Schirm, & Lazar, 2019) in highprofile journals (e.g. Nature and American Statistician) have accomplished three things that decades of research papers and chapters of previous critics of Null Hypothesis Significance Testing (NHST) had not been able to accomplish. They have described in clear and largely non-technical language the deficiencies of NHST as a method for making decisions about the meaning of results. They have gathered the support of hundreds of signatories to statements calling for an end to mechanical reliance on significance testing for evaluating findings
Summary
There are many indications that several sciences, including psychology, have a reproducibility crisis in their hands (Ioannidis, 2005; McNutt, 2014; Pashler & Wagenmakers, 2012). A series of recent articles (Amrhein, Greenland, & McShane, 2019; Wasserstein & Lazar, 2016; Wasserstein, Schirm, & Lazar, 2019) in highprofile journals (e.g. Nature and American Statistician) have accomplished three things that decades of research papers and chapters of previous critics of NHST had not been able to accomplish They have described in clear and largely non-technical language the deficiencies of NHST as a method for making decisions about the meaning of results. They have gathered the support of hundreds of signatories (there were over 800 signatories from over 50 countries within a week of the distribution of Amrhein et al.’s 2019 draft) to statements calling for an end to mechanical reliance on significance testing for evaluating findings. This strikes me as a very useful piece of advice for the South African Journal of Industrial Psychology, its contributing authors and incoming editorial board
Sign-in/Register to view highlights & summary 
Published Version (
Free)
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call