7-days of FREE Audio papers, translation & more with Prime
7-days of FREE Prime access
7-days of FREE Audio papers, translation & more with Prime
7-days of FREE Prime access
https://doi.org/10.1111/eci.12912
Copy DOIPublication Date: Feb 28, 2018 | |
Citations: 30 |
Despite regular criticisms of null hypothesis significance testing (NHST), a focus on testing persists, sometimes in the belief to get published and sometimes encouraged by journal reviewers. This paper aims to demonstrate known key limitations of NHST using simple nontechnical illustrations. The first illustration is based on simulated data of 20000 studies that compare two groups for an outcome event. The true effect size (difference in event rates) and sample size (20-100 per group) were varied. The second illustration used real data from a meta-analysis on alpha-blockers for the treatment of ureteric stones. The simulations demonstrated the large between-study variability in P-values (range between <.0001 and 1 for most simulation conditions). A focus on statistically significant effects (P<.05), notably in small to moderate samples, led to strongly overestimated effect sizes (up to 240%) and many false-positive conclusions, that is statistically significant effects that were, in fact, true null effects. Effect sizes also exerted strong between-study variability, but confidence intervals accounted for this: the interval width decreased with larger sample size, and the percentage of intervals that contained the true effect size was accurate across simulation conditions. Reducing alpha level, as recently suggested, reduced false-positive conclusions but strongly increased the overestimation of significant effects (up to 320%). Researchers and journals should abandon statistical significance as a pivotal element in most scientific publications. Confidence intervals around effect sizes are more informative, but should not merely be reported to comply with journal requirements.
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.