Abstract
BackgroundA recent paper found that terminal digits of statistical values in Nature deviated significantly from an equiprobable distribution, indicating errors or inconsistencies in rounding. This finding, as well as the discovery that a large percentage of p values were inconsistent with reported test statistics, led to a great deal of concern in the popular press and scientific community. The findings ultimately led to new guidelines for all Nature Research Journals.MethodsWe checked the statistical analysis behind the original paper's tests of equiprobability.ResultsThe original paper tested equiprobability with the Kolmogorov-Smirnov test outside its regime of validity. Correct tests find no statistically significant deviations from equiprobability for the statistical values in Nature.ConclusionStatistical tests should be used correctly.
Highlights
A recent paper found that terminal digits of statistical values in Nature deviated significantly from an equiprobable distribution, indicating errors or inconsistencies in rounding
The Kolmogorov-Smirnov test is normally used to test whether data follows a specified continuous distribution [8]
A simple calculation from the raw data in [1] shows that the Z and p values obtained there are those based on comparing the data with a distribution uniform on the continuous interval [0, 9]
Summary
A recent paper concluded that "statistical practice is generally poor, even in the most renowned scientific journals" [1]. The paper prompted significant attention in the popular press, and serious concern within the scientific community [2,3,4,5,6,7] It led the editors of Nature Medicine to review their statistical practices, resulting in new statistical guidelines for all Nature Research Journals [3]. The authors of [1] collected random samples of test statistics and p values published in Nature, and looked at the terminal digits of these numbers. Their raw data is shown in tables 1 and 2.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
