Abstract
What may be a contributing cause of the replication problem in science – multiple testing bias – was examined in this study. Independent analysis was performed on a meta-analysis of cohort studies associating ambient exposure to nitrogen dioxide (NO2) and fine particulate matter (PM2.5) with development of asthma. Statistical tests used in 19 base papers from the meta-analysis were counted. Test statistics and confidence intervals from the base papers used for meta-analysis were converted to p-values. A combined p-value plot for NO2 and PM2.5 was constructed to evaluate the effect heterogeneity of the p-values. Large numbers of statistical tests were estimated in the 19 base papers – median 13,824 (interquartile range 1,536−221,184). Given these numbers, there is little assurance that test statistics used from the base papers for meta-analysis are unbiased. The p-value plot of test statistics showed a two-component mixture. The shape of the p-value plot for NO2 suggests the use of questionable research practices related to small p-values in some of the cohort studies. All p-values for PM2.5 fall on a 45-degree line in the p-value plot indicating randomness. The claim that ambient exposure to NO2 and PM2.5 is associated with development of asthma is not supported by our analysis.
Highlights
Explanation of Figs C3−1 and C3−2 Figure C3−1 presents a set of p-value plots for small meta-analysis datasets (i.e., n
Fig C3−1 (i); left image – presents p-values as a sloped line from left to right at approximately 45-degrees representing a plausible true null hypothesis for a chronic myeloid leukemia causal relationship in petroleum refinery workers (n=12) after Schnatter et al (2018)
Fig C3−2 (i); left image – presents p-values as a sloped line from left to right at approximately 45-degrees representing a plausible true null hypothesis for an elderly long-term exercise training−mortality & morbidity causal relationship (n=69) after de Souto Barreto et al (2019)
Summary
Journal of Occupational and Environmental Medicine, 60(7), e329−e342. Meta-analysis of elderly long-term exercise training−mortality & morbidity risk (after de Souto Barreto et al, 2019*)
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callMore From: International Journal of Statistics and Probability
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
