Statistical Reproducibility of Meta-Analysis for Medical Mask Use in Community Settings to Prevent Airborne Respiratory Virus Infection

Abstract

Many US states, cities, and counties implemented public masking orders during the coronavirus (COVID) pandemic on the notion that this intervention would delay and flatten the epidemic peak and largely benefit public health outcomes. A p-value plot can provide insights into possible inappropriateness (incorrectness) of assumptions of a statistical model. It can be used to confirm, disprove, or identify ambiguity (uncertainty) in a meta-analytic finding and research claim. P-value plotting was used to evaluate statistical reproducibility of meta-analysis studies for disposable medical (surgical) mask use in community settings to prevent airborne respiratory virus infection. Eight studies (seven meta-analysis, one systematic review) published between 1 January 2020 and 7 December 2022 were evaluated. Base studies were randomized control trials with outcomes of medical diagnosis or laboratory-confirmed diagnosis of viral (Influenza or COVID) illness. Self-reported viral illness outcomes were excluded from the evaluation because of awareness bias. No evidence was observed for a medical mask benefit to prevent respiratory virus infection in six p-value plots (five meta-analysis and one systematic review). Research claims of no benefit in three meta-analysis and the systematic review were reproduced in p-value plots. Research claims of a benefit in two other meta-analysis were not reproduced in p-value plots suggesting irreproducibility of these claims. Insufficient data was available to construct p-value plots for two other meta-analysis because of over-reliance on self-reported outcomes. Independent findings of p-value plotting show that meta-analysis of existing randomized control trials fail to demonstrate a benefit of medical mask use in community settings to prevent airborne respiratory virus infection.