An Analysis of Studies Pertaining to Masks in Morbidity and Mortality Weekly Report: Characteristics and Quality of Studies through 2023

Abstract

BackgroundThe purpose of this study was to describe and evaluate the nature and methodology of reports and appropriateness of conclusions in The Morbidity and Mortality Weekly Report (MMWR) pertaining to masks. Because MMWR has substantial influence on United States health policy and is not externally peer-reviewed, it is critical to understand the scientific process within the journal. Mask policies have been highly influenced by data published in the MMWR. MethodsRetrospective cross-sectional study of MMWR publications pertaining to masks through 2023. Outcomes included study design, whether the study was able to assess mask effectiveness, if results were statistically significant, if masks were concluded to be effective, if randomized evidence or conflicting data were mentioned or cited, and appropriateness of causal statements. ResultsThere were 77 studies, all published after 2019, that met our inclusion criteria. The most common study design was observational without a comparator group: 22/77 (28.6%); 0/77 were randomized; 23/77 (29.9%) assessed mask effectiveness; 11/77 (14.3%) were statistically significant, but 58/77 (75.3%) stated that masks were effective. Of these, 41/58 (70.7%) used causal language. One mannequin study used causal language appropriately (1.3%). None cited randomized data; 1/77 (1.3%) cited conflicting evidence. ConclusionsMMWR publications pertaining to masks drew positive conclusions about mask effectiveness >75% of the time despite only 30% testing masks and <15% having statistically significant results. No studies were randomized, yet over half drew causal conclusions. The level of evidence generated was low and the conclusions were most often unsupported by the data. Our findings raise concern about the reliability of the journal for informing health policy.