Conflicting COVID-19 excess mortality estimates – Authors' reply

Abstract

The COVID-19 pandemic has caused almost 8 million reported deaths worldwide since late 2019.1Institute for Health Metrics and EvaluationCumulative deaths.https://covid19.healthdata.org/global?view=cumulative-deaths&tab=trendDate accessed: January 10, 2023Google Scholar Although this is a staggering loss of human lives, 8 million is a vast under-estimation of the true toll of the pandemic. In addition to under-reporting and misclassification of COVID-19 deaths, the pandemic has also resulted in loss of lives due to stressed health-care systems. Excess mortality due to the COVID-19 pandemic, a measurement of net changes in all-cause mortality during the pandemic compared with levels before the pandemic, is widely considered the best measurement of the overall effect of the pandemic and is increasingly a metric used to compare country performance with expectations. Indeed, massive media coverage has been given to excess mortality estimates provided by the COVID-19 Excess Mortality Collaborators,2COVID-19 Excess Mortality CollaboratorsEstimating excess mortality due to the COVID-19 pandemic: a systematic analysis of COVID-19-related mortality, 2020-21.Lancet. 2022; 399: 1513-1536Summary Full Text Full Text PDF PubMed Scopus (222) Google Scholar The Economist,3Solstad S The pandemic's true death toll. 2021. The Economist.https://www.economist.com/graphic-detail/coronavirus-excess-deaths-estimatesDate accessed: January 10, 2023Google Scholar and WHO.4WHOThe true death toll of COVID-19: estimating global excess mortality.https://www.who.int/data/stories/the-true-death-toll-of-covid-19-estimating-global-excess-mortalityDate accessed: January 10, 2023Google Scholar In all cases, both input data processing and modelling approaches have a substantial effect on estimated excess mortality for countries with or without reported vital registration data during the pandemic. In our peer-reviewed2COVID-19 Excess Mortality CollaboratorsEstimating excess mortality due to the COVID-19 pandemic: a systematic analysis of COVID-19-related mortality, 2020-21.Lancet. 2022; 399: 1513-1536Summary Full Text Full Text PDF PubMed Scopus (222) Google Scholar and GATHER5Stevens GA Alkema L Black RE et al.Guidelines for Accurate and Transparent Health Estimates Reporting: the GATHER statement.Lancet. 2016; 388: e19-e23Summary Full Text Full Text PDF PubMed Scopus (448) Google Scholar-compliant work, we attempted to account for both under-registration and late registration, a far more common issue even for high-income countries, in addition to correcting for data affected by the summer heatwaves in both Europe and North America. To our knowledge, our estimation pipeline is the only one that accounted for such input data issues that would bias excess mortality estimates for countries affected. In addition, an ensemble of six different models was developed to estimate expected all-cause mortality in the absence of the pandemic. Weights for each candidate model were generated with out-of-sample predictive validity testing. We opted for such a modelling approach, recognising that the choice of a model configuration has a substantial effect on the estimated expected mortality, and thus the estimated excess mortality. We are aware of no such caution in other estimations. To estimate excess mortality for the entirety of the pandemic for both groups of countries, with or without reported vital registration data before and during the pandemic, we developed a parsimonious regression model in which 15 covariates related to both the COVID-19 pandemic (such as reported COVID-19 death rate, seroprevalence, and infection-detection rate) and background population health indicators (eg, Universal Health Coverage index, proportion of population older than 75 years, and prevalence of diabetes and cigarette smoking) were chosen on the basis of a rigorous covariate selection process. Candidate covariates were selected after evaluating background population health conditions associated with mortality during the pandemic, on the basis of a meta-analysis done by the US Centers for Disease Control and Prevention. The same prediction model is used for all locations that are based on the ensemble model, including those contributed empirical excess mortality estimates, to account for any dropped time period due to the heatwave or late registration. For countries without reported all-cause mortality data to help guide the estimation of excess mortality during the pandemic, COVID-19 seroprevalence rate is perhaps the single most crucial driving force in the estimation of excess mortality in such countries, including countries from sub-Saharan Africa. The COVID-19 forecast project at the Institute for Health Metrics and Evaluation, from which we drew our COVID-19-related covariates, provides the most comprehensive and internally consistent estimates on incidence and prevalence of COVID-19 infections—also peer-reviewed6Barber RM Sorensen RJ Pigott DM et al.Estimating global, regional, and national daily and cumulative infections with SARS-CoV-2 through Nov 14, 2021: a statistical analysis.Lancet. 2022; 399: 2351-2380Summary Full Text Full Text PDF Scopus (34) Google Scholar and GATHER-compliant. In situations without available all-cause mortality data, our estimated excess mortality provides the best metric on the effect of the pandemic in such countries. Jonas Schöley, Mark Woolhouse, Desmond O’Neill, Alberto Donzelli, Peter Bager, and colleagues have pointed out the difference in our estimates compared with those provided by The Economist,3Solstad S The pandemic's true death toll. 2021. The Economist.https://www.economist.com/graphic-detail/coronavirus-excess-deaths-estimatesDate accessed: January 10, 2023Google Scholar The World Mortality Dataset,7Karlinsky A Kobak D Tracking excess mortality across countries during the COVID-19 pandemic with the World Mortality Dataset.eLife. 2021; 10: 10Crossref Scopus (162) Google Scholar and WHO.4WHOThe true death toll of COVID-19: estimating global excess mortality.https://www.who.int/data/stories/the-true-death-toll-of-covid-19-estimating-global-excess-mortalityDate accessed: January 10, 2023Google Scholar Aside from our assessment accounting for under-registration and late registration, a far more prevalent issue, even in low sociodemographic index countries, is how many weeks or months of observed data from before the pandemic are used as a baseline for extrapolation (eg, there is a 20-week delay in vital registration data being complete in the USA). How we extrapolate data and the division timepoint between baseline and extrapolation all contribute greatly to the estimated expected mortality and the resulting excess mortality. Research has and will continue to estimate the toll of the ongoing COVID-19 pandemic.8Msemburi W Karlinsky A Knutson V Aleshin-Guendel S Chatterji S Wakefield J The WHO estimates of excess mortality associated with the COVID-19 pandemic.Nature. 2023; 613: 130-137Crossref PubMed Scopus (3) Google Scholar Our current research focuses on further refining country-specific specifications in improving long-term mortality extrapolation for countries to iteratively improve the methods, as data from all countries become more complete. These data are essential for the next iteration of the Global Burden of Disease Study.9Murray CJL The Global Burden of Disease Study at 30 years.Nat Med. 2022; 28: 2019-2026Crossref PubMed Scopus (3) Google Scholar We declare no competing interests. Conflicting COVID-19 excess mortality estimatesA study1 by the COVID-19 Excess Mortality Collaborators estimates more than 18 million COVID-19 deaths globally by the end of 2021—three times those reported. The COVID-19 Excess Mortality Collaborators claim that under-ascertainment is especially severe in sub-Saharan Africa, with actual deaths 14 times higher than the 150 000 reported—more than 2 million excess deaths across the region in 2020–21. Full-Text PDF Conflicting COVID-19 excess mortality estimatesThe COVID-19 Excess Mortality Collaborators concluded that “The full impact of the pandemic has been much greater than what is indicated by reported deaths due to COVID-19 alone”. They estimate that 18·2 million (95% CI 17·1–19·6) people died worldwide because of the pandemic (as measured by excess mortality) in 2020–21, instead of the reported COVID-19 deaths of 5·94 million worldwide during that period, as WHO claims. Full-Text PDF Conflicting COVID-19 excess mortality estimatesExcess mortality is an important metric summarising COVID-19 disease burden, informing public health policy and future preparedness needs.1 However, separating the deaths that occurred from COVID-19 versus those from all other causes is challenging. Essentially, the unknowns are the counterfactual, should an infection wave not have happened. A solution to this challenge is to estimate expected number of individuals who would have died and compare this with the observed number of deaths. The estimation of expected number of deaths must consider changes in population and seasonal dynamics and be based on an appropriate reference period. Full-Text PDF Conflicting COVID-19 excess mortality estimatesAlthough the global review of excess deaths from the COVID-19 pandemic might seem to give authorities in some countries encouragement for their policies,1 it seems unfortunate that a key vulnerable group was missing from discussion in the paper. Full-Text PDF Conflicting COVID-19 excess mortality estimatesTo estimate the COVID-19 death toll, the COVID-19 Excess Mortality Collaborators1 have presented excess mortality estimates for 2020–21 for all countries in the world. We argue that for many countries, these estimates are implausible because they imply an unrealistic number of expected deaths, inconsistent with trends before the pandemic. A case in point is Japan, where the authors estimated 111 000 (95% CI 103 000–116 000) excess deaths from Jan 1, 2020, to Dec 31, 2021—an order of magnitude higher than the estimate by The Economist2 (12 000) and qualitatively different from the World Mortality Dataset's3 negative estimate (–13 100). Full-Text PDF Estimating excess mortality due to the COVID-19 pandemic: a systematic analysis of COVID-19-related mortality, 2020–21The full impact of the pandemic has been much greater than what is indicated by reported deaths due to COVID-19 alone. Strengthening death registration systems around the world, long understood to be crucial to global public health strategy, is necessary for improved monitoring of this pandemic and future pandemics. In addition, further research is warranted to help distinguish the proportion of excess mortality that was directly caused by SARS-CoV-2 infection and the changes in causes of death as an indirect consequence of the pandemic. Full-Text PDF Open Access