Abstract
Mathematical models of different types and data intensities are highly used by researchers, epidemiologists, and national authorities to explore the inherently unpredictable progression of COVID-19, including the effects of different non-pharmaceutical interventions. Regardless of model complexity, forecasts of future COVID-19 infections, deaths and hospitalization are associated with large uncertainties, and critically depend on the quality of the training data, and in particular how well the recorded national or regional numbers of infections, deaths and recoveries reflect the the actual situation. In turn, this depends on, e.g., local test and abatement strategies, treatment capacities and available technologies. Other influencing factors including temperature and humidity, which are suggested by several authors to affect the spread of COVID-19 in some countries, are generally only considered by the most complex models and further serve to inflate the uncertainty. Here we use comparative and retrospective analyses to illuminate the aggregated effect of these systematic biases on ensemble-based model forecasts. We compare the actual progression of active infections across ten of the most affected countries in the world until late November 2020 with “re-forecasts” produced by two of the most commonly used model types: (i) a compartment-type, susceptible–infected–removed (SIR) model; and (ii) a statistical (Holt-Winters) time series model. We specifically examine the sensitivity of the model parameters, estimated systematically from different subsets of the data and thereby different time windows, to illustrate the associated implications for short- to medium-term forecasting and for probabilistic projections based on (single) model ensembles as inspired by, e.g., weather forecasting and climate research. Our findings portray considerable variations in forecasting skill in between the ten countries and demonstrate that individual model predictions are highly sensitive to parameter assumptions. Significant skill is generally only confirmed for short-term forecasts (up to a few weeks) with some variation across locations and periods.
Highlights
Since the earliest days of the global COVID-19 pandemic, a wide range of mathematical and epidemiological models have been proposed as means of exploring the transmission properties of the disease or as instruments for delivering indicative forecasts of, e.g., total infections, hospitalizations and mortalities, including forecast scenarios assuming combinations of different non-pharmaceutical countermeasures (Jewell et al, 2020; Li et al, 2020; Diaz-Quijano et al, 2020)
While neither of the two test models explored in our comparative analyses (Figs. 2–4, S1–S2) demonstrate superior skill in re-forecasting observed trends and variability across all ten countries, our numerical experiments, and in particular our trial ensemble forecasts, clearly suggest that COVID-19 predictions can be consistently skillful (Figs. 5–6)
The statistical extrapolation is found to be best suited for situations, where the transmission rates of the coronavirus do not change abruptly as was the case in Europe in the last months of 2020
Summary
Since the earliest days of the global COVID-19 pandemic, a wide range of mathematical and epidemiological models have been proposed as means of exploring the transmission properties of the disease or as instruments for delivering indicative forecasts of, e.g., total infections, hospitalizations and mortalities, including forecast scenarios assuming combinations of different non-pharmaceutical countermeasures (Jewell et al, 2020; Li et al, 2020; Diaz-Quijano et al, 2020). Even more to regional climate modellers, who until recently (when this practice was replaced by large collaborative multi-model intercomparison experiments (Gutowski et al, 2016)) would often evaluate (and subsequently improve) in-house regional models on the basis of lessons learned from dedicated experiments within climatic domains other than the “native” ones (Refsgaard et al, 2014) Transferring this analogy to COVID-19 modelling, this paper analyses the results of a systematic and intercomparable modelling effort, where we explore the properties of an ensemble of retrospective forecasts (“re-forecasts”) of the COVID-19 development until late November 2020 across ten of the most highly infected countries in the world and compare them to real-life records reported by, e.g., the Johns Hopkins Corona Virus Resource Centre (Johns Hopkins 2020). Given the assumed differences in COVID-19 trajectories across our suite of countries, our choice of models denote general approaches that would be applicable in all environments, model parameters are readily interpretable, and the two models represent typical modelling philosophies (epidemiological and statistical) currently used for COVID-19 predictions all over the world
Sign-in/Register to view highlights & summary 
Published Version (
Free)
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call