Trade-offs between individual and ensemble forecasts of an emerging infectious disease

Rachel J Oidtman,John H Huber,Manuel García-Herranz,Christopher M Barker,Guido España,Viviana Cañon,Sarah C Hill,Michael A Johansson,Robert C Reiner ,Sandra Patricia Misnaza ,Isabel Rodríguez-Barraquer ,Carlos Castañeda-Orjuela ,Erica Cruz-Rivera,Luz Emilse Rincón ,Amir Siraj ,Elisa Omodei,Pedro A De Alarcón ,Enrique Frı́as-Martı́nez ,Carrie Manore ,Myriam Patricia Cifuentes,Moritz U G Kraemer,T Alex Perkins

doi:10.1038/s41467-021-25695-0

Abstract

Probabilistic forecasts play an indispensable role in answering questions about the spread of newly emerged pathogens. However, uncertainties about the epidemiology of emerging pathogens can make it difficult to choose among alternative model structures and assumptions. To assess the potential for uncertainties about emerging pathogens to affect forecasts of their spread, we evaluated the performance 16 forecasting models in the context of the 2015-2016 Zika epidemic in Colombia. Each model featured a different combination of assumptions about human mobility, spatiotemporal variation in transmission potential, and the number of virus introductions. We found that which model assumptions had the most ensemble weight changed through time. We additionally identified a trade-off whereby some individual models outperformed ensemble models early in the epidemic, but on average the ensembles outperformed all individual models. Our results suggest that multiple models spanning uncertainty across alternative assumptions are necessary to obtain robust forecasts for emerging infectious diseases.

Highlights

We were able to account for the extended generation interval of ZIKV using overlapping pathogen generations across up to five weeks of the generation interval distribution of ZIKV62
We considered a suite of 16 models that spanned all combinations of four assumptions about human mobility across Colombia’s 31 mainland departments, two assumptions about the relationship between environmental conditions and the reproduction number (R), and two assumptions about how many times the Zika virus was introduced to Colombia (Table 1)
Where the first and second arguments represent the number of trials and the probabilities of the outcomes, respectively. By taking this Lagrangian approach to modeling human mobility, transmission across departments in our model can occur either by infected visitors transmitting to local susceptibles or susceptible visitors becoming infected by local infecteds, but not between infected visitors and susceptible visitors in a transient location

Summary

Results

By the time twelve weeks of data had been assimilated into the models, forecasts over the 60-week period of our analysis were considerably higher than the initial forecasts and better aligned with the observed trajectory of the epidemic (Fig. 2 second row, Supplementary Fig. 13). In departments on the Caribbean Coast that experienced intermediate epidemic sizes (e.g., Antioquia, Sucre, and Atlántico), spatially coupled models with a static R outperformed the ensemble model at forecasting the peak week by about 10% (Fig. 5a). At those same locations, the weighted ensemble performed better than or similar to those same models at forecasting peak incidence and onset week (Fig. 5b, c). This trend appeared because initial forecasts from nonspatial models were not updated until the first case appeared in each department, while initial forecasts from spatially coupled models were updated when the first case appeared in the country

Discussion

Methods

Code availability