Forecasting rare disease outbreaks from open source indicators

Abstract

BackgroundRapidly increasing volumes of news feeds from diverse data sources, such as online newspapers, Twitter, and online blogs, are proving to be extremely valuable resources in helping to anticipate, detect, and forecast outbreaks of rare diseases. The goal of this paper is to develop techniques that can effectively forecast the emergence and progression of rare infectious diseases by combining data from disparate data sources.MethodsWe introduce SourceSeer, a novel algorithmic framework that combines spatiotemporal topic models with source‐based anomaly detection techniques. SourceSeer is capable of discovering the location focus of each source, allowing sources to be used as experts with varying degrees of authoritativeness. To fuse the individual source predictions into a final outbreak prediction, we employ a multiplicative weights algorithm taking into account the accuracy of each source.ResultsWe evaluate the performance of SourceSeer using incidence data for hantavirus syndromes in multiple countries of Latin America provided by HealthMap over a time span of 15 months. We demonstrate that SourceSeer makes predictions of increased accuracy compared to several baselines and can forecast disease outbreaks in a timely manner even when no outbreaks were previously reported.