Comparison of statistical algorithms for daily syndromic surveillance aberration detection.

Angela Noufaily,Noel Mccarthy,Iain R Lake,Felipe J Colón-González,Roger A Morbey,Alex J Elliot,Gillian E Smith,Jonathan Wren

doi:10.1093/bioinformatics/bty997

Abstract

MotivationPublic health authorities can provide more effective and timely interventions to protect populations during health events if they have effective multi-purpose surveillance systems. These systems rely on aberration detection algorithms to identify potential threats within large datasets. Ensuring the algorithms are sensitive, specific and timely is crucial for protecting public health. Here, we evaluate the performance of three detection algorithms extensively used for syndromic surveillance: the ‘rising activity, multilevel mixed effects, indicator emphasis’ (RAMMIE) method and the improved quasi-Poisson regression-based method known as ‘Farrington Flexible’ both currently used at Public Health England, and the ‘Early Aberration Reporting System’ (EARS) method used at the US Centre for Disease Control and Prevention. We model the wide range of data structures encountered within the daily syndromic surveillance systems used by PHE. We undertake extensive simulations to identify which algorithms work best across different types of syndromes and different outbreak sizes. We evaluate RAMMIE for the first time since its introduction. Performance metrics were computed and compared in the presence of a range of simulated outbreak types that were added to baseline data.ResultsWe conclude that amongst the algorithm variants that have a high specificity (i.e. >90%), Farrington Flexible has the highest sensitivity and specificity, whereas RAMMIE has the highest probability of outbreak detection and is the most timely, typically detecting outbreaks 2–3 days earlier.Availability and implementationR codes developed for this project are available through https://github.com/FelipeJColon/AlgorithmComparisonSupplementary information Supplementary data are available at Bioinformatics online.

Highlights

Syndromic surveillance involves monitoring big health datasets to provide early warning of threats to public health
Poisson regression-based approach (Noufaily et al, 2013), currently implemented at Public Health England (PHE) for weekly infectious disease laboratory surveillance, and to the Early Aberration Reporting System (EARS) method (Rossi et al, 1999), which is used for syndromic surveillance aberration detection in many other countries
We model syndromic datasets, capturing real data aspects such as long-term trends, seasonality, public holidays, and day-of-the-week effects, with or without added outbreaks

Summary

Introduction

Syndromic surveillance involves monitoring big health datasets to provide early warning of threats to public health. Public health authorities use statistical detection algorithms to interrogate these datasets for aberrations that are indicative of emerging threats. The algorithm currently in use at Public Health England (PHE) for syndromic surveillance is the ‘rising activity, multi-level mixed effects, indicator emphasis’ (RAMMIE) method (Morbey et al, 2015), which fits a mixed model to counts of syndromes on a daily basis. This research checks whether the RAMMIE method works across a range of public health scenarios and how it compares to alternative methods

Methods

Results

Conclusions