Comparison of statistical algorithms for syndromic surveillance aberration detection

Abstract

ObjectiveTo investigate whether alternative statistical approaches can improve daily aberration detection using syndromic surveillance in England.IntroductionSyndromic surveillance involves monitoring big health datasets to provide early warning of threats to public health. Public health authorities use statistical detection algorithms to interrogate these datasets for aberrations that are indicative of emerging threats. The algorithm currently in use at Public Health England (PHE) for syndromic surveillance is the ‘rising activity, multi-level mixed effects, indicator emphasis’ (RAMMIE) method (Morbey et al, 2015), which fits a mixed model to counts of syndromes on a daily basis. This research checks whether the RAMMIE method works across a range of public health scenarios and how it compares to alternative methods.MethodsFor this purpose, we compare RAMMIE to the improved quasi-Poisson regression-based approach (Noufaily et al, 2013), currently implemented at PHE for weekly infectious disease laboratory surveillance, and to the Early Aberration Reporting System (EARS) method (Rossi et al, 1999), which is used for syndromic surveillance aberration detection in many other countries. We model syndromic datasets, capturing real data aspects such as long-term trends, seasonality, public holidays, and day-of-the-week effects, with or without added outbreaks. Then, we compute the sensitivity and specificity to compare how well each of the algorithms detects synthetic outbreaks to provide recommendations for the most suitable statistical methods to use during different public health scenarios.ResultsPreliminary results suggest all methods provide high sensitivity and specificity, with the (Noufaily et al, 2013) approach having the highest sensitivity and specificity. We showed that for syndromes with long-term increasing trends, RAMMIE required modificaiton to prevent excess false alarms. Also, our study suggests further work is needed to fully account for public holidays and day-of-the-week effects.ConclusionsOur study will provide recommendations for which algorithm is most effective for PHE's syndromic surveillance for a range of different syndromes. Furthermore our work to generate standardised synthetic syndromic datasets and a range of outbreaks can be used for future evaluations in England and elsewhere.ReferencesNoufaily, A., Enki, D. G., Farrington, C. P., Garthwaite, P., Andrews, N. and Charlett, A. (2013). An Improved Algorithm for Outbreak Detection in Multiple Surveillance Systems. Statistics in Medicine, 32(7), 1206-1222.Morbey, R. A., Elliot, A. J., Charlett, A., Verlander, A. Q, Andrews, N. and Smith, G. (2013). The application of a novel ‘rising activity, multi-level mixed effects, indicator emphasis’ (RAMMIE) method for syndromic surveillance in England, Bioinformatics, 31(22), 3660-3665.Rossi, G, Lampugnani, L, Marchi, M. (1999), An approximate CUSUM procedure for surveillance of health events. Statistics in Medicine, 18, 2111–2122