Sykdomspulsen One Health - A real time surveillance system in an infrastructure coping with half a million analysis a day

Abstract

Sykdomspulsen is a real time surveillance system developed by the Norwegian Institute of Public Health (NIPH) for One Health surveillance and the surveillance of other infectious diseases in humans like respiratory diseases and lately covid-19. The One Health surveillance comprise of Campylobacter data from humans and chicken farms and also includes diagnosis codes from doctor appointments and weather data with analysis forecasting outbreaks in Norway. It is a joint project between the Norwegian Institute of Public Health (NIPH) and the Norwegian Veterinary Institute (NVI), under the framework of the OHEJP NOVA (Novel approaches for design and evaluation of cost-effective surveillance across the food chain) and MATRIX (Connecting dimensions in One-Health surveillance) projects. The system relies on two pillars, the first being an analytics infrastructure which in real time retrieves data from tens of sources, cleans and harmonizes it, then runs over half a million analyses each day and produces over 20 000 000 rows of results to be used every day. The analytics infrastructure is based on R. Results are notably being used by NIPH for the monitoring of covid-19 development and the surveillance of other transmittable diseases such as influenza and gastro-intestinal illness. The analytics framework also generates hundreds of reports every day, directed at dissemination to municipal health authorities. This framework is not currently publicly available, but an open-source release is expected by the end of 2021. The second pilar is an interactive R Shiny dashboard platform, which is used for communicating the data and the model results to partner organisations. It allows for the easy creation of a website where public and animal health researchers and food safety experts can view real time analyses. This dashboard combines the powerful data visualisation and analysis strength of R with the accessibility, flexibility, structure and interactivity of web-based platforms. The result is a real time interactive surveillance system, that is supported by a solid infrastructure and streamlined data flow, and shared with actors through a beautiful and user-friendly website, based entirely on R.