Evaluation of a web-based tool for labelling potential hospital outbreaks: a mixed methods study

Abstract

Labelling outbreaks in surveillance data is necessary to train advanced analytical methods for outbreak detection, but there is a lack of software tools dedicated to this task. To evaluate the usability of a web-based tool by infection control practitioners for labelling potential outbreaks. A mixed methods design was used to evaluate how 25 experts from France and Canada interacted with a web-based application to identify potential outbreaks. Each expert used the application to retrospectively review 11-12 1-year incidence time series from 23 different types of micro-organism. The interactions between the users and the application were recorded and analysed using mixed effect models. The users' comments were analysed via qualitative methods. From the 240 reviews completed, 439 potential outbreaks were labelled, approximately half with a high probability. Significant heterogeneity was observed between users regarding their answers and behaviours (evaluation time, usage of the different options). A significant learning effect was also observed for the experts' interactions with the tool, but this did not seem to impact their answers. The content analysis of the comments highlighted the difficulty of early outbreak identification for practitioners, but also the potential utility of web applications such as that evaluated for routine surveillance. The interactive web application was both usable and useful for infection control practitioners. Its implementation in routine practice could help professionals to identify potential outbreaks while creating data to train automated detection algorithms.