Informing risk-mitigation priorities using uncertainty measures derived from heterogeneous expert panels: A demonstration using foodborne pathogens

Abstract

This article looks at a new approach to expert elicitation that combines basic elements of conventional expert elicitation protocols with formal survey methods and larger, heterogeneous expert panels. This approach is appropriate where the hazard-estimation task requires a wide range of expertise and professional experience. The ability to judge when to rely on alternative data sources often is critical for successful risk management. We show how a large, heterogeneous sample can support internal validation of not only the experts’ assessments but also prior information that is based on limited historical data. We illustrate the use of this new approach to expert elicitation by addressing a fundamental problem in US food safety management, obtaining comparable food system-wide estimates of the foodborne illness by pathogen–food pair and by food. The only comprehensive basis for food-level hazard analysis throughout the US food supply currently available is outbreak data (i.e., when two or more people become ill from the same food source), but there is good reason to question the portrayal that outbreak data alone gives of food risk. In this paper, we compare results of food and pathogen–food incidence estimates based on expert judgment and based on outbreak data, and we demonstrate a suite of uncertainty measures that allow for a fuller understanding of the results.