Estimating the Type II error of detecting changes in foodborne illnesses via public health surveillance

Abstract

Many countries operate public health surveillance systems to monitor foodborne disease occurrence. The illness counts for pathogens of interest are monitored across time to assess if changes have occurred in case rates or the overall illness burden. Common to all of these systems is that only a fraction of all foodborne illnesses are reported and the relationship between observed and the total number of illnesses is uncertain.Food-safety policies intend to affect both the total and observed number of illnesses. Ideally, the surveillance system would be sufficiently sensitive to detect these changes, but the statistical power of these systems is generally not well understood. This study proposes two approaches for estimating the power of a foodborne illness surveillance system. These methods are then applied to assess the power of detecting specified changes in the total number of illnesses in an existing surveillance system.The findings suggest that the power of a national foodborne disease surveillance system to detect modest annual reductions in Salmonella illnesses may be limited. For example, a naïve presumptive model that assumes observed illnesses are reduced directly by 10% predicts that the power to discern this difference, from one year to the next, is 0.21. A Bayesian model, that accounts for uncertainty in projecting changes in total illnesses to the number of observed illnesses, predicts that the power to detect a true 10% reduction is 0.04 (i.e., 4% confidence of detecting this magnitude of reduction or 96% chance of a Type II error) one year after the change has taken effect. Although the power of a national surveillance system increases as the magnitude of reductions increases – or as the number of years the reduction is maintained increases – the Bayesian model demonstrates that power is less than 0.5 for reductions up to 50% for one year and is less than 0.53 for reductions of 30% that are maintained for four years.The limited power to detect intended changes in total annual illnesses of national public health surveillance systems may highlight the need for regulators to also monitor food contamination evidence to gauge progress towards achieving intended policy effects.