A controlled evaluation of case clinical effect coding by poison center specialists for detection of WMD scenarios

Abstract

Background. Many public health entities employ computer-based syndromic surveillance to monitor for aberrations including possible exposures to weapons of mass destruction (WMD). Often, this is done by screening signs and symptoms reported for cases against syndromic definitions. Poison centers (PCs) may offer significant contributions to public health surveillance because of their detailed clinical effect data field coding and real-time data entry. Because improper clinical effect coding may impede syndromic surveillance, it is important to assess this accuracy for PCs. Methods. An AAPCC-certified regional PC assessed the accuracy of clinical effect coding by specialists in poison information (SPIs) listening to audio recordings of standard cases. Eighteen different standardized cases were used, consisting of six cyanide, six botulism, and six control cases. Cases were scripted to simulate clinically relevant telephone conversations and converted to audio recordings. Ten SPIs were randomly selected from the center's staff to listen to and code case information from the recorded cases. Kappa scores and the percentage of correctly coding a present clinical effect were calculated for individual clinical effects summed over all test cases along with corresponding 95% confidence intervals. The rate of the case coding by the SPIs triggering the PC's automated botulism and cyanide alerts was also determined. Results. The kappa scores and the percentage of correctly coding a present clinical effect varied depending on the specific clinical effect, with greater accuracy observed for the clinical effects of vomiting and agitation/irritability, and poor accuracy observed for the clinical effects of visual defect and anion gap increase. Lack of correct coding resulted in only 60 and 86% of the cases that met the botulism and cyanide surveillance definitions, respectively, triggering the corresponding alert. There was no difference observed in the percentage of coding a present clinical effect between certified (9.0 years experience) and non-certified (2.4 years experience) specialists. There were no cases of coding errors that resulted in the triggering of a false positive alert. Conclusion. The success of syndromic surveillance depends on accurate coding of signs and symptoms. Although PCs generally contribute high-quality data to public health surveillance, it is important to recognize this potential weak link in surveillance methods.