Data management issues for emerging diseases and new tools for managing surveillance and laboratory data.

Abstract

Data Management Issues for Emerging Diseases Since 1976, when Legionnaires’ disease affected attendees at the American Legion Convention in Philadelphia (1), the scope of public health has expanded. During the 1976 outbreak investigation, public attention was drawn to news accounts of the increasing numbers of cases and deaths as well as to speculations about diseases causes and prevention. After the outbreak, public health officials contended with volumes of information, including clinical data, epidemiologic survey results, and records of specimens collected from patients and the environment. This information was managed on mainframe computers. In 1980, a cluster of cases of unrecognized illness, primarily affecting young women, created a data management situation similar to that surrounding the Legionnaires’ disease outbreak. A major epidemiologic investigation, which included examining a multitude of laboratory specimens and analyzing volumes of data, was undertaken by a large team of federal, state, and local public health officials, as well as numerous academic institutions and private industries. The problems with establishing databases and implementing a data management system for toxic shock syndrome (2) were essentially the same as the data management problems of Legionnaires’ disease, except that computer technology had crept forward slightly in public health offices. During the spring of 1993, a cluster of cases of another unknown illness, eventually attributed to hantavirus (3), occurred in the southwestern United States. The reaction to this unknown disease by public health officials reflected a startling fact: even though the epidemiologic and laboratory methods for curtailing the outbreak were in place, a consistent data management strategy had not been established. Ad hoc databases built by outbreak investigators for a multitude of purposes began to bog down the investigation. Cases were recorded in multiple databases that did not recognize duplicate reports of cases. Updates of data about cases were done in some, but not all, databases. Laboratory data about specimens from patients were not linked to other clinical and epidemiologic data about a patient. No single database was available with well-edited, complete data about all the cases. Parallel, fragmented data management efforts evolved in at least 15 locations, with no coordinated mechanism to integrate them into one system. Introducing a single system for data management in the midst of the hantavirus outbreak involved more than the data management issues encountered in the earlier outbreaks. Previously, computer technology was viewed as a solution that, although somewhat cumbersome, enabled officials to move from data management by hand to electronic management. However, during the hantavirus outbreak, computer technology became part of the problem; it initially prevented good data management and may have hindered some of the laboratory and epidemiologic efforts to control the outbreak. Data were essentially being locked into various databases and could not be adequately analyzed or merged with data in other databases. In some instances, this peculiar circumstance caused investigators to perform analyses by hand using printouts from electronic databases or entering data again into other systems. In recent years, legal considerations, such as the Privacy Act enacted in 1974 and the Freedom of Information Act enacted in 1966 (4,5), have also complicated data management. These acts, in their efforts to protect individual privacy and ensure availability of data, have in some cases, constrained public health responses to emergency situations and subsequent surveillance efforts by enforcing strict database design and handling requirements.