Surveillance Bias in Outcomes Reporting

Abstract

THE PREVALENCE AND EXTENT OF PUBLIC REPORTING of adverse medical outcomes are increasing. Many private, public, and government Web sites rank hospitals and report scores on selected quality measures. Health care consumers, including referring physicians, individual patients, and insurers, can use these data to inform decision making by selecting hospitals with better outcomes. However, the science of outcome reporting is young and lags behind the desires of the public in this information age. Reporting quality measures may have benefits but also may pose risks from unintended consequences. When validating outcome measures, the main focus has been strict definitions for numerators to clearly identify cases and for denominators to identify patients at risk. Standardized surveillance for events within the population of those at risk has received little attention, and as a result, surveillance bias is likely an important source of error in currently reported outcome measures. Surveillance bias, a nonrandom type of information bias, refers to the idea that “the more you look, the more you find.” It occurs when some patients are followed up more closely or have more diagnostic tests performed than others, often leading to an outcome diagnosed more frequently in the more closely monitored group. In an article on biases inherent to clinical research, Sackett used the phrase “unmasking (detection signal) bias” to explain how “an innocent exposure may become suspect if, rather than causing a disease, it causes a sign or symptom which precipitates a search for the disease.” As a result, differences in outcomes may be related to surveillance bias rather than differences in quality. If ignored, flawed causal inferences could be suggested from differential rates identified between groups. Surveillance bias is a well-known concept in epidemiology yet is seldom considered in published clinical studies. For example, deep vein thrombosis (DVT) is a significant cause of preventable harm and a commonly monitored quality-of-care measure. DVT is a common, lifethreatening complication among patients who have sustained trauma. Because injured patients are at increased risk for DVT, some clinicians use duplex ultrasound to screen highrisk asymptomatic trauma patients for DVT. Other clinicians argue this approach is neither clinically necessary nor cost-effective and therefore do not routinely screen for DVT in trauma patients. This clinical uncertainly leads to variability in the use of screening duplex ultrasound, creating variability in rates of DVT identified and reported—a typical example of surveillance bias. Evidence for surveillance bias in DVT reporting after trauma is well documented. For instance, after implementation of a DVT screening guideline at one trauma center, duplex ultrasound rates increased 4-fold and DVT rates increased 10-fold. Within the National Trauma Data Bank, DVT rates were 7-fold higher at hospitals in the highest quartile of use of vascular ultrasound, and patients treated at hospitals that performed more duplex ultrasounds were twice as likely to have DVT reported, even controlling for other patient risk factors. Surveillance bias also is an important factor in other highprofile publicly reported outcome measures. For example, rates of central line–associated bloodstream infections (BSIs) increased 3-fold with the use of computer automated surveillance, suggesting that “surveillance practice may complicate interinstitutional comparisons of publicly reported central line–associated BSI rates.” The hazard of error caused by surveillance bias will likely increase with increasing use of penalties and rewards for performance on quality measures. Surveillance bias also has the potential to pose significant harms. For instance, patients may be harmed because clinicians will not know if quality of care is improving, and incentives to improve outcomes may encourage clinicians to avoid appropriate diagnostic testing to minimize reported complications. Because performance measures do not specify surveillance, outcomes that are not sought ordinarily will not be detected. This potential for unintended consequences was summarized in a comment by Alam and Velmahos as: “No screening, no DVT, no punishment.” Thus,