PERFORMANCE MEASURES ARE CREATED AND USED TO improve clinical outcomes, so it is important to know which measures are associated with better outcomes. The study by Werner and Bradlow in this issue of JAMA evaluates the association between performance measures and clinical outcomes in routine hospital care. Despite their large sample size and the use of numerous statistical adjustments and sensitivity analyses, the authors found only small statistically significant or nonsignificant associations between rates of application of performance measures and patient mortality rates for acute myocardial infarction, heart failure, and pneumonia in US hospitals. Even when the associations were statistically significant, the clinical importance was questionable because the mortality rate when a performance measure was applied frequently differed only slightly from the rate when the performance measure was applied infrequently. The authors recognize that “ . . . Hospital Compare measures discrete aspects of care delivery rather than assessing global quality. While these measures have been tightly linked to patient outcomes in clinical trials and are included in clinical practice guidelines [developed by medical experts], riskadjusted mortality rates are likely influenced by many factors that are independent of these measures . . . .” Although their results suggest slightly larger differences in mortality rates between highand low-performance hospitals when all the performance measures are combined (Table 4 in the article), the authors acknowledge that these findings still “capture information about only a small portion of the overall care delivered during a hospital stay.” The results of this study raise questions about the appropriateness of using Hospital Compare performance measures as the basis either for pay-for-performance systems or for consumers to identify better-quality hospitals. If performance measures are not strongly associated with better outcomes, why should clinicians and health care centers be required to collect and submit the data, and why would payers and consumers want to act on them? Perhaps just as important, these findings also raise a more fundamental question: Why are interventions that were found to be effective in randomized controlled trials (RCTs) not associated with substantially better outcomes in actual practice? There are several possible reasons. Inadequate measures of patient characteristics and severity of illness might have been used as statistical controls in the analyses. Findings from homogeneous RCT study samples are not applicable to all patients. The complexity of the care process is not modeled adequately in RCTs, and the analyses reported by the authors may reflect an ecological fallacy (ie, inferences about individuals are based on aggregate statistics collected for the group to which those individuals belong) because the authors’ unit of analysis is the hospital, not the patient. Improving clinical outcomes in actual practice is complex, multidimensional, and likely much more difficult than simply using a few interventions that were found to be statistically significant in RCTs involving a single condition in a homogeneous patient population. Many medical professionals view findings from RCTs as the strongest form of evidence to guide practice (the gold standard); evidence from other study designs such as observational studies may be viewed as less rigorous and less reliable. Randomized controlled trials usually focus on the efficacy of one treatment for one outcome in a controlled and homogeneous group of patients. A performance measure that appeared to improve outcomes for patients in an RCT may not do as well in a heterogeneous group of patients who also are receiving treatments for comorbid conditions that were excluded from an RCT. In routine practice, many combinations of patient and treatment variables affect outcomes, so a much more comprehensive approach is needed to discover how to improve quality of care. To discover inputs associated with meaningful improvements in clinical outcomes in the real world, additional gold standards and conceptual models are needed beyond information in existing RCTs or a limited set of process measures determined from RCTs or practice guidelines. An alternative approach is to use data that represent the population and circumstances of interest; however, because these data are not obtained using participant randomization, it can be difficult to determine whether different outcomes can be attributed to different treatments or other