The California Hospital Outcomes Project: How Useful Is California’s Report Card for Quality Improvement?

Abstract

Hospital report cards have proliferated in the 1990s but remain controversial because risk-adjusted outcomes measures are complex and have uncertain validity. Despite this controversy, little is known about their value and impact. A two-stage survey of hospital leaders in California was undertaken in September 1996 and July 1997 to explore how the 1996 reports and data from the California Hospital Outcomes Project (CHOP) were used to improve organizations' performance. In the first stage, a questionnaire was mailed to the chief executive officer of each hospital in the report. In the second stage, a stratified random sample of the respondents who indicated a willingness to provide further information was interviewed. Thirty-nine interviews were completed, representing 87% yield after replacing informants who failed to return six messages. About three-quarters of the interviewees found some aspect of the CHOP report to be useful, especially for benchmarking performance, improving ICD-9-CM (International Classification of Diseases, 9th Revision, Clinical Modification) coding, and educating physicians about documentation and clinical pathways. The most common criticisms were that the reports were not timely and described death rates without providing practical information about the process of care. Although the 1996 CHOP reports and data were widely disseminated within hospitals, most reported uses did not directly affect the process of care for patients with acute myocardial infarction. This finding reflects two critical weaknesses of the project--nontimely data and lack of information about the process of care. Nevertheless, hospital quality managers recognize that public report cards are here to stay, and some carefully studied their outcomes data to identify areas for improvement.