A Framework for Health Care Organizations to Develop and Evaluate a Safety Scorecard

Abstract

THE DEMAND TO IMPROVE PATIENT SAFETY IS INCREASing within health care organizations. Boards of trustees have a fiduciary responsibility to ensure patient safety, and senior management is often charged with evaluating and improving patient safety. External agencies such as the Centers for Medicare & Medicaid Services (CMS), the Leapfrog Group, and the Joint Commission have developed measures to evaluate patient safety and quality of care. Many hospitals have responded to this heightened focus on patient safety by creating scorecards to evaluate and publicly report progress in improving quality and safety. Scorecards are attractive because hospital leaders and other interested parties can quickly obtain a broad overview of patient safety performance. These scorecards tend to include measures required by the CMS, The Joint Commission, and insurers, as well as measures developed by individual hospitals for local improvement. Scorecards are increasingly used to evaluate the performance of physicians and senior management. Despite this increasing interest in scorecards, the science of measuring patient safety is immature. Many health care organizations lack scientifically sound measures to evaluate their progress toward improving patient safety and quality. Measures should be important, scientifically sound (valid and reliable), useful, and feasible. A model to differentiate measures that evaluate progress in patient safety and those that identify hazards has been described. This model includes 2 rate-based measures (how often patients experience harm and how often they receive evidence-based interventions) and 2 non–rate-based measures (whether an organization learns from its mistakes and whether it has a culture of safety). Because of substantial measurement and selection bias, measures obtained from patient safety reporting systems should not be used to evaluate progress in patient safety. Rather, these systems help to identify hazards, and the measurement should focus on whether the organization reduced the risk to future patients. This Commentary presents a potential framework to help health care organizations develop their safety scorecard, evaluate its validity, and understand measures appropriate to present as rates. The term “safety scorecard” is used while acknowledging an overlap between quality and safety. This framework is based on the premise that the goal of the scorecard is to monitor progress in improving patient safety over time or relative to a benchmark. Organizations need to stop conceptualizing safety as a dichotomous variable (ie, safe or unsafe) and start viewing safety as a continuous variable (ie, is it improving?).