Improving the Science and Politics of Quality Improvement

Abstract

QUALITY AND SAFETY ARE CRITICAL ISSUES FOR health care systems around the world and have appropriately been highlighted in ongoing efforts to reform and improve health care. Care of critically ill patients is a domain in which issues of quality and safety take on monumental importance because of the severity of illness and the complexity of providing highquality critical care. In this issue of JAMA, Scales and colleagues report the results of an important study designed to evaluate the effectiveness of quality improvement efforts in communitybased critical care units in Ontario. The authors use a multifaceted “knowledge transfer” intervention including education, dissemination of algorithms, and audit and feedback, transferred through an interactive telecommunication strategy. The goal of the intervention was to increase adherence to 6 quality measures that have been documented to improve patient outcomes: prevention of ventilator-associated pneumonia (VAP), prophylaxis for deep venous thrombosis (DVT), daily spontaneous breathing trials, prevention of catheter-related bloodstream infections, early enteral feeding, and prevention of decubitus ulcers. Although debate remains about some of the evidence supporting these measures, there is general consensus that appropriate implementation of each measure can decrease risk of harm for critically ill patients. Numerous studies have investigated the value and implementation of these quality measures, but the study by Scales and colleagues has several notable features. First, the design and implementation reflect state-of-the art methods for a study evaluating quality improvement and advancing the state of science. Second, the study design, knowledge transfer intervention, and analyses are extremely complex but appropriate to the complex phenomenon under study. Third, the results were positive, documenting significant improvement in quality, although benefits were modest. Fourth, this study—with its state-of-the-art, complex methods—was funded by the Ontario health care delivery system rather than a research funding agency, which has important policy implications for improving health care in North America and beyond. Each of these 4 points warrants further discussion. Many studies have been designed to improve health care quality, but relatively few are randomized trials. Although much can be learned from nonrandomized and nonexperimental studies, the authors’ use of a randomized design with an appropriate control group improves their ability to control for bias and greatly enhances the ability to draw causal inferences. An observational study with historical controls would be unable to exclude temporal change as an explanation for the findings. These authors also addressed the problems that result from having the control group receive no intervention by using the same knowledge transfer intervention targeting different quality measures in pairs. Each intensive care unit was randomly assigned to simultaneously be the experimental group receiving the knowledge transfer intervention for one quality measure (eg, VAP prevention) and the control group receiving no knowledge transfer intervention for the other measure of the pair (eg, DVT prophylaxis). Additional innovative methods included a “decay-monitoring period” demonstrating that the improvements persist and qualitative interviews with participants to explore mechanisms of action and facilitators of success. Another feature of this study is its complexity. Rather than using the standard scientific method of changing one variable and examining its effect, the authors designed a multifaceted knowledge transfer intervention. Yet this complexity is not only appropriate; it is essential. Years of research suggest that multifaceted interventions are needed to change clinician behavior and transfer knowledge of best practices into the complicated health care system. The cluster randomized design is appropriate for an intervention targeting the hospital, yet requires complex statistical approaches to account for clustering by center, can significantly reduce effective sample sizes (if intracluster correlations are high), and substantially reduces effectiveness of randomization, thereby increasing risk of randomization imbal-