Reliability of the health care delivery system

Abstract

In this issue of The Journal, Cloutier et al1Cloutier M.M. Hall C.B. Wakefield D.B. Bailit H. Use of asthma guidelines by primary care providers to reduce hospitalizations and emergency department visits in poor, minority, urban children.J Pediatr. 2005; 146: 591-597Abstract Full Text Full Text PDF PubMed Scopus (146) Google Scholar describe their efforts to improve quality of care delivered to a poor, urban, pediatric asthma population. Cloutier et al have outlined the impact of a locally developed guideline and associated treatment plan in six primary care sites that care for 85% of the children in Hartford, Connecticut. The primary outcome measures were hospital admissions and emergency department visits. They also tracked inhaled corticosteroid usage. Using Medicaid claims data, children were compared before and after enrollment in the program. Several significant differences were noted after enrollment: a 35% decrease in hospitalizations, a 27% decrease in emergency department visits, and a 25% increase in claims for inhaled corticosteroids. The study was neither randomized nor masked; however, the use of each child as his or her own control and the use of a time-adjusted control group mitigate these concerns. In addition, because the evidence is fairly convincing and the risk of this intervention is minimal, it is reasonable for the authors to take this approach to improve the care delivery system. The results are effectively displayed in a 4-year run chart. We expect to see run charts and statistical process control charts used more commonly as part of delivery system improvement.Flores et al2Flores G. Lee M. Bauchner H. Kastner B. Pediatricians' attitudes, beliefs, and practices regarding clinical practice guidelines: a national survey.Pediatrics. 2000; 105: 496-501Crossref PubMed Scopus (172) Google Scholar have documented a wide range of willingness among pediatricians to use evidence-based guidelines developed at the national level. Muething et al3Muething S. Schoettker P.J. Gerhardt W.E. Atherton H.D. Britto M.T. Kotagal U.R. Decreasing overuse of therapies in the treatment of bronchiolitis by incorporating evidence at the point of care.J Pediatr. 2004; 144: 703-710PubMed Scopus (40) Google Scholar previously showed that guidelines with clear diagnostic criteria and simple point-of-care tools developed locally decrease unwarranted variation, increase adherence to evidence-based practice, and improve outcomes. This study by Cloutier et al confirms these findings and extends the experience to an outpatient setting for a high-risk population.Asthma is the most common chronic condition affecting children. The condition is particularly prevalent in poor, urban, minority populations, affecting as many as one-third of such children. Cloutier's findings have added significance for immediate application. Mannino et al,4Mannino D.M. Homa D.M. Akinbami L.J. Moorman J.E. Gwynn C. Redd S.C. Surveillance for asthma—United States, 1980-1999.MMWR Surveill Summ. 2002; 29: 1-13Google Scholar analyzing Centers for Disease Control data, reported a national rate for asthma hospitalizations of 55.4/10,000 children 0 to 4 years of age. For urban populations, Blaisdell et al5Blaisdell C.J. Weiss S.R. Kimes D.S. Levine E.R. Myers M. Timmins S. et al.Using seasonal variations in asthma hospitalizations in children to predict hospitalization frequency.J Asthma. 2002; 39: 567-575Crossref PubMed Scopus (38) Google Scholar reported an asthma hospitalization rate of 134/10,000 children 0 to 4 years of age in Baltimore. Although the exact cost of the program described in Cloutier et al's study is not reported, the intervention costs appear to be relatively low. The potential for a significant reduction in hospitalizations and emergency department visits for such a high-risk group makes the findings even more compelling.Of note, however, is what the results also highlight: the reliability of the delivery system in place. Three years after introducing this program to the providers in these six clinics approximately half of the children believed to have asthma had been enrolled. This finding is consistent with measures of reliability of the US healthcare delivery system in general. In a national survey, McGlynn et al6McGlynn E.A. Asch S.M. Adams J. Keesey J. Hicks J. DeCristofaro A. et al.The quality of health care delivered to adults in the United States.N Engl J Med. 2003; 48: 2635-2645Crossref Scopus (3911) Google Scholar reported patients received only 55% of recommended care and preventative therapy. Although this study focused on adults, it is reasonable to believe the reliability of care delivery for children is similar. Berwick and Nolan defined reliability for healthcare as, “the capability of a process, procedure or health service to perform its intended function in the required time under existing conditions.”7Berwick D, Nolan T. High reliability health care. Presented at the Institute for Healthcare Improvement's 15th Annual National Forum on Quality Improvement in Health Care, December 2003 in New Orleans, Louisiana. Available at: http://www.ihi.org/IHI/Topics/Reliability/ReliabilityGeneral/EmergingContent/HighReliabilityHealthCarePresentation.htm.Google Scholar The Cloutier et al and McGlynn et al studies demonstrate delivery systems with a reliability of approximately 50%, or error rates of 5 of 10.Weick and Sutcliffe8Weick K.E. Sutcliffe K.M. Managing the Unexpected: Assuring High Performance in an Age of Complexity. Jossey-Bass, San Francisco2001Google Scholar have described the attributes of “High Reliability Organizations” and have outlined the key principles of the system design required to achieve the desired level of performance reliability. Different levels of reliability will require different design principles. Providers working to improve a delivery system for evidence-based asthma care in children may have a goal of 90% reliability, or an error rate of 1 of 10 (Level 1 reliability). A system designed to prevent postoperative infections may establish a goal of 99% reliability, equal to an infection or error in 1 of 100 patients (Level 2 reliability). Hospital systems designed to prevent adverse drug events will seek error rates of 1 of 1,000 patients or reliability of 99.9% (Level 3 reliability). This concept of system design linked to system reliability is fundamental to reliability science. Berwick and Nolan7Berwick D, Nolan T. High reliability health care. Presented at the Institute for Healthcare Improvement's 15th Annual National Forum on Quality Improvement in Health Care, December 2003 in New Orleans, Louisiana. Available at: http://www.ihi.org/IHI/Topics/Reliability/ReliabilityGeneral/EmergingContent/HighReliabilityHealthCarePresentation.htm.Google Scholar have outlined the design principles necessary to reach Level 1 or 90% reliability in healthcare delivery systems: standardization of approach, awareness and training, feedback of data, and reminders. A Level 1 evidence-based system that includes agreement by the providers to use the standard protocol whenever appropriate, monthly or quarterly feedback of results to providers, reminder stickers on charts of children with asthma, and knowledge of the protocol by all staff, as well as patients and families, will attain 90% reliability.Higher levels of reliability will not be achieved by merely expecting more of the current system; it will require redesign. Level 2 reliability principles include: checklists, redundancy, making the desired action the default, and real-time identification of errors. A practice achieving 97% reliability on an evidence-based asthma protocol will likely have widespread use of standard order sets or prescriptions, staff tracking all asthma patients using an electronic registry, and patients automatically receiving appropriate education unless otherwise ordered by the provider.As knowledge of reliability science increases in the healthcare field, we can expect to see a growing body of work demonstrating improved outcomes for patients based on redesign of systems. The addition of reliability science to the underlying principles of evidence-based care already outlined in this manuscript will help further reduce unnecessary hospitalizations. We call on pediatric organizations and practices to use these concepts in care for children with asthma. In this issue of The Journal, Cloutier et al1Cloutier M.M. Hall C.B. Wakefield D.B. Bailit H. Use of asthma guidelines by primary care providers to reduce hospitalizations and emergency department visits in poor, minority, urban children.J Pediatr. 2005; 146: 591-597Abstract Full Text Full Text PDF PubMed Scopus (146) Google Scholar describe their efforts to improve quality of care delivered to a poor, urban, pediatric asthma population. Cloutier et al have outlined the impact of a locally developed guideline and associated treatment plan in six primary care sites that care for 85% of the children in Hartford, Connecticut. The primary outcome measures were hospital admissions and emergency department visits. They also tracked inhaled corticosteroid usage. Using Medicaid claims data, children were compared before and after enrollment in the program. Several significant differences were noted after enrollment: a 35% decrease in hospitalizations, a 27% decrease in emergency department visits, and a 25% increase in claims for inhaled corticosteroids. The study was neither randomized nor masked; however, the use of each child as his or her own control and the use of a time-adjusted control group mitigate these concerns. In addition, because the evidence is fairly convincing and the risk of this intervention is minimal, it is reasonable for the authors to take this approach to improve the care delivery system. The results are effectively displayed in a 4-year run chart. We expect to see run charts and statistical process control charts used more commonly as part of delivery system improvement. Flores et al2Flores G. Lee M. Bauchner H. Kastner B. Pediatricians' attitudes, beliefs, and practices regarding clinical practice guidelines: a national survey.Pediatrics. 2000; 105: 496-501Crossref PubMed Scopus (172) Google Scholar have documented a wide range of willingness among pediatricians to use evidence-based guidelines developed at the national level. Muething et al3Muething S. Schoettker P.J. Gerhardt W.E. Atherton H.D. Britto M.T. Kotagal U.R. Decreasing overuse of therapies in the treatment of bronchiolitis by incorporating evidence at the point of care.J Pediatr. 2004; 144: 703-710PubMed Scopus (40) Google Scholar previously showed that guidelines with clear diagnostic criteria and simple point-of-care tools developed locally decrease unwarranted variation, increase adherence to evidence-based practice, and improve outcomes. This study by Cloutier et al confirms these findings and extends the experience to an outpatient setting for a high-risk population. Asthma is the most common chronic condition affecting children. The condition is particularly prevalent in poor, urban, minority populations, affecting as many as one-third of such children. Cloutier's findings have added significance for immediate application. Mannino et al,4Mannino D.M. Homa D.M. Akinbami L.J. Moorman J.E. Gwynn C. Redd S.C. Surveillance for asthma—United States, 1980-1999.MMWR Surveill Summ. 2002; 29: 1-13Google Scholar analyzing Centers for Disease Control data, reported a national rate for asthma hospitalizations of 55.4/10,000 children 0 to 4 years of age. For urban populations, Blaisdell et al5Blaisdell C.J. Weiss S.R. Kimes D.S. Levine E.R. Myers M. Timmins S. et al.Using seasonal variations in asthma hospitalizations in children to predict hospitalization frequency.J Asthma. 2002; 39: 567-575Crossref PubMed Scopus (38) Google Scholar reported an asthma hospitalization rate of 134/10,000 children 0 to 4 years of age in Baltimore. Although the exact cost of the program described in Cloutier et al's study is not reported, the intervention costs appear to be relatively low. The potential for a significant reduction in hospitalizations and emergency department visits for such a high-risk group makes the findings even more compelling. Of note, however, is what the results also highlight: the reliability of the delivery system in place. Three years after introducing this program to the providers in these six clinics approximately half of the children believed to have asthma had been enrolled. This finding is consistent with measures of reliability of the US healthcare delivery system in general. In a national survey, McGlynn et al6McGlynn E.A. Asch S.M. Adams J. Keesey J. Hicks J. DeCristofaro A. et al.The quality of health care delivered to adults in the United States.N Engl J Med. 2003; 48: 2635-2645Crossref Scopus (3911) Google Scholar reported patients received only 55% of recommended care and preventative therapy. Although this study focused on adults, it is reasonable to believe the reliability of care delivery for children is similar. Berwick and Nolan defined reliability for healthcare as, “the capability of a process, procedure or health service to perform its intended function in the required time under existing conditions.”7Berwick D, Nolan T. High reliability health care. Presented at the Institute for Healthcare Improvement's 15th Annual National Forum on Quality Improvement in Health Care, December 2003 in New Orleans, Louisiana. Available at: http://www.ihi.org/IHI/Topics/Reliability/ReliabilityGeneral/EmergingContent/HighReliabilityHealthCarePresentation.htm.Google Scholar The Cloutier et al and McGlynn et al studies demonstrate delivery systems with a reliability of approximately 50%, or error rates of 5 of 10. Weick and Sutcliffe8Weick K.E. Sutcliffe K.M. Managing the Unexpected: Assuring High Performance in an Age of Complexity. Jossey-Bass, San Francisco2001Google Scholar have described the attributes of “High Reliability Organizations” and have outlined the key principles of the system design required to achieve the desired level of performance reliability. Different levels of reliability will require different design principles. Providers working to improve a delivery system for evidence-based asthma care in children may have a goal of 90% reliability, or an error rate of 1 of 10 (Level 1 reliability). A system designed to prevent postoperative infections may establish a goal of 99% reliability, equal to an infection or error in 1 of 100 patients (Level 2 reliability). Hospital systems designed to prevent adverse drug events will seek error rates of 1 of 1,000 patients or reliability of 99.9% (Level 3 reliability). This concept of system design linked to system reliability is fundamental to reliability science. Berwick and Nolan7Berwick D, Nolan T. High reliability health care. Presented at the Institute for Healthcare Improvement's 15th Annual National Forum on Quality Improvement in Health Care, December 2003 in New Orleans, Louisiana. Available at: http://www.ihi.org/IHI/Topics/Reliability/ReliabilityGeneral/EmergingContent/HighReliabilityHealthCarePresentation.htm.Google Scholar have outlined the design principles necessary to reach Level 1 or 90% reliability in healthcare delivery systems: standardization of approach, awareness and training, feedback of data, and reminders. A Level 1 evidence-based system that includes agreement by the providers to use the standard protocol whenever appropriate, monthly or quarterly feedback of results to providers, reminder stickers on charts of children with asthma, and knowledge of the protocol by all staff, as well as patients and families, will attain 90% reliability. Higher levels of reliability will not be achieved by merely expecting more of the current system; it will require redesign. Level 2 reliability principles include: checklists, redundancy, making the desired action the default, and real-time identification of errors. A practice achieving 97% reliability on an evidence-based asthma protocol will likely have widespread use of standard order sets or prescriptions, staff tracking all asthma patients using an electronic registry, and patients automatically receiving appropriate education unless otherwise ordered by the provider. As knowledge of reliability science increases in the healthcare field, we can expect to see a growing body of work demonstrating improved outcomes for patients based on redesign of systems. The addition of reliability science to the underlying principles of evidence-based care already outlined in this manuscript will help further reduce unnecessary hospitalizations. We call on pediatric organizations and practices to use these concepts in care for children with asthma. Use of asthma guidelines by primary care providers to reduce hospitalizations and emergency department visits in poor, minority, urban childrenThe Journal of PediatricsVol. 146Issue 5PreviewTo determine whether an organized, citywide asthma management program delivered by primary care providers (PCPs) increases adherence to the National Asthma Education and Prevention Program (NAEPP) Asthma Guidelines and whether adherence to the guidelines by PCPs decreases medical services utilization in low-income, minority children. Full-Text PDF