Quality Improvement Feedback Research Articles

Development of a new oncology quality metric: The rate of evidence-based adherence.

84 Background: The degree to which electronic health records enhance the quality of patient care depends on how the system is used to monitor and improve practice. In planning the transition to Epic’s electronic chemotherapy ordering platform, we saw an opportunity to increase evidence-based practice. Methods: Our Chemotherapy Council vetted more than 600 electronic protocols based entirely on published clinical trials and NCCN guidelines; modifications made to these protocols represent a divergence from clinical evidence. We examined the number of times the chemotherapy section of a protocol was modified before the first cycle to avoid counting dose modifications made for toxicities. To get the rate for the cancer center we calculated the # of discrete times all protocols were used (denominator) and the number of times all protocols were modified (numerator). We subtracted the ratio from 1 to reflect the proportion of protocols that were not modified. This ratio serves as a new quality metric – the Rate of Evidence-Based Adherence (REBA). REBA institution= 1 - Total # of modifications prior to first cycle for each protocol. The total # of uses of all protocols. Additionally, REBA can be used to examine disease group rates (REBA group), individual provider rates (REBA provider) and rates for individual protocols (REBA protocol). Institutions that have implemented evidence-based clinical pathways have strived for 80% (.80) adherence. We set this as our REBA benchmark. Results: The REBA institution was .86, higher than our pre-defined goal (p<0.001). There was wide variation in the REBA group (range 0.50-0.95). The REBA protocol identified two protocols (rates of .33 and .42 respectively) that were dose reduced the majority of the time. In the example of our Breast Group, the REBA provider, ranged from (.78 to .97) reflecting differing practice among 4 physicians. Conclusions: The REBA can identify faculty and disease groups who underutilize clinical evidence, facilitating feedback for quality improvement. The REBA protocol is a useful tool to identify specific chemotherapy templates that may need modification. The REBA is a powerful electronic tool, which can be utilized to monitor and enhance the rate of evidence-based practice in oncology.

Regulated provider perceptions of feedback reports

This paper reports on regulated (or licensed) care providers' understanding and perceptions of feedback reports in a sample of Canadian long-term care settings using a cross-sectional survey design. Audit with feedback quality improvement studies have seldom targeted front-line providers in long-term care to receive feedback information. Feedback reports were delivered to front-line regulated care providers in four long-term care facilities for 13 months in 2009-10. Providers completed a postfeedback survey. Most (78%) regulated care providers (n = 126) understood the reports and felt they provided useful information for making changes to resident care (64%). Perceptions of the report differed, depending on the role of the regulated care provider. In multivariable logistic regression, the regulated nurses' understanding of more than half the report was negatively associated with 'usefulness of information for changing resident care', and perceiving the report as generally useful had a positive association. Front-line regulated providers are an appropriate target for feedback reports in long-term care. Long-term care administrators should share unit-level information on care quality with unit-level managers and other professional front-line direct care providers.

Methods to Achieve High Interrater Reliability in Data Collection From Primary Care Medical Records

We assessed interrater reliability (IRR) of chart abstractors within a randomized trial of cardiovascular care in primary care. We report our findings, and outline issues and provide recommendations related to determining sample size, frequency of verification, and minimum thresholds for 2 measures of IRR: the κ statistic and percent agreement. We designed a data quality monitoring procedure having 4 parts: use of standardized protocols and forms, extensive training, continuous monitoring of IRR, and a quality improvement feedback mechanism. Four abstractors checked a 5% sample of charts at 3 time points for a predefined set of indicators of the quality of care. We set our quality threshold for IRR at a κ of 0.75, a percent agreement of 95%, or both. Abstractors reabstracted a sample of charts in 16 of 27 primary care practices, checking a total of 132 charts with 38 indicators per chart. The overall κ across all items was 0.91 (95% confidence interval, 0.90-0.92) and the overall percent agreement was 94.3%, signifying excellent agreement between abstractors. We gave feedback to the abstractors to highlight items that had a κ of less than 0.70 or a percent agreement less than 95%. No practice had to have its charts abstracted again because of poor quality. A 5% sampling of charts for quality control using IRR analysis yielded κ and agreement levels that met or exceeded our quality thresholds. Using 3 time points during the chart audit phase allows for early quality control as well as ongoing quality monitoring. Our results can be used as a guide and benchmark for other medical chart review studies in primary care.

Implementation of a bundle of quality indicators for the early management of severe sepsis and septic shock is associated with decreased mortality*

The purpose of this study was to examine the outcome implications of implementing a severe sepsis bundle in an emergency department as a quality indicator set with feedback to modify physician behavior related to the early management of severe sepsis and septic shock. Two-year prospective observational cohort. Academic tertiary care facility. Patients were 330 patients presenting to the emergency department who met criteria for severe sepsis or septic shock. Five quality indicators comprised the bundle for severe sepsis management in the emergency department: a) initiate central venous pressure (CVP)/central venous oxygen saturation (Scvo2) monitoring within 2 hrs; b) give broad-spectrum antibiotics within 4 hrs; c) complete early goal-directed therapy at 6 hrs; d) give corticosteroid if the patient is on vasopressor or if adrenal insufficiency is suspected; and e) monitor for lactate clearance. Patients had a mean age of 63.8 +/- 18.5 yrs, Acute Physiology and Chronic Health Evaluation II score 29.6 +/- 10.6, emergency department length of stay 8.5 +/- 4.4 hrs, hospital length of stay 11.3 +/- 12.9 days, and in-hospital mortality 35.2%. Bundle compliance increased from zero to 51.2% at the end of the study period. During the emergency department stay, patients with the bundle completed received more CVP/Scvo2 monitoring (100.0 vs. 64.8%, p < .01), more antibiotics (100.0 vs. 89.7%, p = .04), and more corticosteroid (29.9 vs. 16.2%, p = .01) compared with patients with the bundle not completed. In a multivariate regression analysis including the five quality indicators, completion of early goal-directed therapy was significantly associated with decreased mortality (odds ratio, 0.36; 95% confidence interval, 0.17-0.79; p = .01). In-hospital mortality was less in patients with the bundle completed compared with patients with the bundle not completed (20.8 vs. 39.5%, p < .01). Implementation of a severe sepsis bundle using a quality improvement feedback to modify physician behavior in the emergency department setting was feasible and was associated with decreased in-hospital mortality.

A 1999 Quality Improvement Initiative—Reassessment in 2004

We obtained a longitudinal overview of physician compliance with the American Academy of Pediatrics Policy RE9539 regarding early newborn discharge between 1999 and 2004. In previous studies we reported the results of a Quality Improvement/Feedback (QI/F) initiative at one community hospital (intervention) in 1999, during which time physician education occurred at a grand rounds and the hospital QI department reviewed all newborn charts for a 1-year period, notifying physicians of any deviations from the policy. We also assessed the very same physicians at another community hospital (control), which did not have this initiative and found significant changes in physician behavior only at the intervention hospital. In this study we reassessed the same physicians in the year 2004 and compared their performance with that in 2000, after the intervention was well established, once again at both the intervention and control hospitals. Physicians caring for newborns at the intervention hospital continued to demonstrate improvement in compliance with the Early Newborn Discharge Policy (p = 0.0036), whereas there was no significant change in physician performance over time at the control hospital (p = 0.6874). We conclude, similarly to the first study, that improvement in physician practice continued, but there was still no overall change in physician culture.

Early newborn discharge: a tale of two hospitals.

This study reviews physician documentation of compliance with The American Academy of Pediatrics (AAP) policy RE9539 regarding early newborn hospital discharge and follow-up. All pediatricians in Kalamazoo, MI, were educated at a grand rounds regarding the AAP early newborn discharge policy. Newborns are seen at 2 community hospitals. One of them simultaneously instituted a Quality Improvement/Feedback (QI/F) program regarding early newborn discharge, (intervention) and the other 1 did not (control). This is a retrospective chart analysis that compares performance of 4 pediatric practices seeing newborns at each hospital. Each practice was compared at each hospital for appropriateness of discharge orders before and after the educational grand rounds and the QI/F initiative. Statistical analysis was done using the chi square test and the Breslow-Day test for homogeneity, and the Fisher's Exact Test. Odds ratios with a 95% confidence interval based on Taylor's approximation were used. There were no significant differences between the pediatric practices' performance before and after the educational initiative at the control hospital. There were significant differences before and after the educational initiative in the intervention hospital with the QI/F initiative. There was a significant reduction in variation among the practices after the QI/F initiative at the intervention hospital. When both hospitals were compared after the educational initiative, there was a significant difference between compliance among the same practices at each hospital, with better compliance at the intervention hospital with the QI/F initiative. There were significant differences in physicians' performance at the intervention hospital before and after the educational and QI/F initiatives. However, it was noted that the very same physicians did not comply as well in the control hospital without the QI/F initiative, thus still raising questions as to whether QI measures alter physician "culture." It is possible that the driving force for change in physician behavior was more intragroup peer pressure than an external QI/F initiative.

T HE E FFECT OF A Q UALITY I MPROVEMENT F EEDBACK L OOP ON P ARAMEDIC-INITIATED N ONTRANSPORT OF E LDERLY P ATIENTS

Objective. To examine the effect of a paramedic educational program and quality improvement feedback loop on paramedic-initiated nontransport of patients 65 years of age and older. Methods. Prospective observational study. Patients 65 years of age and older who were evaluated but not transported by paramedics were contacted by telephone within two weeks of emergency medical services (EMS) contact and asked: 1) whether the patient sought medical help within 24 hours after contact; 2) whether the patient was admitted to a hospital and, if so, what was the diagnosis; 3) who was responsible for the nontransport decision (patient, paramedic, or mutual); and 4) how satisfied the patient was with the EMS service. After six weeks of data collection, the results were presented in a nonjudgmental fashion to the paramedics. After this intervention, the data collection continued for another five weeks without the paramedics' knowledge. Results. After the intervention, the overall nontransport rate remained constant (11.5% vs. 10.7%). The percentage of patients seeking further medical attention within 24 hours also remained constant (37.1% vs. 33.9%). The percentage of patients who required hospitalization within 24 hours of the nontransport declined from 12.6% to 6.4%. The percentage of patients who refused ambulance transportation by paramedics declined from 9.3% to 3.7%. Overall satisfaction level rose from 94.7% to 100%. Conclusion. When paramedics were provided with objective feedback regarding outcome of patients not transported, the paramedic-initiated nontransportation and delayed hospitalization rates decreased, and the patient satisfaction level rose to 100%.

Successful strategies for recruiting African Americans to prevention trials

In 1992, the Pennington Biomedical Research Center instituted a multifactorial recruitment and retention strategy, including 1) policies promoting institution-wide cultural competence; 2) study specific recruitment plans implemented by a team of recruiter, public communication specialist, nurse study coordinator, computer specialist, and principal investigator; 3) a management plan using regular team meetings, a computerized tracking system, and volunteer feed back for quality improvement; and 4) reduction of barriers to participation coupled with positive reinforcement incentives. Recruitment methods included 1) radio/television/newsprint ads; 2) presentations at churches, social organizations, and work sites; 3) health fair screening; 4) direct mail; and 5) word of mouth. The results of this strategy are that, in 1995, there were 5456 volunteers for research trials conducted at the Pennington Center, 58% of whom were African Americans. Although investigators and their immediate research team members may profit from specific recruiting techniques, the commitment of the institution to minority recruitment is an essential element of success. Cancer 1998;83:1833-1835. © 1998 American Cancer Society.

Using Patient Feedback for Quality Improvement

This article describes the results of a study designed to understand h how health care organizations use patient feedback. The article examines the organizational factors and the barriers that influence patient feedback use and concludes with propositions that can serve to guide future action and research in this area.

The effect of a quality improvement feedback loop on paramedic skills, charting, and behavior.

A mechanism was initiated for conveying quality improvement (QI) results to paramedics as a means of improving chart documentation in difficult-to-correct areas. This study examines the impact of this QI feedback loop on charting, resuscitation rates from cardiac arrest, endotracheal intubation (ETI) success rates, and trauma scene times. Paramedic trip sheets were reviewed before and after the institution of the QI feedback loop in this interrupted time series design. The New Castle County, Delaware, Paramedic Program. All New Castle County paramedics participated in the study. In January 1990, the medical director began to circulate a QI summary among the paramedics in an effort to improve performance and chart documentation. The summary focused on the management of respiratory distress or arrest, cardiac arrest, and major trauma. The success rate for ETI was compared with the rate of field resuscitation from cardiac arrest, the percentage of unjustified prolonged trauma scene times (longer than 10 minutes), and the percent compliance with minimum endotracheal intubation documentation (ETID) requirements from a six-month period before institution of the QI feedback mechanism with data obtained from a six-month period after the program had been operational for one year. Comparing results from before with after the initiation of the QI program, the ETI success rate was 273 of 295 (92.5%) before and 300 of 340 (88.2%) after (chi 2 = 3.04, p < .1, ns); field resuscitations totaled 26 of 187 (13.9%) before and 44 of 237 (18.6) after (chi 2 = 1.40, p < .25, ns); ETID rate was 249 of 295 (84.4%) before and 336 of 340 (98.8%) after (chi 2 = 44.24, p < .001), and unjustified prolonged trauma scene times were 69 of 278 (24.8%) before and seven of 501 (1.4%) after (chi 2 = 320.5, p < .001). The use of QI feedback had little effect on psychomotor skills as the ETI success rate or resuscitation rate, but had a dramatic effect on chart documentation, as evidenced by ETID rate, and behavior, as evidenced by the reduction in prolonged trauma scene times. The use of QI feedback is recommended as a means of correcting charting deficiencies or modifying behavior.