Individual Performance Feedback Research Articles

Evaluating the impact of feedback on elementary aged students' fluency growth in written expression: A randomized controlled trial

The purpose of this randomized controlled trial was to evaluate elementary-aged students' writing fluency growth in response to (a) instructional practices, (b) sex differences, and (c) student's initial level of writing fluency. Third-grade students (n=133) in three urban elementary schools were randomly assigned to either an individualized performance feedback condition (n=46), a practice-only condition (i.e., weekly writing practice; n=39), or an instructional control condition (n=48) for 8weeks. Findings included support for use of performance feedback as an instructional component in general education classrooms (Hedges' g=0.66), whereas simple practice with curriculum-based measurement in written expression did not produce growth significantly greater than standard instructional practices. The hypothesis that girls write significantly more than boys was supported. However, girls and boys did not differ in their rate of growth. Finally, students' initial risk status in writing fluency did not differentially predict growth in writing fluency over the course of the study. Implications for incorporating feedback as a basic component of intervention in writing are discussed.

The Effect of Real-Time Individual Process Performance Feedback on Computer-based Group Idea Generation

In computer-mediated idea generation where contributions can be anonymous, the ability to accurately monitor performances is limited, inducing social loafing. Prior research has suggested that social loafing is likely an important factor in reducing task performance. Researchers have theorized that loafing could be minimized if clear performance feedback is provided. Our prior study evidences a substantial performance gain by the provision of real time performance information about who is contributing and who is not. However, our prior study incorporated the quantity feedback only to create a larger pool of ideas based on the long-standing assumption (i.e., quantity breeds quality), not considering the quality feedback. As a result, taking advantage of anonymity in the form of pseudonymity, individuals in almost all groups exhibited a tendency of self-presentation by capitalizing on ideas of which quality was low and even frivolous (i.e., junk comments) toward the later stages of the session. Thus, we have learned that the quantity performance feedback alone does not have enough restrictiveness to consistently control the performance behavior throughout the session. Since a process chart allows participants to monitor process variation by comparing new performance data to past performance data, we incorporated real-time visual process performance feedback to reveal performance histories by connecting the sequence of idea quality scores in a time-series format. Using this environment, a laboratory experiment was conducted with five-member groups that examined the influence of both identifiability (i.e., anonymity versus pseudonymity) and process performance feedback (i.e., yes or no) in a <TEX>$2{\times}2$</TEX> factorial design. The result showed that groups in the process performance feedback treatment outperformed groups in the no feedback treatment. Additionally, process performance feedback and identifiability interacted such that groups in the process performance feedback/pseudonymity treatment had the highest performance. The implications of these findings for future research, as well as the implications for the design of group idea generation procedures are discussed.

Improving Hand-hygiene Compliance and Reducing Healthcare Associated Infections with Automated Hand-hygiene Compliance Monitoring

Hand-hygiene (HH) is recognized as one of the most successful ways to reduce infections and improve patient outcomes. This organization used lessons learned from implementing an automated hand-hygiene monitoring system in one unit to accelerate a successful implementation in a second unit. A program of routine, comprehensive, individual performance feedback based on data derived from an automated measurement system contributed to significant and sustained HH compliance improvements in both locations.

The Use of Individualized Pharmacist Performance Reports to Reduce Pharmacist-Related Medication Order Entry Errors following Electronic Medical Record Implementation

Purpose To measure the impact of an individual pharmacist performance report (IPPR) program on pharmacist-related medication order entry errors (MOEEs) at an academic medical center. Methods The number and type of pharmacist-related MOEEs were collected at 2 different times: immediately following implementation of an electronic medical record (baseline) and following completion of the IPPR program. Three different collection methods were utilized to identify and categorize pharmacist-related MOEEs: 1) Patient Safety Net (PSN) incident reporting system, 2) manual event reporting, and 3) a nursing to pharmacy electronic messaging system. The IPPR program consisted of mandatory educational sessions for pharmacist staff. The program content focused on strategies to reduce pharmacist-related MOEEs identified in the baseline data collection period as well as an individualized report generated for each pharmacist having caused an MOEE that showed their performance compared to the department's performance. Results The percentage of event reports containing a pharmacist-related MOEE decreased from baseline to post IPPR program (13.7% and 6.3%, respectively; P &lt; .001). In addition, the total number of pharmacist-related MOEEs was halved after the IPPR program (321 vs 148; P &lt; .001). Significant reductions were noted in the following MOEE categories: duplicate orders, missed orders, wrong frequency, wrong dose, and other. Nonsignificant reductions were noted in errors related to no order and wrong drug. Conclusion Based on the observations and results of this study, it is proposed that individualized performance feedback can be a successful method to improve MOEE performance by pharmacists.

Radiation oncology and medical physicists quality assurance in British Columbia Cancer Agency Provincial Prostate Brachytherapy Program

To describe in detail British Columbia (BC) Cancer Agency (BCCA) Provincial Prostate Brachytherapy (PB) Quality Assurance (QA) Program. The BCCA PB Program was established in 1997. It operates as one system, unified and supported by electronic and information systems, making it a single PB treatment provider for province of BC and Yukon. To date, >4000 patients have received PB (450 implants in 2011), making it the largest program in Canada. The Program maintains a large provincial prospective electronic database with records on all patients, including disease characteristics, risk stratification, pathology, preplan and postimplant dosimetric data, follow-up of prostate-specific antigen, and toxicity outcomes. QA was an integral part of the program since its inception. A formal QA Program was established in 2002, with key components that include: unified eligibility criteria and planning system, comprehensive database, physics and oncologist training and mentorship programs, peer review process, individual performance outcomes and feedback process, structured continuing education and routine assessment of the program's dosimetry, toxicity and prostate-specific antigen outcomes, administration and program leadership that promotes a strong culture of patient safety. The emphasis on creating a robust, broad-based network of skilled providers has been achieved by the program's requirements for training, education, and the QA process. The formal QA process is considered a key factor for the success of cancer control outcomes achieved at BCCA. Although this QA model may not be wholly transferable to all PB programs, some of its key components may be applicable to other programs to ensure quality in PB and patient safety.

Abstract 102: FOCUS: Appropriate Use of Cardiac Imaging

Introduction The American College of Cardiology has created a new initiative (FOCUS) to help reduce inappropriate use of clinical imaging in cardiology. In accord with the effort, the Sanger Heart and Vascular Institute (SHVI) has designed a program to enhance appropriate use of cardiac imaging. Hypothesis: By implementing a quality improvement program to evaluate the appropriateness of cardiac imaging, there will be a 10% improvement in the percentage of studies ordered for appropriate indications and a corresponding decrease in the number of studies ordered for inappropriate indications. Methods: The initiative utilized an integrated multifaceted approach to quality improvement: 1) Staff / physician education with video teleconferences: Presentations were given at SHVI’s Grand Round, focusing on the appropriate use criteria as outlined in the ACCF/ASE/ASNC documents. An important component was interactive case studies highlighting the principles of appropriate use criteria. 2) Peer review and audits: Each practitioner and site received individual performance feedback. This was a complete analysis which included testing appropriateness as an indication. 3) System support tools: A comprehensive order sheet with check boxes for all appropriate indications was designed. All ordering physicians were asked to use this form for ordering all cardiac imaging studies. For those studies in which the form was not utilized, the imaging technologist attempted to obtain the appropriate information. All information from the forms was entered into a database designed specifically for this study. A baseline survey of 30 consecutive studies was selected for analysis from each echocardiography and cardiac nuclear imaging site within SHVI for a total of 360 studies. The ongoing post intervention survey conducts a monthly review of 25% of all studies per site. Results: At baseline, 77% of transthoracic echocardiographic studies were ordered for appropriate indications. Post intervention there was significant improvement: 91% (p&lt;0.0013) of studies were categorized as appropriate and 1% of studies categorized as inappropriate (14% improvement in each). Of the remaining studies, 9% could not be categorized. With regards to stress cardiac imaging, 71% of studies were ordered for appropriate indications at baseline. Post intervention, there was significant improvement: 94% (p&lt;0.0008) of studies categorized as appropriate (23% improvement), 16% categorized as inappropriate, and 6% could not be categorized. Conclusion: Although there is limited data on a national basis, published literature would suggest 70-80% appropriate use of cardiac imaging as a baseline benchmark. The results of this initiative demonstrated a statistically significant benefit in the appropriate use of cardiac imaging for SHVI from this baseline. The data gathering and database capabilities established during this initiative will allow quarterly reviews to providers, which will help ensure continued compliance with the guidelines.

Quality-Improvement Initiative Sustains Improvement in Pediatric Health Care Worker Hand Hygiene

To use quality-improvement (QI) methods to develop and test a multimodal intervention to improve hand-hygiene compliance among health care workers (HCWs) to >90%. We used a quasi-experimental staggered intervention that was conducted on 2 similar general pediatric units within a 475-bed tertiary children's hospital. Compliance was defined as acceptable hand hygiene both before and after contact with the patient or the patient's care environment. Measurement of HCW hand-hygiene compliance was performed by covert observations made during routine patient care. Twelve months of preintervention data were collected. QI methods were used to test and implement interventions sequentially in each unit. Interventions addressed leadership support, improving HCW knowledge, hand-hygiene supply availability, and HCW behavior. Interventions began on unit A on November 10, 2008. Similar interventions were later tested on unit B starting March 23, 2009. By April 1, 2009, compliance increased on unit A (from 65% to 91%) and unit B (from 74% to 92%). Improvement on each unit occurred only after the interventions were introduced. Identifying HCWs who failed to perform hand hygiene and offering alcohol-based hand rub to them before patient contact resulted in the greatest improvement. Improvements were sustained on both units for 18 months. Use of QI methods to implement a multimodal intervention resulted in sustained improvement in hand-hygiene compliance. Real-time individual performance feedback or other high-reliability human-factor interventions seem to be necessary to reach and sustain high levels of hand-hygiene compliance.

Is Physician Engagement With Web-Based CME Associated With Patientsʼ Baseline Hemoglobin A1c Levels? The Rural Diabetes Online Care Study

To investigate the association between physician participants' levels of engagement in a Web-based educational intervention and their patients' baseline diabetes measures. The authors conducted a randomized trial of online CME activities designed to improve diabetes care provided by family, general, and internal medicine physicians in rural areas of 11 southeastern states between September 2006 and July 2008. Using incidence rate ratios derived from negative binomial models, the relationship between physicians' engagement with the study Web site and baseline proportion of their patients having controlled diabetes (hemoglobin A1c < or = 7%) was explored. One hundred thirty-three participants (intervention = 64; control = 69) provided information for 1,637 patients with diabetes. In the intervention group, physicians in practices in the worst quartiles of A1c control were least engaged with the study Web site in nearly all dimensions. Total number of pages viewed decreased as quartile of A1c control worsened (137, 73, 68, 57; P = .007); similarly, for a given 10% increase in proportion of patients with controlled A1c, participants viewed 1.13 times more pages (95% CI: 1.02-1.26, P = .02). In the control group, engagement was neither correlated with A1c control nor different across quartiles of A1c control. Engagement in Web-based interventions is measurable and has important implications for research and education. Because physicians of patients with the greatest need for improvement in A1c control may not use online educational resources as intensely as others, other strategies may be necessary to engage these physicians in professional development activities.

Ongoing Provision of Individual Clinician Performance Data Improves Practice Behavior

Clinical practice guidelines summarize evidence from science and attempt to translate those findings into clinical practice. Pervasive and consistent adoption of these guidelines into daily provider practice has proven slow. Using postoperative nausea and vomiting (PONV) prophylaxis guideline compliance as our metric, we compared the effects of continuing medical education (CME) alone (I), CME with a single snapshot of provider compliance (II), and ongoing reporting of provider compliance data without further CME (III). We retrospectively analyzed guideline compliance of 23,279 anesthetics at the University of Texas M.D. Anderson Cancer Center. Compliance was defined as a patient with 1 risk factor for PONV receiving at least 1 antiemetic, 2 risk factors receiving at least 2 antiemetics, and 3 risk factors receiving at least 3 antiemetics. Drugs of the same class were counted as single antiemetic administration. Propofol-based anesthetic techniques were counted as receiving 1 antiemetic. Patients with 0 risk factors for PONV were not included. We estimated the compliance rates for each of the 4 time periods of the study adjusting for multiple observations on the same clinician. Individual performance feedback was given once at 6 months after intervention I coincident with a refresher presentation on PONV (start of intervention II) and on an ongoing quarterly basis during intervention III. Compliance rates were not significantly influenced with CME (intervention I) compared with baseline behavior (54.5% vs 54.4%, P = 0.9140). Significant improvement occurred during the time period when CME was paired with performance data (intervention II) compared with intervention I (59.2% vs 54.4%, P = 0.0002). Further significant improvement occurred when data alone were presented (intervention III) compared with intervention II (65.1% vs 59.2%, P < 0.0001). For patients with 3 risk factors, we saw significant improvement in compliance rates during intervention III (P = 0.0002). In post hoc analysis of overtreatment, the percentage differences between the baseline and time period III decreased as the number of risk factors increased. We observed the greatest improvement in guideline compliance with ongoing personal performance feedback. Provider feedback can be an effective tool to modify clinical practice but can have unanticipated consequences.

ORGANIZATIONAL BEHAVIOR Conference Paper Abstracts.

One of the great challenges in information-sharing and knowledge management in teams is the sharing of expertise. In this paper, a model of expertise in teams is suggested that conceptualizes expertise not only as professional knowledge but also as expert role from a status perspective. The model predicts that experts will reduce their contributions to the team goals if their expert status is threatened in any way, but will increase their knowledge sharing behaviour if their status in the team is confirmed. The hypotheses were confirmed in two experimental studies (N1=98, N2=144), showing that explicitly declaring status differences increased expert contributions. These contributions could be further enhanced by giving individual performance feedback to experts, and were decreased under team feedback conditions. The findings are consistent with recent research showing that diversity can contribute to team performance if the diversity is recognized within the team.

Effect of performance feedback on tracheal suctioning knowledge and skills: randomized controlled trial

This paper is a report of a study to determine whether individualized performance feedback improved nurses' and physiotherapists' knowledge and practice of tracheal suctioning. Nurses' knowledge and practice of tracheal suctioning is often deficient. Whilst teaching has been shown to improve suctioning knowledge and practice, this is not sustained over time. Ninety-five qualified healthcare professionals (nurses and physiotherapists) in two acute hospitals were randomly allocated to receive either individualized performance feedback or no additional feedback after a standardized lecture and practical demonstration of tracheal suctioning. Randomization was stratified by profession, seniority and site. Data were collected in 2005 in a clinical setting involving patients and a simulation setting. The outcome measures were knowledge and practice of tracheal suctioning, assessed by self-completion questionnaire and structured observation. In both settings, intervention groups performed statistically significantly better in terms of knowledge (P = 0.014) and practice (P = 0.037) at final follow-up. Those who received performance feedback had statistically significantly higher knowledge (P = 0.004) and practice (P < 0.01) scores than the control group. For practice, there was also a relationship between professions (P < 0.01), with physiotherapists performing better than nurses overall, and an interaction between group and setting (P < 0.01), with performance feedback showing a stronger positive effect in the simulation setting. Retention of knowledge and tracheal suctioning practice is improved when training is followed up by tailored feedback on performance. Further research would shed light on how long such improvements are sustained, and whether the improvements seen in a simulated setting can be generalized to clinical settings.

Effect of group categorization, group performance feedback and individual performance feedback on choice of outgroup member

Effect of group categorization, group performance feedback and individual performance feedback on choice of outgroup member

Assessing the practising physician using patient surveys: a systematic review of instruments and feedback methods

Individual physician performance assessment is a vital part of the medical regulation debate. In this context, the patient perspective is seen as a potentially valid component. Yet, the theoretical and empirical evidence base for such patient assessments is unclear. To identify and evaluate instruments designed to assess patients' experiences with an individual practising physician, and to provide performance feedback at the individual level. Nine electronic databases were searched with no language restrictions: PubMed (1985-), Embase (1985-), PsycInfo (1985-), SIGLE (1985-), HMIC (1985-), ASSIA (1985-), CINAHL (1985-), Cochrane (1985-) and Dare (1985-). Inclusion: (i) completed by patients; (ii) assess practising doctors; (iii) have capacity to assess individual doctors for performance feedback; and (iv) used for individual performance feedback. Exclusion: (i) completed by colleagues, observers or third parties; (ii) assess medical students, nurses or non-physicians; (iii) assess purely at an organizational level; and (iv) not been used for individual feedback. All electronic outputs were independently assessed by three reviewers. Data were extracted independently by two of three reviewers using a defined template. Six instruments met the inclusion criteria. They all combine evaluation at both organizational and individual level and implementation methods lack standardization. There is limited data on their construct validity or correlations with other attributes. The purpose and method of individual feedback are not well specified, and the evidence to date about the effectiveness of feedback to obtain improvement indicates professional resistance. For formative goals, more clarity is needed about the aim of providing patient assessments feedback to individual doctors: 'who' should do it and 'how' to do so to best effect. We need to know whether feedback improves doctor performance and how these evaluations correlate with other physician attributes. For summative purposes more research is required on validity and reliability.

Implementing Achievable Benchmarks in Preventive Health: A Controlled Trial in Residency Education

To evaluate the Preventive Health Achievable Benchmarks Curriculum, a multifaceted improvement intervention that included an objective, practice-based performance evaluation of internal medicine and pediatric residents' delivery of preventive services. The authors conducted a nonrandomized experiment of intervention versus control group residents with baseline and follow-up of performance audited for 2001-2004. All 130 internal medicine and 78 pediatric residents at two continuity clinics at the University of Alabama School of Medicine, Birmingham, participated. Performance of preventive care was assessed by structured chart review. The multifaceted feedback curriculum included individualized performance feedback, academic detailing by faculty, and collective didactic sessions. The main outcome was difference in receipt of preventive care for patients seen by intervention and control residents, comparing baseline and follow-up. Charts were reviewed for 3,958 patients. Receipt of preventive care increased for patients of intervention residents, but not for patients of control residents. For the intervention group, significant increases occurred for five of six indicators in internal medicine: smoking screening, quit smoking advice, colon cancer screening, pneumonia vaccine, and lipid screening; and four of six in pediatrics: parental quit smoking advice, car seats, car restraints, and eye alignment (p < .05 for all). For control residents, no consistent improvements were seen. There was greater improvement for intervention than for control residents for four of six indicators in internal medicine, and two of six in pediatrics. Using a multifaceted feedback curriculum, the authors taught residents about the care they provide and improved documented patient care.

Individualized performance feedback to increase prenatal domestic violence screening

Universal domestic violence (DV) screening once per trimester of pregnancy is recommended but rarely accomplished. Clinical leaders in this setting sought to improve adherence with this protocol. This prospective study used medical record audit and individualized performance feedback with peer comparison (IPF) to improve DV screening among first and second year obstetrics and gynecology (ob/gyn) residents. The setting is a northeastern, urban, hospital-based, prenatal clinic serving low-income women. Most patients are Latina (75%); 11% are black and 9% are white. Few begin care in the first trimester (8.5%). We gave all residents DV training. Next we gave IPF-four reports at seven-week intervals. We reviewed medical record notes on patient visits corresponding to the first medical encounter and week 16 and week 28 of pregnancy. We used this data to compare screening immediately before IPF and following each IPF report. Screening increased steadily over time, from 60% of appropriate visits before IPF to 91% after the fourth report (Chi Square 28.4, p<.001). Adjusting for key factors, the odds of screening after the last IPF report were seven and a half times greater than the odds of screening before IPF (Odds Ratio: 7.6; 95% Confidence Interval: 3.0, 19.0). IPF was associated with increased DV screening among first and second year ob/gyn residents in this setting. Increased screening improved compliance with the clinic protocol and increased opportunities for patient disclosure, education, and treatment, critical public health objectives.

Enhancing staff performance measures in an acquired brain injury setting: combating the habituation to organizational behavioral interventions

AbstractThe current study examines the utility of a range of behavioral strategies involving group and individual performance feedback for direct care staff in an in‐patient acquired brain injury setting. We first assessed the level of staff completion of behavioral programs for persons with brain injury who resided in one of two residential houses. A weekly percentage score for the entire house was obtained that reflected the completion of behavior programs in each residence. In a multiple baseline fashion across houses we varied the types of feedback intervention introduced to staff, which included in‐service training on the importance of completing behavior programs, a weekly public posting of overall program completion by house, and a weekly public posting of completion of programs by specific staff members. Copyright © 2005 John Wiley &amp; Sons, Ltd.

Adjunctive Role for Immediate Feedback in the Acquisition and Retention of Mathematical Fact Series by Elementary School Students Classified with Mild Mental Retardation

The effects of feedback on the acquisition and retention of mathematical fact series by elementary school students classified with mild mental retardation was examined in 4 studies. Immediate feedback was provided by either an educator or the Immediate Feedback Assessment Technique (IF AT), at the end of a test series by a review of correct solutions (delayed feedback), or for control purposes, with a Scantron form. Reductions in errors and inaccurate perseverative responding during intervention periods were higher, and the repetition of errors during maintenance test was lower, when feedback was provided by an educator in Study 1. These results were replicated and extended in Study 2 to the operation of subtraction. In Study 3, the concurrent presentation of both forms of immediate feedback was more effective than the presentation of either form separately. In Study 4, the adjunctive value of the IIF AT to facilitate the teaching-learning process was established, with higher levels of independent learning and retention demonstrated when the I F AT was available. The IF AT, as a simple paper and pencil tool, can assist the educator through the provision of individualized performance feedback and the encouraging of students to continue responding while simultaneously promoting independent learning.

A Multiple-Goal, Multilevel Model of Feedback Effects on the Regulation of Individual and Team Performance.

When working as a member of a team, individuals must make decisions concerning the allocation of resources (e.g., effort) toward individual goals and team goals. As a result, individual and team goals, and feedback related to progress toward these goals, should be potent levers for affecting resource allocation decisions. This research develops a multilevel, multiple-goal model of individual and team regulatory processes that affect the allocation of resources across individual and team goals resulting in individual and team performance. On the basis of this model, predictions concerning the impact of individual and team performance feedback are examined empirically to evaluate the model and to understand the influence of feedback on regulatory processes and resource allocation. Two hundred thirty-seven participants were randomly formed into 79 teams of 3 that performed a simulated radar task that required teamwork. Results support the model and the predicted role of feedback in affecting the allocation of resources when individuals strive to accomplish both individual and team goals.

Provider–user differences in perceived usefulness of forecasting formats

This paper aims to examine potential differences in perceived usefulness of various forecasting formats from the perspectives of providers and users of predictions. Experimental procedure consists of asking participants to assume the role of forecast providers and to construct forecasts using different formats, followed by requesting usefulness ratings for these formats (Phase 1). Usefulness of the formats are rated again in hindsight after receiving individualized performance feedback (Phase 2). In the ensuing role switch exercise, given new series and external predictions, participants are required to assign usefulness ratings as forecast users (Phase 3). In the last phase, participants are given performance feedback and asked to rate the usefulness in hindsight as users of predictions (Phase 4). Results reveal that regardless of the forecasting role, 95% prediction intervals are considered to be the most useful format, followed by directional predictions, 50% interval forecasts, and lastly, point forecasts. Finally, for all formats and for both roles, usefulness in hindsight is found to be lower than usefulness prior to performance feedback presentation.

Conceptions of Self and Performance-Related Feedback in the U.S., Japan and China

Effective international management requires knowledge of culturally patterned self-concepts and concomitant feedback desires. This paper argues that the cultural forces of individualism versus collectivism shape the self-concept, and by extension individual performance evaluation and feedback seeking. We present a comparative study to test the hypotheses that U.S. respondents desire success feedback (whereas Japanese and Chinese desire feedback), show more initiative and perceive more direct workplace feedback than do either Japanese or Chinese respondents. Strong support was found for comparisons between the U.S. and Japan, but the Chinese consistently departed from predictions. We interpret these results in terms of recent reforms in the People's Republic of China, and discuss implications of the study for international management.