Faculty Evaluations Of Residents Research Articles

Utilization of marketing automation tools for delivery of a faculty development curriculum

Background Physician clinical educators play important roles in teaching, providing feedback, and evaluating trainees, but they often have variable preparation and competing demands on their time that make universal participation in workshops, seminars, or short courses designed to foster these skillsets inefficient or impossible. Methods We designed and implemented a 52-week synchronous curriculum designed to address faculty opportunities to improve teaching skills, feedback for residents and medical students, and evaluation skills, which were delivered using marketing automation tools, including text messaging and email. We evaluated the programmatic impact and feasibility of using the implementation science framework. Results Over a 104-week evaluation period, there were at least 10,499 total content impressions and 4558 unique recipients, indicating the significant reach of this program to approximately 120 faculty members. Faculty engagement with continuing education materials remained stable or increased over the 2-year evaluation period, indicating that programs like ours can have sustainable impacts. Resident evaluations of faculty across the six key domains also improved after the implementation of the program. Conclusions Our experience with digital marketing tools reflects that they can be used to deliver impactful curricular content to faculty for continuing educational purposes and that faculty can use these resources in a sustainable way. However, because of the incomplete reach with any single communication, this type of content delivery is not appropriate for isolation as a material of critical importance. More research is needed to identify the best practices and additional education-related uses of this technology.

Gender and the Receipt of the Association of Residents in Radiation Oncology (ARRO) Educator of the Year Award

Despite an increase in female physicians in recent years, a gender disparity remains in the field of radiation oncology. We hypothesized that there may be a gender disparity in the receipt of ARRO educator awards. Using a database provided by the American Society for Radiation Oncology (ASTRO), annual award recipients were identified from 2010-2022. Data on Academic Institution, Year of Award, and Repeat Awardee Status were obtained through the ASTRO database. Publicly available websites were accessed to obtain data regarding gender, years post residency graduation, percent female/male faculty, size of residency program, and program director designation. H- indices were obtained from Scopus and used as a marker of academic productivity. Basic summary statistics, stratified by gender, were calculated for award year, years since graduation, percent of male faculty, and program size. A one-sample Z-test for proportions was utilized to assess if the proportion of female ARRO award winners each year from 2010 to 2022 was significantly less than the population average, defined as the proportion of female radiation oncology faculty members in the nominating universities that year. Secondary analyses used univariable binary logistic regression to identify any global associations between gender, year since gradation, or program size. The lowest proportion of female awardees occurred in 2013 (8 [14.3%] females vs. 48 [85.7%] males) and the greatest proportion of female awardees occurred in 2022 (19 [30.6%] females vs. 43 [69.4%] males). As compared to the proportion of female faculty members in nominating programs for that respective year, there were significantly fewer female awardees in the years of 2010 (18% female awardees vs. 32% female faculty members, p = 0.02) and 2013 (14% female awardees vs. 31% female faculty members, p = 0.01). There was a statistically significant rise in female awardees over the study period (p<0.01). On logistic regression analysis, large program size (10+ residents) (odds ratio [OR]: 6.86, 95% confidence interval [CI]: 2.71-23.1, p<0.001) and medium program size (5-9 residents) (OR: 4.05, 95% CI: 1.60-13.7, p<0.001) were associated with a greater proportion of female awardees as compared to small program size (1-4 residents). There was no association between awardee gender and years since graduation. A gender disparity is present in the receipt of ARRO Educator Awards. At an institutional level, residency Chiefs, Program Directors, and Chairs should work to ensure that a diverse slate of faculty are considered annually for the ARRO Educator Award with self-assessment of awardees over time to ensure demographic representation and inclusion. Efforts should also be made to recognize biases in resident evaluations of faculty with continued education about inherent biases.

Words Matter: Examining Gender Differences in the Language Used to Evaluate Pediatrics Residents.

Gender disparities in academic medicine continue to be pervasive. Written evaluations of residents may provide insight into perceptions of residents by faculty, which may influence letters of recommendation for positions beyond residency and reinforce perceived stereotype threat experienced by trainees. To examine language used in faculty evaluations of pediatrics residents to determine if there are differences in language used with respect to gender of resident. All faculty evaluations of residents in 3 consecutive intern classes from 2016 to 2018 were collected and redacted for name and gender identifiers. We performed a qualitative analysis of written comments in 2 mandatory free text sections. The study team initially coded text collectively, generating a code book, then individually to apply the coding scheme. Next, evaluations were unblinded to gender. Code applications were aggregated by resident, and frequencies of code application by resident were compared by standardized mean differences to detect imbalances between genders. A total of 448 evaluations were analyzed: 88 evaluations of 17 male residents, and 360 evaluations of 70 female residents. Codes more frequently applied to women included "enthusiasm," and "caring," while codes more frequently applied to men included "intelligence," and "prepared." A conceptual model was created to reflect potential impacts of these differences using a lens of social role theory. We identified differences in the way male and female residents are evaluated by faculty, which may have negative downstream effects on female residents, who may experience negative self-perception, differential development of clinical skills, and divergent career opportunities as a result.

Resident Evaluations of Faculty: Resident Versus Faculty Perspectives

Resident Evaluations of Faculty: Resident Versus Faculty Perspectives

Accuracy of faculty evaluations of residents and #8217; medical knowledge: comparison of subjective summative evaluations to in-training examination scores

Although faculty evaluations are an integral component in the assessment of residents’ medical knowledge, there is conflicting evidence about the accuracy of such evaluations. In contrast, the in-training examination (ITE) is an objective measure of residents’ medical knowledge that has proven to predict future performance. This study investigates the accuracy of faculty evaluations by comparing scores of medical knowledge on faculty completed evaluations to in-training examination scores. Subjects included categorical pediatrics and pediatric neurology residents at Mayo Clinic from 2006-2010. For each year of training the standard ITE score was compared to the corresponding composite faculty evaluation score for medical knowledge and patient care competencies using the Spearman’s rank correlation coefficient. Forty-two residents were analyzed for the first year of training and 35 for the second. For both years, there was only a mild correlation between ITE score and either the medical knowledge (r=0.26 and 0.19, respectively) or patient care competency (r=0.27 and 0.29, respectively). Faculty assessments of residents’ medical knowledge did not correlate well with a standard objective measure. Training programs should consider targeted faculty development to improve evaluation skills and supplementing existing evaluation methods with additional assessment tools.

Anesthesiology resident personality type correlates with faculty assessment of resident performance

Study ObjectiveTo study the association between anesthesiology residents' personality preference types, faculty evaluations of residents' performance, and knowledge. DesignConvenience sample and prospective study. SettingAcademic department of anesthesiology. SubjectsConsenting anesthesiology residents (n = 36). InterventionsAll participants completed the Myers Briggs Type Indicator® (MBTI®). MeasurementsAll residents' 6-month summation of daily focal evaluations completed by faculty [daily performance score (DPS); 1 = unsatisfactory, 2 = needs improvement, 3 = meets expectations, 4 = exceeds expectations], as well as a global assessment of performance (GAP) score based on placement of each resident into perceived quartile compared with their peers (ie,1 = first, or top, quartile) by senior faculty (n = 7) who also completed the MBTI, were obtained. The resident MBTI personality preferences were compared with the DPS and GAP scores, the United States Medical Licensing Examination (USMLE) I and II scores, and faculty MBTI personality type. Main ResultsThere was no association between personality preference type and performance on standardized examinations (USMLE I, II). The mean GAP score was better (higher quartile score) for Extraverts than Introverts (median 2.0 vs 2.6, P = 0.0047) and for Sensing versus Intuition (median 2.0 vs 2.6, P = 0.0206) preference. Faculty evaluator MBTI preference type did not influence the GAP scores they assigned residents. Like GAP, the DPS was better for residents with Sensing versus Intuition preference (median 3.5 vs 3.3, P = 0.0111). No difference in DPS was noted between Extraverts and Introverts. ConclusionsPersonality preference type was not associated with resident performance on standardized examinations, but it was associated with faculty evaluations of resident performance. Residents with Sensing personality preference were evaluated more favorably on global and focal faculty evaluations than those residents who chose the Intuition preference. Extraverted residents were evaluated more favorably on global but not focal assessment of performance.

Are Successful Resident Clinicians Good Teachers? Multidimensional Resident Assessment

GME training programs must demonstrate residents are prepared as both teachers and clinicians. We examined the relationship between faculty evaluations of residents' clinical performance and student evaluations of residents' teaching performance. Concordance tables for mean ratings and qualitative analysis of comments among 95 residents receiving evaluations by 267 faculty and 106 students. A total of 88% of residents received concordant ratings from faculty and students. When discordant, faculty gave higher ratings (P = .003). Ninety and ninety-one percent of faculty and student comments exhibited a positive tone. Faculty and students tended to agree on the successful and unsuccessful residents, supporting the validity of rating scale evaluations. Future research might focus on how to tailor resident remediation when discordant ratings or comments occur.

Clinical teaching rounds. A case-oriented faculty development program.

To improve clinical teaching with emphasis on improving provision of feedback through a faculty development series modeled on clinical rounds. Seven 1-hour conferences were held for the pediatric faculty during the academic year 1994-1995. Clinical rounds were emulated, with a simulated learner functioning as the patient with a chief complaint of some instructional problem. The conferences progressed from discussion about teaching in a particular situation, to videotapes of clinical teaching, and finally to live clinical teaching. Evaluation of the conferences was assessed by attendance records, participants' evaluations of the conferences, and comparing student and resident evaluations of faculty who attended (i.e., those who attended > or = 2) with faculty who did not attend. Comparisons were made for the academic year before and after the conferences using paired t tests. Forty percent of the faculty attended 2 or more conferences. Mean conference ratings were 4.00 to 4.35, (1 is poor; 5, excellent). Faculty who attended had a significant improvement in ratings for feedback (P = .01) and overall teaching effectiveness (P = .04). Ratings for faculty who did not attend did not change. These conferences were well received by the faculty and are an effective way to improve clinical teaching.

Measurement and prediction of diagnostic performance during radiology training.

Faculty evaluations of residents' diagnostic performance in radiology subspecialty rotations were examined in two studies in order to ascertain whether the relative capabilities of residents change during training and to predict residents' diagnostic performance using a three-dimensional form perception test. In the first study, numeric ratings by faculty members were averaged to provide interpretive/diagnostic scores for each of 16 residents in each of 5 consecutive half-years. Sixty-seven percent of the relative differences among residents' diagnostic proficiencies persisted during training. The magnitude of these unchanging differences between individuals in diagnostic performance strongly favors resident selection based upon diagnostic potential. An additional 22% of the relative differences correlated with an abrupt change in the rank order of residents in diagnostic performance at the beginning of the second year of residency. This rearrangement of ranks may have resulted from an abrupt change in the tasks and expectations assigned to residents. In the second study, correlations between scores on a recently described Form Test and monthly faculty ratings of diagnostic performance were computed. An average diagnostic performance score for each resident was generated for all rotations combined, each of eleven subspecialties, and all rotations completed within a given half-year. Scores on the Form Test correlated well with these combined diagnostic scores, further substantiating results reported previously. Form Test scores were highly correlated with subspecialty diagnostic performance scores in subspecialties using cross-sectional imaging methods, such as neuroradiology. Although Form Test scores were poorly correlated with half-year diagnostic performance during the first year, they were highly correlated with performance beginning in the second year.