Faculty evaluations: Diagnostic and therapeutic

Abstract

An excellent teaching program is one that creates an atmosphere of mentorship, leadership, and professional development. The key to all of these may lie in the faculty evaluation. Currently, favorable faculty evaluations are often rewarded, whereas the more critical evaluations are downplayed and seldom used for faculty development. The purpose of this study is to investigate the efficacy of the faculty evaluation as a diagnostic tool to objectively identify individual strengths and weaknesses. Can such a tool then be used as a therapeutic modality to improve and enhance the faculty as a whole? Fourth- and fifth-year surgical residents from 1 residency program were asked to collaboratively define 9 characteristics that make a surgical role model. They then anonymously evaluated each of the 44 teaching attendings on each of the 9 criteria. The results of the evaluations, which consisted of the overall mean scores and SD for each of the 9 criteria, the attending's score and SDs above or below the mean, as well as whether the residents considered them to be a Role Model, were sent to each faculty member as an intervention. Six months after the intervention, the residents evaluated the attendings using the same 9 criteria. Evaluations were collected and analyzed following the same procedure as 6 months prior. T-tests and p-values were calculated to determine any significant differences between the 2 data sets. Based on the results of the pre-intervention evaluations, we were able to delineate faculty members into 3 groups based on their Role Model score. Of the 10 faculty members with the lowest scores (defined as >1 SD below average), 7 improved after the intervention, 3 of them were statistically significant (2 with p < 0.05 and 1 with p < 0.10). In the middle group of 26 faculty members, 1 showed statistically significant improvement (p < 0.05). Interestingly, this was the faculty member with the lowest score in this group. Six faculty originally in the middle group improved to a score greater than 1 SD above average, defining them as a Role Model. The remaining 8 faculty members originally designated as Role Models showed no statistically significant changes in their scores after the intervention. After the intervention, the average score improved for all of the 9 criteria that make a faculty Role Model. Statistically significant improvements were seen in the categories of Provides Feedback (p < 0.05) and Didactic Teaching, Attendance at Didactic activities, and Stimulates Critical Thinking with the Use of Literature (all p < 0.10). (1) Faculty evaluations are a diagnostic tool with the ability to select out Role Models from the lower scoring groups. (2) After the intervention, the overall average score improved for all of the 9 criteria that make a Role Model, showing improvement of the faculty as a whole. (3) The therapeutic benefits of faculty evaluations were best seen in the faculty with the lowest scores.