Anesthesia residents' global (departmental) evaluation of faculty anesthesiologists' supervision can be less than their average evaluations of individual anesthesiologists.

Abstract

Faculty anesthesiologists' supervision of anesthesiology residents is required for both postgraduate medical education and billing compliance. Previously, using the de Oliveira Filho et al. supervision question set, De Oliveira et al. found that residents who reported mean department-wide supervision scores <3.0 ("frequent") reported a significantly more frequent occurrence of mistakes with negative consequences to patients, as well as medication errors. In our department, residents provide daily evaluations of the supervision received by individual faculty. Using a survey study, we compared relationships between residents' daily supervision scores for individual faculty anesthesiologists and residents' supervision scores for the entire department (comprised these faculty). We studied all anesthesiology residents in clinical years 1, 2, and 3 (i.e., neither in the "base year" nor in fellowship). There were daily evaluations of individual faculty supervision of operative anesthesia for 36 weeks. Residents clicked a hyperlink on the invitation e-mail taking them to a secure Web page to provide their global (departmental) assessment of faculty supervision. We calculated the ratio of each resident's global (departmental) faculty supervision score (i.e., mean among 9 questions × 1 evaluation) to the same resident's daily evaluations of individual faculty (i.e., mean among 9 questions × many evaluations). All 39 of 39 residents chose to participate. The mean departmental supervision score was significantly less (P < 0.0001) than the mean of individual faculty scores. The median ratio of scores was 86% (95% confidence interval, 83%-89%). Kendall's rank correlation between global and (mean) individual faculty scores was τb = 0.34 ± 0.11 (P = 0.0032). The ratios were uniformly distributed (P = 0.64) between the observed minimums and maximums; were not correlated with the mean value of individual faculty scores previously provided by each resident (P = 0.64); were not correlated with the number of individual faculty evaluations previously provided by each resident (P = 0.49); and did not differ among the first, second, or third year residents (P = 0.37). Residents' perceptions of overall (departmental) faculty supervision were less than overall averages of their perceptions of individual faculty supervision. This should be considered when interpreting national survey results (e.g., of patient safety), residency program evaluations, and individual faculty anesthesiologist performance.