Patient, faculty, and self-assessment of radiology resident performance:: A 360-degree method of measuring professionalism and interpersonal/communication skills 1

Abstract

Rationale and objectives To develop and test the reliability, validity, and feasibility of a 360-degree evaluation to measure radiology resident competence in professionalism and interpersonal/communication skills. Materials and methods An evaluation form with 10 Likert-type items related to professionalism and interpersonal/communication skills was completed by a resident, supervising radiologist and patient after resident-patient interactions related to breast biopsy procedures. Residents were also evaluated by faculty, using an end-of-rotation global rating form. Residents, faculty, and technologists were queried regarding their reaction to the assessments after a 7-month period. Results Fifty-six complete 360-degree data sets (range, 2–14 per resident) and seven rotational evaluations for seven residents were analyzed and compared. Internal consistency reliability estimates were 0.85, 0.86, and 0.87 for resident, patient, and faculty 360-degree evaluations, respectively. Correlations between resident-versus-patient, resident-versus-faculty, and patient-versus-faculty ratings for the 56 interactions were −0.06 ( P = .64), 0.31 ( P < .02), and 0.45 ( P < .0006), respectively. Pearson correlation coefficients approached significant correlation (0.70) between the faculty global rating and patient 360-degree scores ( P = .08) but not with faculty 360-degree scores. Residents and faculty felt that completing the 360-degree forms was easy, but the requirement for faculty presence during the consent process was burdensome. Conclusion Results from this pilot study suggest that self, faculty, and patient evaluations of resident performance constitutes a valid and reliable assessment of resident competence. Additional data are needed to determine whether the 360-degree assessment should be incorporated into residency programs and how frequently the assessment should be performed. Requiring only a specified number of assessments per rotation would make the process less burdensome for residents and faculty.