Variation in assessment of first-year medical students' interprofessional competencies by rater profession

Abstract

It is important to assess and provide formative feedback to medical students. During interprofessional experiences, it is essential for students to receive formative, multi-source feedback during these experiences from interprofessional colleagues. Yet, implementing methods to assess students' collaboration and team-based patient care has remained a challenge. This study investigated the impact of the assessor's profession on students' performance scores. In the fall of 2015, for 8 sessions, first-year medical students were placed in diverse clinical settings during their Interprofessional Clinical Experience course and paired with health professionals from whom they requested assessments. Students were assessed on communication, professionalism, leadership/teamwork/interprofessionalism, and overall performance using a 9-point Likert-type scale. Assessments were analyzed by profession using multilevel modeling. The total number of students enrolled in the course was 167 and the total number of assessments completed was 743 with a completion rate of 66.9% with a mean of 4.4 assessments/students (SD 1.6) and a mean overall score of 4.0 (SD 2.3). Three-hundred fifty-four assessors from 9 professions provided 648. Results from multilevel analysis showed statistically significant differences in assessment by profession, with medical assistants and clerks scoring students significantly higher than nurses, nurse practitioners/physician assistants, rehabilitation therapists, and physicians ( p < 0.05). Physicians scored students significantly lower than 6 of 8 other professions. The assessor's profession made a significant impact on assessment scores of students with some professions more or less stringent. Variability in assessment by profession should be considered when interpreting student performance and providing feedback. While training may help reliability of assessors, this may be difficult given the number and breadth of assessors.