Peer-level multiple source feedback for fitness to practice

Abstract

Context and setting The attitude of tomorrow's doctors is considered vital, as ineffective and inappropriate communication between patients and clinical staff remains the greatest source of patient dissatisfaction. It is therefore important to integrate evaluation of medical student attitudes into the modern medical curriculum alongside problem-orientated and evidence-based group learning approaches. The use of multiple-source feedback by peers is an attractive potential tool. Why the idea was necessary The use of 360-degree evaluation utilises multiple, independent perspectives to assess teamwork, communication skills, management skills and clinical decision making. This type of multiple source feedback can be a powerful driver for attitude development, particularly when incorporating the views of peers from a shared working environment. Its increasing and widespread use in medical practice means that it is desirable to familiarise students to this process early in their training. What was done Fitness-to-practice evaluation forms, normally completed for each student by 3 tutors with close student contact, were distributed to members of a problem-based learning (PBL) group. Students completed forms for each of the other group members and results were compared to those prepared by the tutors. Results for each group member were presented to them in individual interviews. Evaluation of the results and impact Sets of 3 tutor reports were compared for each student; 8 criteria were examined. Of 240 scores obtained only 42 varied from the modal value − a very close (82·5%) correlation; 44 scores were exceptional (18%) and no unsatisfactory scores were given in the semester chosen. A total of 8 peer reports were compared for each student. From 584 scores only 155 varied from the modal value of each criterion, indicating good (74%) correlation − only marginally less than the degree of correlation observed between tutors. In 245 out of 584, scores were in the exceptional category (42%), far higher that those obtained from tutors. Scores in 10 out of 584 (2%) were unsatisfactory: still low, but significantly higher than the tutors' scores, indicating more polarity in the students' marking of each other. Peer reports were sharply critical in places, although these criticisms had not surfaced previously during PBL tutorials. Students felt liberated by the exercise to voice criticisms in areas inaccessible to academic and clinical tutors. Tutor scores did not correlate well with student scores, with 99 of the 240 scores diverging from the student modal value (59% correlation), reflecting the increased general scoring of the students compared with tutors. Self-evaluations also correlated poorly, with scores diverging from peers (65% correlation) and tutors (55% correlation). Students appeared to find it difficult to be honest and objective about their own performance. The use of self-assessment, however, was perceived as a useful tool formatively, as it emphasised to the student how their self-perception conflicted with how others perceived them, prompting reconciliation. Students' peer evaluations gauged and compensated accurately for extenuating circumstances, which tutors could not. Students reacted strongly to behaviour which affected them directly, making their evaluations more polar, and found difficulty in evaluating close colleagues objectively, suggesting that moderation might be necessary for utility in summative evaluation.