A structured assessment of newly qualified medical graduates

Abstract

While there is extensive published experience with the assessment of procedural skills in undergraduate students, this is limited in newly qualified medical graduates at the time of entry to the pre-registration (internship) year. The few studies that have been published suggest that these skills are frequently deficient when objectively tested. We therefore chose to assess the competence of a group of South African medical graduates on entry to their pre-registration year. A total of 58 graduates of South African medical schools were assessed. Each subject participated in a 7-station objective structured clinical examination (OSCE); 6 of these assessed individual competence in phlebotomy, intramuscular injection, female pelvic examination, bladder catheterisation, tracheal intubation and prescription writing, while competence in cardiopulmonary resuscitation was assessed in a seventh station in randomly allocated teams of 3 candidates. Candidates' opinions of their own competence was sought by questionnaire. There was a wide variation in competence between subjects and across the range of tasks studied. Mean scores ranged from 85.4% for phlebotomy to 55.3% for prescription writing. The average score across all stations was 67.5%, and no student obtained an overall cut-off score of 85% or more, which was established using a modified Angoff method. Subjects' assessment of their own performance was unduly optimistic; most believed that they had demonstrated competence despite clear shortcomings in technique. Objective scores for subjects who had been exposed to a structured skills laboratory programme were not significantly higher than for those who had not, although their self-assessed performance was indeed higher. Most of the South African medical graduates who participated in this study were unable to satisfactorily perform technical procedures appropriate to the house officer on entry to the pre-registration year. This is in line with the conclusions of the few studies published in other countries. We suggest that the learning outcomes of undergraduate medical programmes should include an explicit statement of the competencies required for practice in the pre-registration year, and that these should be adequately taught and rigorously assessed before graduation.