Correlation of selection scores with subsequent assessment scores during surgical training

Abstract

Determining admission criteria to select candidates most likely to succeed in surgical training in Australia and New Zealand has been an imprecise art with little empirical evidence informing decisions. Selection to the Royal Australasian College of Surgeons' Surgical Education and Training programme is based entirely on applicants' performance in structured curriculum vitae (CV), referees' reports and interviews. This retrospective review compared General Surgery (GS) trainees' performance in selection with subsequent performance in assessments during training. Data from three cohorts of GS trainees were sourced. Scores for four selection items were compared with scores from six training assessments. Interrelationships within each of the sets of selection and assessment variables were determined. A single significant relationship was found between scores on the three selection tools. High scores in the CV did not correlate with higher scores in any subsequent assessments. The structured referee report score, multi-station interview score and total selection score all correlated with performance in subsequent work-based assessments and examinations. Direct observation of procedural skills (DOPS) scores appear to reflect increasing acquisition of operative skills. Performance in mini clinical examinations (Mini-CEX) was variable, perhaps reflecting limitations of this assessment. Candidates who perform well in one examination tend to perform well in all three examinations. No selection tool demonstrated strong relationships with scores in all subsequent assessments; however referee reports, multi-station interviews and total selection scores are indicators for performance in particular assessments. This may engender confidence that candidates admitted into the GS training programme are likely to progress successfully through the programme.