Assessing Competence of Orthopaedic Residents: The Reliability and Validity of an Objective Structured Clinical Examination After a Sports Medicine Rotation

Abstract

The traditional method of orthopaedic training in Canada utilizes a time-based system, combined with a formal exit examination. Increasing interest in competency-based curriculum has raised the issue of how to test in-training competence. Currently, the most commonly used method is the In-Training Evaluation Report (ITER), which is known to be relatively subjective in nature; the true establishment of competence would likely benefit from the addition of an objective assessment1,2. The definition of competence varies, but relates to an individual’s ability to perform in the workplace to the required standard3. In surgery, competence is required in many areas, including surgical skill and clinical decision-making. Currently, orthopaedics is lacking a reproducible, objective measurement of resident competence. Objective Structured Clinical Examinations (OSCEs) are an important aspect of certifying examinations in many countries. First developed in the late 1970s, an OSCE consists of multiple, timed stations at which each candidate is faced with a discrete patient or case-based clinical task and is evaluated in an objective and structured way; both the task and the assessment are standardized4-6. An OSCE is especially suited to testing aspects of the role of medical expert4,7,8. To our knowledge, there is very little published literature on the validity and reliability of OSCEs in orthopaedics. Recently, Beran et al. used an OSCE to assess history-taking and physical examination skills of orthopaedic residents, which served to highlight deficiencies in resident knowledge9. The ability to formulate valid and reliable in-training OSCEs may become an essential tool in competency-based orthopaedic training and may be used to assess resident competence regularly throughout training, to identify residents with knowledge deficits compared with their peers, and to allow appropriate remediation. A compulsory module at our university is a …