Editor's Spotlight/Take 5: Objective Structured Assessments of Technical Skills (OSATS) Does Not Assess the Quality of the Surgical Result Effectively.

Abstract

H ow we train our residents to become orthopaedists affects all of us, whether as educators now, or patients of these young surgeons later. So naturally we ask, given time and resource constraints in our training programs, are residents competent when they graduate? And how do we determine competence, anyway? The Accreditation Council for Graduate Medical Education and the American Board of Orthopaedic Surgery’s Milestones program provide some more objective definitions, and perhaps more than before, we now assess competence rather than merely track time in service [6]. Against this backdrop of more rigorous assessments in surgical education, surgical educators have developed some more objective tools. In particular, the Objective Structured Assessment of Technical Skills (OSATS) has gained wider acceptance, with recent iterations including global rating scales and detailed checklists [7]. A recent review [3] concluded that the OSATS is generally a valid tool for formative feedback to the learner, but evidence of OSATS’s validity remains insufficiently robust to use this tool for high-stakes examinations and certifications. Why might there be problems with the OSATS’s validity? First, we must remember that validity is a reflection of how well something conforms to reality, and depending on what aspect is measured, validity can be divided into different types. The OSATS, as an evaluation tool for technique and skill, performs well in terms of construct validity; that is, the test seems to measure accurately the things it sets out to measure [2]. However, it is less clear that we are measuring the most important things in the first place (content validity), and we just do not know to what degree the measurements predict important realworld endpoints, like the effective performance of actual surgery (concurrent validity). How might we measure the technical proficiency of a surgical learner in the context of actual (or simulated) surgery? When piecing together intraarticular fracture fragments, we strive Note from the Editor-In-Chief: In ‘‘Editor’s Spotlight,’’ one of our editors provides brief commentary on a paper we believe is especially important and worthy of general interest. Following the explanation of our choice, we present ‘‘Take Five,’’ in which the editor goes behind the discovery with a oneon-one interview with an author of the article featured in ‘‘Editor’s Spotlight.’’ The author certifies that he, or any members of his immediate family, has no commercial associations (eg, consultancies, stock ownership, equity interest, patent/licensing arrangements, etc) that might pose a conflict of interest in connection with the submitted article. All ICMJE Conflict of Interest Forms for authors and Clinical Orthopaedics and Related Research editors and board members are on file with the publication and can be viewed on request. The opinions expressed are those of the writers, and do not reflect the opinion or policy of CORR or The Association of Bone and Joint Surgeons. This comment refers to the article available at: DOI: 10.1007/s11999-015-4603-4. M. D. Wongworawat MD (&) Clinical Orthopaedics and Related Research1, 1600 Spruce Street, Philadelphia, PA 19013, USA e-mail: mwongworawat@clinorthop.org Editor’s Spotlight/Take 5 Published online: 4 January 2016 The Association of Bone and Joint Surgeons1 2015