Abstract
BackgroundMost assessments in health professions education consist of knowledge-based examinations as well as practical and clinical examinations. Among the most challenging aspects of clinical assessments is decision making related to borderline grades assigned by examiners. Borderline grades are commonly used by examiners when they do not have sufficient information to make clear pass/fail decisions. The interpretation of these borderline grades is rarely discussed in the literature. This study reports the application of the Objective Borderline Method (version 2, henceforth: OBM2) to a high stakes Objective Structured Clinical Examination undertaken at the end of the final year of a Medicine program in Australia.MethodsThe OBM2 uses all examination data to reclassify borderline grades as either pass or fail. Factor analysis was used to estimate the suitability of data for application of OBM2. Student’s t-tests, utilising bootstrapping, were used to compare the OBM2 with ‘traditional’ results. Interclass correlations were used to estimate the association between the grade reclassification and all other grades in this examination.ResultsThe correlations between scores for each station and pass/fail outcomes increased significantly after the mark reclassification, yet the reclassification did not significantly impact on students’ total scores. Examiners, students and program leaders expressed high levels of satisfaction and the Faculty’s Curriculum Development Committee has decided that the OBM2 will be used for all future clinical examinations. Implications of the OBM2 are discussed.ConclusionsThe OBM2 provides a feasible, defensible and acceptable solution for classification of borderline grades as either pass or fail.
Highlights
Most assessments in health professions education consist of knowledge-based examinations as well as practical and clinical examinations
This study describes the application of a revised version of the Objective Borderline Method (OBM), known as OBM2 [23, 24], to a high stakes Objective Structured Clinical Examinations (OSCE) undertaken at the end of the final year of the Medicine program at UNSW Medicine, Sydney, Australia
The OBM2 provides an effective, feasible and defensible solution for utilisation of borderline grades generated in clinical examinations
Summary
Most assessments in health professions education consist of knowledge-based examinations as well as practical and clinical examinations. Borderline grades are commonly used by examiners when they do not have sufficient information to make clear pass/fail decisions. For clinical skills assessment in health professions education, it is commonly believed that examiners apply their best judgement when providing feedback on student performance in Objective Structured Clinical Examinations (OSCE) [1]. Previous studies suggest that examiners felt less confident when giving a fail grade than when giving a pass grade [2, 3]. Influences such as examiners’ familiarity with the examinees [4], examinees’ first impression on examiners [5], and other biases such as gender and culture, may impact on examiners’ judgements [6]. A recent study found that changing examiners at a station during the United Kingdom postgraduate surgery OSCEs made a significant difference to students’ scores, the reliability of the OSCE did not change [10]
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
