Abstract
PurposeSetting standards is critical in health professions. However, appropriate standard setting methods do not always apply to the set cut score in performance assessment. The aim of this study was to compare the cut score when the standard setting is changed from the norm-referenced method to the borderline group method (BGM) and borderline regression method (BRM) in an objective structured clinical examination (OSCE) in medical school.MethodsThis was an explorative study to model the implementation of the BGM and BRM. A total of 107 fourth-year medical students attended the OSCE at 7 stations for encountering standardized patients (SPs) and at 1 station for performing skills on a manikin on July 15th, 2021. Thirty-two physician examiners evaluated the performance by completing a checklist and global rating scales.ResultsThe cut score of the norm-referenced method was lower than that of the BGM (P<0.01) and BRM (P<0.02). There was no significant difference in the cut score between the BGM and BRM (P=0.40). The station with the highest standard deviation and the highest proportion of the borderline group showed the largest cut score difference in standard setting methods.ConclusionPrefixed cut scores by the norm-referenced method without considering station contents or examinee performance can vary due to station difficulty and content, affecting the appropriateness of standard setting decisions. If there is an adequate consensus on the criteria for the borderline group, standard setting with the BRM could be applied as a practical and defensible method to determine the cut score for OSCE.
Highlights
Background/rationale Standard setting is a process by which human judgment can be synthesized in a rational and defensible way to classify score scales into categories [1]
This study reported and provided a real example of the implementation of 2 standard setting methods for objective structured clinical examination (OSCE)
The cut score of the norm-referenced method was lower than that of the borderline group method (BGM) and borderline regression method (BRM), and there was no significant difference in the cut score between the BGM and BRM
Summary
Background/rationale Standard setting is a process by which human judgment can be synthesized in a rational and defensible way to classify score scales into categories [1]. There is no ‘gold standard’ in regard to setting the cut score in real examinations, this activity is critical in health professions [2]. This is the minimum judgment on the per-. Criterion-referenced standard setting methods are of 2 types: test-centered and examinee-centered. The former is appealing for setting a cut score on knowledge assessment, such as multiple-choice examination, and the latter is well suited for performance assessment, such as objective structured clinical examination (OSCE) [3]. Well-known examinee-centered standard settings are the contrasting groups method and the borderline group method (BGM)
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callMore From: Journal of educational evaluation for health professions
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
