Standard Setting in a Small Scale OSCE: A Comparison of the Modified Borderline-Group Method and the Borderline Regression Method

Abstract

When setting standards, administrators of small-scale OSCEs often face several challenges, including a lack of resources, a lack of available expertise in statistics, and difficulty in recruiting judges. The Modified Borderline-Group Method is a standard setting procedure that compensates for these challenges by using physician examiners and is easy to use making it a good choice for small scale OSCEs. Unfortunately, the use of this approach may introduce a new challenge. Because a small scale OSCE has a small number of examinees, there may be few examinees in the borderline range, which could introduce an unintentional bias. A standard setting method called The Borderline Regression Method will be described. This standard setting method is similar to the Modified Borderline-Group Method but incorporates a linear regression approach allowing the cut score to be set using the scores from all examinees and not from a subset. The current study uses confidence intervals to analyze the precision of cut scores derived from both approaches when applied to a small scale OSCE.