Consistency of standards and stability of pass/fail decisions with examinee-based standard-setting methods in a small-scale objective structured clinical examination.

Abstract

The apparent feasibility and the face validity of the examinee-based methods such as the borderline-group methods provide support for their increasing adoption by health profession schools. Before that can occur, however, more information on the quality of the standards produced by these techniques is required. The purpose of the present study was to assess the quality of the standards produced on a small-scale objective structured clinical examination (OSCE) by the borderline-group and contrasting-groups examinee-based standard-setting procedures. These two examinee-based criterion-referenced standard-setting procedures were done for an undergraduate fourth-year surgical OSCE and the consistency of the standards and the decisions arising from the standards were assessed. Both techniques provided consistent and realistic standards. There is sufficient theoretical and empirical evidence to support the use of examinee-based standard-setting techniques in small-scale OSCEs that use expert examiners.