Abstract
PurposeIn a sequential objective structured clinical examination (OSCE), all students initially take a short screening OSCE. Examinees who pass are excused from further testing, but an additional OSCE is administered to the remaining examinees. Previous investigations of sequential OSCE were based on classical test theory. We aimed to design and evaluate screening OSCEs based on item response theory (IRT).MethodsWe carried out a retrospective observational study. At each station of a 10-station OSCE, the students’ performance was graded on a Likert-type scale. Since the data were polytomous, the difficulty parameters, discrimination parameters, and students’ ability were calculated using a graded response model. To design several screening OSCEs, we identified the 5 most difficult stations and the 5 most discriminative ones. For each test, 5, 4, or 3 stations were selected. Normal and stringent cut-scores were defined for each test. We compared the results of each of the 12 screening OSCEs to the main OSCE and calculated the positive and negative predictive values (PPV and NPV), as well as the exam cost.ResultsA total of 253 students (95.1%) passed the main OSCE, while 72.6% to 94.4% of examinees passed the screening tests. The PPV values ranged from 0.98 to 1.00, and the NPV values ranged from 0.18 to 0.59. Two tests effectively predicted the results of the main exam, resulting in financial savings of 34% to 40%.ConclusionIf stations with the highest IRT-based discrimination values and stringent cut-scores are utilized in the screening test, sequential OSCE can be an efficient and convenient way to conduct an OSCE.
Highlights
Objective structured clinical examinations (OSCEs), especially if composed of a large number of stations and administered to a signif icant number of candidates, consume an excessive amount of reso urces
It has been suggested that the cost of the OSCE can be reduced by using a sequential design [1,2], which can be considered a simple form of adaptive testing
Several studies have explored the use of sequential OSCE
Summary
Objective structured clinical examinations (OSCEs), especially if composed of a large number of stations and administered to a signif icant number of candidates, consume an excessive amount of reso urces. A sequential test was not administered; instead, a hypothetical testing situation was considered, in which several screen ing tests were designed based on retrospective data and their results were compared to the main OSCE decisions [3,4,5]. It seems that a substantial reduction in testing time and resources can be achi eved through this approach, careful consideration is required when designing sequential OSCEs. The screening OSCE, which is a rela tively short and a fairly unreliable test, should accurately and efficiently identify students at the upper ability levels.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callMore From: Journal of educational evaluation for health professions
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
