Evaluation of Passing Scores in Semiotics: An Objective Structured Clinical Examination for Medical Students of Mashhad University of Medical Sciences, Mashhad, Iran, 2015

Abstract

Background Numerous exams are held at different levels and in different fields of medical sciences to evaluate students’ practical knowledge. In pass-fail exams where several examiners score the students, it is important to determine “the minimum passing score” or “the passing score” to determine whether students have passed or failed; this score is sometimes called the “cut-off point” or “standard score.” The objective structured clinical examination (OSCE) method is employed for the final assessment of medical students in Semiotics I in Mashhad University of Medical Sciences, Mashhad, Iran. The commonly used standard scoring method for this lesson is the fixed score method, which sometimes results in a discrepancy between educational management and the lecturers. Hence, the current study aims to compare 4 different methods—the Cohen, borderline-group, borderline regression, and Hofstee methods—of determining the passing score in the semiotics course and comparing the results with those of the fixed score method. Methods A 6-station OSCE was used to assess Semiotics I in Mashhad University of Medical Sciences in 2015. In the current study, in order to determine a standard scale for scoring the students, two forms, Forms 1 and 2, and a checklist were completed for each student. In Form 1, a 5-option Likert scale scoring system, graded from poor to excellent, was used. Data from Form 1 were analyzed using the borderline regression and borderline-group methods. Form 2 included 4 items and the collected data were analyzed using the Hofstee method. Data collected from both forms were analyzed, after the exams, using SPSS version 16. Results The cut-off point established by the Cohen method was very close to that of the common method. In other words, there was no significant difference between the cut-off point determined by the Cohen method (11.73) and that of the common method (12). The other study methods, however, such as borderline regression and borderline-group methods proposed higher cut-off points, which were significantly different from that of the common method: more students failed Semiotics I using these methods. The Hofstee method cannot be used in the OSCE, as the results were insignificant. Conclusions Because there was a significant difference in the number of students who passed the exam based on the fixed score and Cohen methods, and on the borderline-group and borderline regression methods, it is recommended that the latter methods not be widely employed. In addition, it is suggested that different methods should be used to define a mean standard passing score because, according to the statistics, an accurate and efficient estimator with minimum variances accuracy should be employed to evaluate population parameters, and the mean estimator would benefit from such advantages.