The Examination of Accreditation for Foreign Medical Graduates in Greece: an Evaluation of the Multiple Choice Questions Format Using the Difficulty and Discrimination Indices

Abstract

BACKGROUND: The multiple choice question (MCQ) format is the most commonly used written assessment technique for the accreditation of foreign medical graduates in Greece. OBJECTIVES: To evaluate the appropriateness of the range of the multiple choice questions used in the examination for the accreditation of foreign medical graduates in Greece, and to compare the performance among foreign medical graduates, Greek medical students and interns (medical doctors during their residency). METHODS: Twenty-six items from the internal medicine question paper and 24 items from the surgery question paper were randomly selected from an MCQ format used in one assessment. For these items discrimination and difficulty indices were calculated for separate groups of candidates and volunteer participants (53 medical students and 30 interns). Comparisons were made between group scores, first considering the whole questionnaire as a single entity and then using scores for each discipline calculated separately. RESULTS: A significant number of “inappropriate” questions were included in the examination. Surgery questions were more candidate-oriented, given the best range of acceptable difficulty index values for that group of participants, while internal medicine questions proved to be more appropriate for medical students. Furthermore, comparisons of groups performed using a total score over the whole range of the two disciplines revealed a significantly better performance of interns compared with students (p<0.001), whereas comparisons performed separately for each discipline revealed no significant difference between interns and students in surgery scores but a significant difference in internal medicine scores (p<0.001). CONCLUSION: These findings suggest the importance of the evaluation of the MCQs before using them in examinations, aiming at revising inappropriate questions. In order to evaluate the participants’ performance, calculation of scores across separate disciplines is proposed, since it is less likely to be biased towards good performance in the questions of one discipline. ORIGINAL ARTICLE Department of Clinical Therapeutics, Medical School, University of Athens, “Alexandra” Hospital, Athens, Greece HOSPITAL CHRONICLES 2011, 6(2): 79–83