An evaluation of a final year multiple choice questions examination at Faculty of medicine-university of Benghazi

Abstract

Background: Multiple choice questions (MCQs) are widely used in medical education, they have the feature of sampling broad domains of knowledge efficiently and reliably. They represent one of the most important examination tools used in undergraduate and postgraduate medical examinations. Exam analysis is an assessment tool that provides important information about validity and reliability of test. Item analysis is used to analyze the difficulty and the discrimination ability of individual items and the total test scores. The purpose of this study is to examine the quality of one of the final exam papers, that is composed of single best of five MCQs, and to examine the correlation between the difficulty and discrimination index of each item. Subject and method: The authors collected the exam results of 425 students who set for the final exam at the end of the a academic year 2021- 2022. The exam consisted of 70 MCQs and was aiming to assess the students` knowledge, understanding, application and analysis of the subject matter. The test was analyzed regarding content validity and reliability of the test as a whole as well as the difficulty coefficient and discrimination coefficient for each item. Results: Out of the 425 students who set for the exam about 86% has passed (scored ≥ 60% of the total marks), the test reliability was excellent (0.91) and the mean difficulty index (P) was 75.22 ± 19.4 (easy exam). Out of the 70 items six (8.57%) had a poor discrimination (D < 0.1). When comparing the exam questions with the blueprint 90% of the topics of the subject materials were either over or under represented. About 77% of the exam questions were testing either students` knowledge or understanding. The correlation between the difficulty index and discrimination index was moderately positive which was statistically significant (0.633, P = 0.000). Conclusion: although the current study showed a good exam reliability, however the exam content validity was suffering from many weakness regarding the total number of the items, and distribution of the items according to the topics and the cognitive levels, therefore the number of exam questions should be increased and a blueprint should be developed for assessment. The overall exam difficulty was low as there were too many easy questions and this could impair both exam validity and reliability. The number of average and difficult questions should be considerably increased.