BACKGROUND: MCQs can test different levels of cognitive knowledge from recall to analysis. This study was to determine the item writing flaws in multiple choice questions and their effects on students’ academic achievements and test reliabilities. Methods: This study was conducted at the Azad Jammu Kashmir (AJK) Medical College Muzaffarabad from December 2017 to June 2019. Ten end-of-block exams were used in this study. The item review committee went over every MCQ. Two tests from each class of MBBS were taken. The initial tests without being reviewed were deemed to be flawed tests. The outcomes of each test were assessed, and the students were ranked into three achievement levels high, moderate, and low based on their scores of >79.9, between 50 and 79.9, and <50%, respectively. After removal of flawed items from the exams, the scores of each test (the standard test) were calculated, compared to the flawed tests, and its impacts were evaluated among three achievement groups. The post-exam analysis of test items was done on optical mark reading (OMR) classic-4 programme and data analysis on SPSS-25. Results: In one test i.e. test no 6 the null hypothesis was rejected with p<0.05. However, in all other tests null hypothesis could not be rejected on statistical basis. Moderate achievers and low achievers had statistically significant correlation (p<0.05). The flawed tests had better reliabilities with statistically significant (p<0.05). Conclusion: Flawed MCQs had negative impact on high and moderate achievers. Reliabilities of flawed test were greater than standard tests.
Read full abstract