Abstract
BackgroundMultiple choice questions are commonly used in summative assessment. It is still common practice for tertiary institutions and accrediting bodies to use five-option single best answer multiple choice questions, despite a substantial body of evidence showing that multiple choice questions with only three or four options provide effective and discriminatory assessment.MethodsIn this study we investigated the distribution of distractor efficacy in exams from four large first-year undergraduate courses in chemistry and in anatomy and physiology in a Health Faculty; assessed the impact on overall student score after changing from five-option to four-option single best answer multiple choice questions; and assessed the impact of changing from five options to four options on item difficulty and discrimination.ResultsFor the five-option questions analysed, 19% had four effective distractors, which is higher than previous studies, but still a minority of questions. After changing from five to four options, the overall student performance on all multiple choice questions was slightly lower in the second offering of one course, slightly higher in the second offering of another course, and similar in the second offering for two courses. For a subset of questions that were used in both offerings, there were negligible differences in item difficulty and item discrimination between offerings.ConclusionsThese results provide further evidence that five-option questions are not superior to four-option questions, with reduction to four options making little if any difference to overall performance, particularly when MCQ is used in conjunction with other assessment types (including short answer questions, and practical or laboratory assessment). Further areas of study that arise from these findings are: to investigate the reasons for resistance to changing established assessment practice within institutions and by accrediting bodies; and to analyse student perceptions of the impact of a reduced number of options in MCQ-based assessment.
Highlights
Multiple choice questions are commonly used in summative assessment
This was in part justified by the fact that some health-related professional bodies use this format of Multiple Choice Question (MCQ), including the Australian Medical Council
Discrimination index (DI) measures the extent to which a particular item response is able to discriminate between individuals who attain a high score on the overall MCQ result and those that attain a low score; for these data, DI = (U-L)÷NU, where U is the number of students in the upper quartile that selected that response, L is the number in the lower quartile, and NU is the total number of people in the upper quartile
Summary
Multiple choice questions are commonly used in summative assessment. It is still common practice for tertiary institutions and accrediting bodies to use five-option single best answer multiple choice questions, despite a substantial body of evidence showing that multiple choice questions with only three or four options provide effective and discriminatory assessment. Multiple Choice Question (MCQ) assessments provide the advantage of rapid (usually automatic) marking and return of results, which are important considerations for large class sizes requiring rapid turnaround of results. At Griffith University, students entering a range of undergraduate health programs undertake a foundation year, with common courses in their first two semesters. These courses cater for a large number of students with diverse academic abilities, and many proceed to later postgraduate degrees in health professions. Courses within this foundation program use MCQs to achieve rapid turnaround in marks with very large classes. This was in part justified by the fact that some health-related professional bodies use this format of MCQs, including the Australian Medical Council
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
