A solution to the cueing effects of multiple choice questions: the Un-Q format.

Abstract

Although the cueing effects inherent in conventional multiple choice questions (MCQs) present serious limitations, this format continues to dominate testing programmes. The present study was undertaken to estimate the effects of cueing when MCQs are used to test medical students, and to evaluate the reliability, validity and feasibility of an alternative testing format. Equivalent items in both MCQ and open-ended, or uncued (Un-Q), formats were administered to 34 third- and fourth-year medical students. The students' mean % correct score on the MCQs was 11 percentage points higher than their mean level of performance on equivalent Un-Qs. When a second set of more difficult items was administered to 16 of these students, their mean performance on the MCQ items was 22 percentage points higher than their performance on equivalent Un-Qs. The results support the feasibility of large group administration of tests constructed in an open-ended format that can be scored by computer. Not only is this format equally reliable and economical when compared with the MCQ, but it also provides important advantages that strengthen its face validity. The Un-Q format can be used to test either simple recall or certain higher level problem-solving skills that cannot be tested by MCQs. Even more important, the results also suggest that the Un-Q format may be a more effective discriminator of academically marginal examinees.