CONVERGING ON CORRECT ANSWERS: A PECULIARITY OF MULTIPLE CHOICE ITEMS

Abstract

The primary mode of mental measurement in the United States today is the multiple choice item. The multiple choice format has many commendable qualities: it is relatively straightforward in terms of what is required from the subject, it is well-understood by most test-takers, it affords highly objective scoring, and it is computer scannable. The format does have some drawbacks: it is somewhat limiting in the type of mental behaviors which can be elicited, it is subject to guessing, and it interacts with characteristics of test-takers. It is this final aspect that is the subject of this research. There are certain aspects of multiple choice items which are consistent across the traits or content areas being measured. The basic form of a multiple choice item includes: (1) a set of data pertinent to the item, (2) a question or task which must be responded to, and (3) a set of alternative answers, one of which is correct. Some multiple chdice items require the test-taker to solve a problem and then locate his answer among the distractors (such as mathematics and some abstract reasoning items). Other multiple choice items require the test-taker to compare and contrast the alternative responses and pick the best one (such as reading comprehension and sentence completion items). For some items either approach may be used. An unstated assumption of multiple choice testing is that all test-takers are equally adept at responding to problems presented in the fashion just described. If aspects of the multiple choice format interact with characteristics of test-takers, then the resultant measures will be confounded to some degree. Specifically, some individuals may find the multiple choice format confusing while others find that it provides inadvertant clues to correct answers. As a result, scores are deflated or inflated due to an ability which is logically independent of the trait being measured. When the interaction between the characteristics of an item and a person is viewed from the perspective of the item, it is generally considered to be an issue of item construction; when viewed from the perspective of the person, it is considered to be an issue of testwiseness. Although there have been a number of interesting and potentially useful advances in item writing over the past decade (e.g., Bormuth, 1970 and Guttman, 1970, echoed more recently by Wood, 1977 and Nunnally, 1978), the contention of Wesman (1971) that item writing is still an art holds true in general. If item writing has proven somewhat resistant to being turned into a science, it is highly restrictive as an art form. Most texts on test construction provide a list of prescriptions and proscriptions for budding item writers. One major purpose of these lists is to keep the item writer from inadvertantly cueing the test-taker as to which of the choices is correct. There are a variety of ways in which the correct answer might be revealed to the adept test-taker. Gronlund (1977) has an excellent list of pitfalls for those who are not familiar with them.