A Criticism of Tests Requiring Alternative Responses

Abstract

Before further use is made of Yes-No Questions, Opposite Tests, Alternative Questions, or True-False State ments for testing mental ability or achievement in subject matter, the nature and reliability of such tests should once more be critically examined. In this article this form of tests will be discussed, first, as to its accuracy and, second, as to its influence on teaching when used to measure achievement. Obviously one of its dangers is the guessing factor. Because the right answer is either Yes or No, True or False, Same or Opposite, one has a chance of guessing one-half of the answers correctly. For this reason an attempt must be made in scoring to eliminate the effect of guessing. The method adopted is to subtract the wrong from the right answers, and to accept the remainder as an accurate measure of ability or achievement. But does subtracting the wrong answers from the right eliminate the effect of guessing from the test? If one holds that by subtracting the wrong from the right answers one eliminates the guessing factor, no more and no less, one must assume that individuals in such tests always guess an even number of times; for if they should guess an odd number of times the effect of guessing could not be wholly eliminated by this method. One must further assume that if an individual happens to guess wrong the first time his second guess must be right, his third wrong, his fourth right, and so on, guessing wrong and right alternately; for if he should guess wrong or right twice in succession, the method could not eliminate the guessing effect. That the law of chance does not operate even approximately in this way can easily be demonstrated. For the following experiment seventy buttons were used. These buttons were alike in every way except that thirty-five 236