Cognition and the question of test item format

Abstract

The effects of testing on cognition bear not only on the meaning of what is measured, but also its consequences. Test item formats vary in their typical cognitive demand and in the range of cognitions they sample. Multiple-choice items, in particular, often elicit low-level cognitive processing whereas constructed-response items more often evoke complex thinking. However, the typical cognitions elicited by differing response formats are less a reflection of the limitations of those formats than they are of typical use. Even when research demonstrates that item response formats vary in cognitive features, psychometric characteristics, and costs of administration and scoring, policy implications remain ambiguous because of tradeoffs along these dimensions. No one format is appropriate for all purposes and on all occasions. This article presents the state of the cumulative research bearing on the differential cognitive demand of item formats, and explores implications of the extant research for testing practice.