Effects of Response Format on Difficulty of SAT‐Mathematics Items: It's Not the Strategy

Abstract

Problem‐solving strategy is frequently cited as mediating the effects of response format (multiple‐choice, constructed response) on item difficulty, yet there are few direct investigations of examinee solution procedures. Fifty‐five high school students solved parallel constructed response and multiple‐choice items that differed only in the presence of response options. Student performance was videotaped to assess solution strategies. Strategies were categorized as “traditional”–those associated with constructed response problem solving (e.g., writing and solving algebraic equations)–or “nontraditional”–those associated with multiple‐choice problem solving (e.g., estimating a potential solution). Surprisingly, participants sometimes adopted nontraditional strategies to solve constructed response items. Furthermore, differences in difficulty between response formats did not correspond to differences in strategy choice: some items showed a format effect on strategy but no effect on difficulty; other items showed the reverse. We interpret these results in light of the relative comprehension challenges posed by the two groups of items.