Asymmetry in student achievement on multiple-choice and constructed-response items in reversible mathematics processes

Christopher J. Sangwin,Ian Jones

doi:10.1007/s10649-016-9725-4

Christopher J. Sangwin, Ian Jones

Open Access

https://doi.org/10.1007/s10649-016-9725-4

Copy DOI

Abstract

In this paper we report the results of an experiment designed to test the hypothesis that when faced with a question involving the inverse direction of a reversible mathematical process, students solve a multiple-choice version by verifying the answers presented to them by the direct method, not by undertaking the actual inverse calculation. Participants responded to an online test containing equivalent multiple-choice and constructed-response items in two reversible algebraic techniques: factor/expand and solve/verify. The findings supported this hypothesis: Overall scores were higher in the multiple-choice condition compared to the constructed-response condition, but this advantage was significantly greater for items concerning the inverse direction of reversible processes compared to those involving direct processes.

Highlights

Summative assessment of students is a key part of education
The improved performance for multiple choice (MC) items was greater for items intended to test competence with inverse processes compared to items intended to test direct processes
Our research found evidence for an item format (MC/constructed response (CR)) and process direction interaction for reversible mathematical processes

Summary

Introduction

Assessments typically attempt to measure one or both of procedural knowledge and conceptual understanding (Rittle-Johnson & Siegler, 1999). Our focus here is on procedural knowledge, which has been defined as “the ability to execute action sequences to solve problems” (Rittle-Johnson, Siegler, & Alibali, 2001). High-stakes examinations around the world have been criticised for privileging procedural over conceptual items (e.g., Berube, 2004; Iannone & Simpson, 2012; Noyes, Wake, Drake, & Murphy, 2011). Jones this emphasis on procedural items is that they are relatively easy to produce and can be scored objectively (Swan & Burkhardt, 2012). As such, scoring reliabilities tend to be very high in mathematics compared to other subjects (Brooks, 2004)

Methods

Findings

Discussion

Conclusion