Beyond Accuracy: A Process for Analysis of Constructed Responses in Large Datasets and Insights into Students’ Equation Solving

Abstract

This article reports a detailed analysis of errors made by early algebra students who are learning equation solving. Whilst we aim to deepen the knowledge base that can be used by teachers and curriculum writers to plan instruction, the main goal of this article is to demonstrate a 5-step process that we have developed to analyse, summarize, visualize and grasp meaning from large numbers of short constructed responses to mathematics questions. This data-driven process is illustrated using responses from 3,010 students to an online test, which is intended to be highly diagnostic and used for formative purposes. The test consisted of 14 very carefully selected linear equations to solve. Students entered only the numerical answers. Although nearly all the errors identified have been previously reported in the literature, the study provides insights into their prevalence in this target population and demonstrates how they correspond to other characteristics of the students’ algebra, including the strategies that they use to solve these linear equations. It also provides a practical example of using short constructed responses as a window into students’ thinking.