Taking advantage of automated assessment of student‐constructed graphs in science

Abstract

We present a new system for automated scoring of graph construction items that address complex science concepts, feature qualitative prompts, and support a range of possible solutions. This system utilizes analysis of spatial features (e.g., slope of a line) to evaluate potential student ideas represented within graphs. Student ideas are then scored with rubrics based upon the knowledge integration framework (Linn & Eylon, 2011). We tested the effectiveness of this system on graphs constructed by 397 8th–12th grade students preceding, during, and following a curriculum focusing on graphs of motion. Comparison with human-coded responses indicates that the automated scoring system is very accurate (κ = 0.9). Also, ideas represented in constructions were generally similar to those demonstrated in written explanations; although individual students often shifted ideas between items. Learning gains were similar in both written and graph construction formats. Overall, these results suggest that graph construction is a valid and efficient means of evaluating students' complex ideas about data representation in science. We discuss the opportunities for incorporating graph construction into new science content areas, such as graphs representing density. We consider the implications of this system for generating automated, adaptive guidance to support instruction. © 2015 Wiley Periodicals, Inc. J Res Sci Teach 52: 1426–1450, 2015.