Using Sankey diagrams to visualize drag and drop action sequences in technology-enhanced items.

Abstract

We propose a Sankey diagram-based visualization method to illustrate test-takers' action sequences in technology-enhanced, drag-and-drop items. This method extends existing studies focusing only on initial or last attempt(s) of answer formulation, and extracts, at the item level, comprehensive information of test-takers' drag-and-drop actions. Using the process data from three mathematical items in the National Assessment of Educational Progress, we exemplify how to develop Sankey diagrams, discover frequent actions at each stage of problem solving, deduce test-takers' response strategies, and discuss the relationship among domain (i.e., mathematics) knowledge, response strategy, and item performance. The applications and future extensions of this method can contribute to process-based problem-solving research in educational assessments.