Identifying Problem-Solving Solution Patterns Using Network Analysis of Operation Sequences and Response Times

Abstract

ABSTRACT Process data from educational assessments enhance the understanding of how students answer cognitive items. However, effectively making use of these data is challenging. We propose an approach to identify solution patterns from operation sequences and response times by generating networks from process data and defining network features that extract essential information from them. With these features, we group respondents to a problem-solving task from PISA 2012 using Gaussian mixture models. The results indicate the presence of two and four clusters for groups defined by failure and success on the task, respectively. We interpret the clusters as less-able, low-effort, adaptable, back-and-forth, deliberate, and trial-and-error clusters by considering the cluster-specific feature statistics. The proposed approach sheds light on students’ problem-solving mental processes, which can aid item development and facilitate individualized feedback to students. The method is applicable to many computer-based problems, but a limitation is that the feature definitions can be task-dependent.