Applying Log Data Analytics to Measure Problem Solving in Simulation-Based Learning Environments

Abstract

Interactive tasks embedded in open-ended digital learning environments offer a promising approach to measuring students’ higher-order competencies efficiently and at scale. More research is needed at the intersection of learning analytics and educational measurement to make these interactive tasks useful assessments in classrooms. This chapter represents our research efforts toward understanding how the log data of students’ interactions within an educational simulation can be translated into meaningful evidence about their problem-solving process. Our analyses reveal that features extracted from log data are both significant predictors of students’ problem-solving outcomes and indicators of specific problem-solving practices. Specifically, instances of deliberate pause during the problem-solving process could be an important and generalizable feature associated with students’ problem-solving competencies across different tasks. The results highlight the utility of log data generated in interactive learning environments to provide unobtrusive observations of students’ problem-solving processes and the power of learning analytics techniques to extract semantically meaningful behavior patterns associated with specific problem-solving practices.