Modeling Complex Data from Simulations to Assess Middle School Students’ NGSS-Aligned Science Learning

Abstract

ABSTRACT The Next Generation Science Standards (NGSS) delineate three interrelated dimensions that describe what students should know and how they should engage in science learning. These present significant challenges for assessment because traditional assessments may not be able to capture the ways in which students engage with content. Science simulations engage students with science phenomena while collecting data on student simulation use. However, there are limited guidelines on how to analyze this type of complex data for assessment. This study aims to investigate how two statistical approaches (Bayesian Networks and Item Response Theory) model student learning in a science simulation. The simulation data included process data, embedded table responses, and open-response reflection answers. The results did not suggest a best model for this data, but highlighted uses and challenges of the different models and illustrate a method for utilizing complex data to help understand both what and how students are learning.