Quantifying Scientific Thinking Using Multichannel Data With Crystal Island: Implications for Individualized Game-Learning Analytics

Abstract

Individualizing learning by quantifying scientific thinking using multichannel data during game-based learning remains a significant challenge for researchers and educators. Not only do empirical studies find that learners do not possess sufficient scientific-thinking skills to deal with the demands of the 21st century, but there is little agreement in how researchers should accurately and dynamically capture scientific thinking with game-based learning environments (GBLEs). Traditionally, in-game actions, collected through log files, are used to define if, when, and for how long learners think scientifically about solving complex problems with GBLEs. But can in-game actions distinguish between learners who are thinking scientifically while solving problems versus those who are not? We argue that collecting multiple channels of data identifies if, when, and for how long learners think scientifically during game-based learning compared to only in-game actions. In this study, we examined relationships between 68 undergraduates’ pre-test scores (i.e., prior knowledge), degree of agency, eye movements, in-game actions related to scientific-thinking actions during game-based learning about microbiology with Crystal Island, and performance outcomes. Results showed significant predictive relationships between eye-gaze, pre-test scores, and in-game actions related to scientific thinking, suggesting that eye movements, pre-test scores, and degree of agency play a crucial role in scientific thinking and performance with GBLEs. Our findings provide insight into using multichannel data to capture scientific thinking and inform game-learning analytics that is individualized to guide instructional decision making. Findings from this study have implications for enhancing our understanding of scientific thinking within GBLEs. We discuss GBLEs designed to guide individualized and adaptive instructional decision making using learners’ multichannel data to optimize scientific-thinking skills and performance.