Using Visualization to Understand the Problem-Solving Processes of Elementary Students in a Computer-Assisted Math Learning Program

Abstract

CAL (Computer Assisted Learning) programs are widespread today in schools and families due to the effectiveness of CAL programs in improving students’ learning and task performance. The flourishing of CAL programs in education has brought large amounts of students’ learning data including log data, performance data, mouse movement data, eye movement data, video data, etc. These data can present students’ learning or problem-solving processes and reflect underlying cognitive processes. These data are valuable resources for educators to comprehend students’ learning and difficulties. However, few data analysis methods can analyze and present CAL data for educators quickly and clearly. Traditional video analysis methods can be time-consuming. Current visualization analysis methods are limited to simple charts or visualizations of a single data type. In this dissertation, I propose a visual learning analytic approach to analyze and present students' problem-solving data from CAL programs. More specifically, a visualization system was developed to present students’ problem-solving data, including eye movement, mouse movement, and performance data, to help educational researchers understand student problem-solving processes and identify students’ problem-solving strategies and difficulties. An evaluation experiment was conducted to compare the visualization system with traditional video analysis methods. Seven educational researchers were recruited to diagnose students’ problem-solving patterns, strategies, and difficulties using either the visualization system or video. The diagnosis task loads and evaluators’ diagnosis processes were measured and the evaluators were interviewed. The results showed that analyzing student problem-solving tasks using the proposed visualization method was significantly quicker than using the video method. In addition, diagnosis using the visualization system can achieve results at least as reliable as the video analysis method. Evaluators’ preferences between the two methods are summarized and illustrated in the dissertation. Finally, the implications of the visual analytic approach in education and data visualization areas are discussed.