Abstract
During this digital era, France, like many other countries, is undergoing a transition from paper-based assessments to digital assessments in education. There is a rising interest in technology-enhanced items which offer innovative ways to assess traditional competencies, as well as addressing problem solving skills, specifically in mathematics. The rich log data captured by these items allows insight into how students approach the problem and their process strategies. Educational data mining is an emerging discipline developing methods suited for exploring the unique and increasingly large-scale data that come from such settings. Data-driven methods can be helpful when trying to make sense of process data. However, studies have shown that didactically meaningful findings are most likely generated when data mining techniques are guided by theoretical principles on subjects’ skills. In this study, theoretical didactical grounding has been essential for developing and describing interactive mathematical tasks as well as defining and identifying strategic behaviors from the log data. Interactive instruments from France’s national large-scale assessment in mathematics have been pilot tested in May 2017. Feature engineering and classical machine learning analysis were then applied to the process data of one specific technology-enhanced item. Supervised learning was implemented to determine the model’s predictive power of students’ achievement and estimate the weight of the variables in the prediction. Unsupervised learning aimed at clustering the samples. The obtained clusters are interpreted by the mean values of the important features. Both the analytical model and the clusters enable us to identify among students two conceptual approaches that can be interpreted in theoretically meaningful ways. If there are limitations to relying on log data analysis in order to determine learning profiles, one of them is the fact that this information remains partial when it comes to describing the complete cognitive activity at play, the potential of technology-enriched problem solving situations in large-scale assessments is nevertheless obvious. The type of findings this study produced is actionable from teachers’ perspective in order to address students’ specific needs.
Highlights
During this digital era, France, like many other countries, is undergoing a transition from paper-based assessments to digital assessments in order to measure student performance in education
The exercises and questions included in the interactive items engage the students on multiple levels, and capture not just their responses, but their thought process as well
We will form a model based on the collected data and some secondary data constructed from it. These secondary elements come from the didactic analysis carried out a priori
Summary
France, like many other countries, is undergoing a transition from paper-based assessments to digital assessments in order to measure student performance in education. There is a rising interest in France in technology-enhanced items which offer innovative ways to assess traditional competencies, as well as to address 21st century skills and to link assessment feedback to learning. They provide scoring information–whether the response is correct or not–but allow rich data to be collected that enable the way that the students have arrived at their answers to be determined (Greiff et al 2015). These complex technology-enhanced items can be used to reflect how students interact in a given situation to analyze and solve a problem. The rich log data captured by these items, such as the time at which students start and stop their work, mouse movements, the use of different onscreen tools, idle time, and a screenshot of the last actions, allow insights to be gained into how students approach the problem, and to identify areas that might require additional focus
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
