Minute-Paper Dashboard: Identification of Learner’s Misconceptions Using Topic Modeling on Formative Reflections

Abstract

This work in progress proposes a method for automated analysis of students’ short reflections using NLP to get insights into their challenges and learning outcomes in the course. The importance of self-reflection in engineering education has been emphasized more recently as it improves the students’ learning experience and helps instructors to remove students’ learning gaps. Towards this goal, educational scholars have developed different types of reflection tools as well as analysis methods to get feedback on students’ learning outcomes. Reviewing narratives of the reflections is time taking specially in large class settings. One of the known approaches to get feedback from students are mini-reflections called ‘minute paper’ that asks students to answer two questions briefly about what they learned and what they didn’t learn in the class. Although these short surveys reduce the narrative load and help in the quicker review of the reflections still they require instructors to review them one by one. In this work, we apply clustering methods to the students’ reflection responses in a software engineering course to extract their challenge areas as well as learning outcomes from each session of the class. The result of the analysis is visualized in a dashboard that dynamically shows an overview of the challenging topics and learning outcomes based on their weight and frequency of their occurrence in students’ responses. This means the more students mentioned a certain topic as their challenge or learning outcomes, the topic acquires higher weight as a result. The application of this dashboard helps educators to get quick and real-time insight into students’ misconceptions in a formative style. It enables them to narrow students’ learning gap by discussing the challenging topics in the upcoming class sessions. We found this method to be very helpful in both improving students’ learning experience as well as creating an open channel for students to communicate their misunderstandings with confidence and feel being heard and supported.