Evaluation of Students’ Learning Through Reflection on Doing Based on Sentiment Analysis

Abstract

Abstract In this paper, we address the following question: How can instructors leverage assessment instruments used to process information gleaned from design, build, and test courses to simultaneously improve student outcomes and assess student learning well enough to improve the course in the future? A take-away is unstructured text written by students in AME4163: Principles of engineering design to record what they understand by reflecting on authentic and immersive experiences that occur throughout the semester. The immersive experiences include lectures, assignments, reviews, building, testing, and post-analysis for the design of an electro-mechanical system to address a given customer need. In the context of a take-away, a student then writes a learning statement that is a structured sentence written as a triple, i.e., (Experience—Learning—Value). Between 2019 and 2021, we collected about 10,000 take-aways and learning statements from almost 400 students. In this paper, we address the question from the perspective of students’ feelings and use dictionary-based sentiment analysis to evaluate students’ subjective feelings toward what they are learning. Through quantitative results, we get an overview of students’ sentiments, analyze the underlying reasons, and provide evidence-based guidance to instructors on how to improve the delivery of the course in the future. Our focus in this paper is on explaining the method using data from AME4163, which is general and can be extended to other courses.