Abstract
Audience Response System (ARS), like “clicker,” has proven their effectiveness in students’ engagement and in enhancing their learning. Apart from close-ended questions, ARS can help instructors to pose open-ended questions. Such questions are not scored automatically for that Automated Text Scoring; ATS is vastly used. This paper presents the findings of the development of an intelligent Automated Text Scoring, iATS, which provides instantaneous scoring of students’ responses to STEM-related factual questions. iATS is integrated with an Audience Response System (ARS), known as iRes, which captures students’ responses in traditional classrooms environment using smartphones. iATS Research is conducted to code and test three Natural Language Processing (NLP), text similarity methods. The codes were developed in PHP and Python environments. Experiments were performed to test Cosine similarity, Jaccard Index and Corpus-based and knowledge-based measures, (CKM), scores against instructor’s manual grades. The research suggested that the cosine similarity and Jaccard index are underestimating with an error of 22% and 26%, respectively. CKM has a low error (18%), but it is overestimating the score. It is concluded that codes need to be modified with a corpus developed within the knowledge domain and a new regression model should be created to improve the accuracy of automatic scoring.
Highlights
Creating active learning and improve participation and engagement in a sizeable live classroom is challenging
This paper describes phase one, which includes the development of the automated scoring system using Natural Language Processing (NLP), features, and calculating similarity scores using three different NLP similarity methods
Testing was performed in a freshman introductory mechanics class using a simple physics question; “What is Newton's first law of motion?” The question was posed using iRes, and student's responses were captured and were automatically graded by the iATS
Summary
Creating active learning and improve participation and engagement in a sizeable live classroom is challenging. Audience Response System (ARS), like clickers, has provided a tool that has proved to be very useful in such an environment, [3] It is very well documented how clicker has helped in creating an active learning environment and improving students’ participation and engagement, [4, 5, 6]. Such a meeting includes students responding to a question in live class and been evaluated by the system or instructor immediately. ARS uses websites and Apps to communicate with instructors These online ARS grade students’ responses to close-end questions only, and illustrate results as bar charts to create discussion in the class.
Sign-in/Register to view highlights & summary 
Published Version (
Free)
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call