A Hybrid Automatic Scoring System: Artificial Intelligence- Based Evaluation of Physics Concept Comprehension Essay Test

Abstract

Conceptual understanding is one of the keys in physics. The purpose of this research was to develop an artificial intelligence-based system used to assist instructors in evaluating students’ conceptual understanding essay test results. This research used a method with Analysis, Design, Development, Implementation, and Evaluation (ADDIE) model (). The research subjects were lecturers and undergraduate students of physics education at the University of Jambi. Data collection was done through interviews and tests. The instruments used in this research were interview sheets and essay questions. The results of the feasibility test analysis showed that the hybrid automatic scoring system was overall suitable with an average score of 4.40 (very suitable category) in the aspects of software engineering and visual communication. In addition, the analysis of students’ conceptual understanding using the Hybrid Automatic Scoring System showed that only 21.9% of students had good conceptual understanding. The remaining 40.6% of students fell into the partial understanding category, 9.4% fell into the specific misconception category, and 28.2% had poor conceptual understanding. These results are not very different from the results of the manual analysis. The manual analysis showed that 21.9% of the students had good concept understanding, 40.6% had partial understanding, 12.5% had specific misconceptions, and 25% had poor concept understanding. These results indicate that the Automated Hybrid Scoring System can be used as an assessment tool for essay tests of conceptual understanding. The results of this research contribute in the form of a technology that can assist physics instructors in correcting, scoring, and providing feedback on physics test scores based on essay questions.