An Educational System for Learning Search Algorithms and Automatically Assessing Student Performance

Abstract

In this paper, first we present an educational system that assists students in learning and tutors in teaching search algorithms, an artificial intelligence topic. Learning is achieved through a wide range of learning activities. Algorithm visualizations demonstrate the operational functionality of algorithms according to the principles of active learning. So, a visualization process can stop and request from a student to specify the next step or explain the way that a decision was made by the algorithm. Similarly, interactive exercises assist students in learning to apply algorithms in a step-by-step interactive way. Students can apply an algorithm to an example case, specifying the algorithm’s steps interactively, with the system’s guidance and help, when necessary. Next, we present assessment approaches integrated in the system that aim to assist tutors in assessing the performance of students, reduce their marking task workload and provide immediate and meaningful feedback to students. Automatic assessment is achieved in four stages, which constitute a general assessment framework. First, the system calculates the similarity between the student’s and the correct answer using the edit distance metric. In the next stage, it identifies the type of the answer, based on an introduced answer categorization scheme related to completeness and accuracy of an answer, taking into account student carelessness too. Afterwards, the types of errors are identified, based on an introduced error categorization scheme. Finally, answer is automatically marked via an automated marker, based on its type, the edit distance and the type of errors made. To assess the learning effectiveness of the system an extended evaluation study was conducted in real class conditions. The experiment showed very encouraging results. Furthermore, to evaluate the performance of the assessment system, we compared the assessment mechanism against expert (human) tutors. A total of 400 students’ answers were assessed by three tutors and the results showed a very good agreement between the automatic assessment system and the tutors.