Evaluating the user’s experience, adaptivity and learning outcomes of a fuzzy-based intelligent tutoring system for computer programming for academic students in Greece

Abstract

Nowadays, the improvement of digital learning with Artificial Intelligence has attracted a lot of research, as it provides solutions for individualized education styles which are independent of place and time. This is particularly the case for computer science, as a tutoring domain, which is rapidly growing and changing and as such, learners need frequent update courses. In this paper, we present a thorough evaluation of a fuzzy-based intelligent tutoring system (ITS), that teaches computer programming. The evaluation concerns multiple aspects of the ITS. The evaluation criteria are: (i) context, (ii) effectiveness, (iii) efficiency, (iv) accuracy, (v) usability and satisfaction, and (vi) engagement and motivation. In the evaluation process students of an undergraduate program in Informatics of the University of Piraeus in Greece participated. The evaluation method that was used included questionnaires, analysis of log files and experiments. Also, t-tests were conducted to certify the validity of the evaluation results. Indeed, the evaluation results are very positive and show that the incorporated fuzzy mechanism to the presented ITS enhances the system with Artificial Intelligence and through this, it increases the learners’ satisfaction and new knowledge learning and mastering, improves the recommendation accuracy of the system, the efficacy of interactions, and contributes positively to the learners’ engagement in the learning process.