Analysis of Conditions for Reliable Predictions by Moodle Machine Learning Models

Gábor Zsolt Nagy,Tibor Fauszt,László Bognár

doi:10.3991/ijet.v16i06.18347

Abstract

In this paper the issue of bias-variance trade-off in building and operating Moodle Machine Learning (ML) models are discussed to avoid traps of get-ting unreliable predictions. Moodle is one of the world’s most popular open source Learning Management System (LMS) with millions of users. Although since Moodle 3.4 release it is possible to create ML models within the LMS system very few studies have been published so far about the conditions of its proper application. Using these models as black boxes hold serious risks to get unreliable predictions and false alarms. From a comprehensive study of differently built machine learning models elaborated at the University of Dunaújváros in Hungary, one specific issue is addressed here, namely the in-fluence of the size and the row-column ratio of the predictor matrix on the goodness of the predictions. In the so-called Time Splitting Method in Moo-dle Learning Analytics the effect of varying numbers of time splits and of predictors has also been studied to see their influence on the bias and the variance of the models. An Applied Statistics course is used to demonstrate the consequences of the different model set up.

Highlights

According to the classical definition Learning Analytics (LA) is "the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs" [22]
Machine Learning (ML) technics have been widely used to predict students’ success at fulfilling the course requirements and to help those students who seem to be at risk of dropping out
There are multiple machine learning models used in education [3], [8], [17], [20]

Summary

Introduction

According to the classical definition Learning Analytics (LA) is "the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs" [22]. The Learning Management Systems (LMS) have been continuously revised and different tools are developed to identify students at risk of dropping out [2], [15]. Tan end Shao developed and compared three prediction models based on different ML algorithms [21] They found all algorithm to be appropriate for student dropout prediction but they found Decision Tree algorithm presented a better performance. Metrics were defined to infer students’ social integration from smart card transactions These new features were effective in significantly improving precision and recall rates in identifying drop-out students. Even in case of a well-developed LMS system the continual supervision of the applicability of the machine learning models should be inevitable Using these models as black boxes hold serious risks to get unreliable predictions and false alarms (false no alarms). The conditions for reliable predictions by Moodle Machine Learning models are analyzed

Machine Learning Models in Education

The Courses and the Predictors of the Present Study

Performance Metrics

Findings

Conclusion