Using Bayesian networks to improve knowledge assessment

Abstract

In this paper, we describe the integration and evaluation of an existing generic Bayesian student model (GBSM) into an existing computerized testing system within the Mathematics Education Project (PmatE - Projecto Matematica Ensino) of the University of Aveiro. This generic Bayesian student model had been previously evaluated with simulated students, but a real application was still missing. In the work presented here, we have used the GBSM to define Bayesian Student Models (BSMs) for a concrete domain: first degree equations. In order to test the diagnosis capabilities of such BSMs, an evaluation with 152 students has been performed. Each of the 152 students took both a computerized test within PMatE and a written exam, both of them designed to measure students' knowledge in 12 concepts related to first degree equations. The written exam was graded by three experts. Then two BSMs were developed, one for the computer test and another one for the written exam. These BSMs were used to obtain estimations of student's knowledge on the same 12 concepts, and the inter-rater agreement among the different measures was computed. Results show a high degree of agreement among the scores given by the experts and also among the diagnosis provided by the BSM in the written exam and expert's average, but a low degree of agreement among the diagnosis provided by the BSM in the computer test and expert's average.