A cognitive diagnosis framework based on peer assessment

Abstract

Given examinees' performance (i. e., scores) on each problem, cognitive diagnosis models can discover the latent characteristics of examinees. Traditional cognitive diagnosis models require teachers to provide scores in time. Thus we can hardly apply traditional models in large-scale scenarios, such as Massive Open Online Courses (MOOC). Peer assessment refers to a teaching activity in which students evaluate each other's assignments. The scores given by students could replace the teacher's assessments to a certain extent. In this paper, we propose a novel cognitive diagnosis model named Peer-Assessment Cognitive Diagnosis Framework (PACDF). This model combines peer assessments with cognitive diagnosis, aiming at reduce the burden of teachers. Specifically, we propose a novel probabilistic graphic model at first. This model characterizes not only the relationships between real scores and scores given by peer assessment, but also the relationship between examinees' skill proficiency and problem mastery. Then we adopt Monte Carol Markov Chain (MCMC) sampling algorithm to estimate the parameters of the model. Lastly, we use the model to predict examinees' performance. The experimental results show that PACDF could quantitatively explain and analyze skill proficiencies of examinees, thus perform better in predicting examinees' performances.