Generative Model of Knowledge Gaps Discovery from Multiple Choice Tests

Abstract

This submission represents a proposition of the Full Paper in the Research-to-Practice category in which we present our innovative work on automated student knowledge mapping and gaps discovery.Students’ ability to learn and retain new knowledge depends on the precise state of their current knowledge distributed along a complex network of related concepts. Uneven distribution of competencies in the required knowledge concepts, or even worse - knowledge gaps - in certain critical concepts, may significantly hinder students’ ability to acquire knowledge efficiently and prevent them from maximizing the potential of their talents. Discovery of and bridging these gaps at the individual student and group level is critically important for students to maximize their learning efficiency, for the educators to maximize the impact of the teaching, and for the governments to maximize the effectiveness of their educational programs and policies.We propose a novel methodology for automated student knowledge mapping and gaps discovery from the results of multiple choice test questions extracted from a single .pdf report through pattern recognition techniques. Automatically lifted test performance data are then decomposed into student competencies in the related concepts required to correctly answer the test questions. Simple constrained linear optimization is deployed to find the optimal student competencies in the related concepts such that their linear combination best reconstructs the achieved test scores. Extracted knowledge maps in the related concepts allowed to discover common concept gaps for individual students and their groups, which has been successfully illustrated in the context of mathematics preparatory course at the university level with hundreds of students and thousands of test questions.