A systematic literature review on knowledge tracing in learning programming

Abstract

This Research Full Paper presents a systematic review on knowledge tracing of learning programming based on student performance data in exercises. Programming has become an essential skill to solve realworld problems in modern engineering disciplines. However, when students start to learn how to program, they face a lot of challenges in acquiring various programming knowledge and skills. While it is beneficial to customise learning material to fit individual learning progress, the widely different learning pace of students in an introductory programming course has made it impractical for teachers to track the knowledge acquisition of individual. Hence, many recent works take a data-driven approach to model students' learning progress based on the performance data in programming-related exercises, which include the submitted program codes and answers to closed-ended programming exercises. By analyzing these performance data, a system can evaluate the students' knowledge level of various concepts and skills in programming. This paper performs a systematic review and reports key information about recent works on programming knowledge tracing based on student performance data. An overview of the different choices of knowledge representation, domain knowledge model, performance measure and knowledge tracing algorithms is provided. The nature and granularity of knowledge components and the relationships between them are compared across the reviewed works. The different choice of programming knowledge representation leads to varied methods to assess knowledge levels from empirical performance data in programming-related exercises. Two broad categories of works are identified. The first is to overlay a student model on the domain knowledge model, and the student knowledge levels are updated in distinct time steps. The second trains temporal knowledge tracing models to predict students' future performance based on their performance in previous exercises. In addition, this review discusses the distinct challenges in knowledge tracing in programming education. It also points out limitations in current works and opportunities to improve knowledge tracing in learning programming.