Knowledge Tracing Model Research Articles

Tracing Knowledge States through Student Assessment in a Blended Learning Environment

Blended learning has recently acquired popularity in a variety of educational settings. This approach has the advantage of being able to autonomously monitor students' knowledge states using the collected learning data. Moodle is the most widely used learning management system in blended learning environments. Students can access Moodle to obtain supplementary materials, exercises, and assessments to complement their face-to-face meetings. However, its performance can be improved by more effectively tailoring students' skills and pace of learning. Several studies have been conducted on knowledge tracing; however, we have not discovered any studies that particularly investigate knowledge tracing in a blended learning setting with Moodle as a component. This study proposes a scheme for assessment using the features of the Moodle quiz platform. The assessment data is used to conduct knowledge tracing with the Bayesian Knowledge Tracing (BKT) model, which improves interpretability. The aforementioned data were collected from information engineering undergraduate students who completed 88 exercises that assessed 23 knowledge components within the course. We measure RMSE and MAE to evaluate the performance of the BKT model on our dataset. Furthermore, we compare the knowledge tracing performance to other well-known datasets. Our results show that the BKT model performed better with our dataset, with an RMSE of 0.314 and an MAE of 0.197. Moreover, the BKT model can be used to assess student performance and determine the level of mastery for each knowledge component. Thus, the outcomes can be applied to personalized learning in the future.

Investigation of recent advances in exercise-enhanced knowledge tracing models

Exercising plays a significant role in Knowledge Tracing (KT), however, the majority of KT models struggle to effectively extract the abundant information embedded within students' exercise histories, leading to a prevalent issue of information erosion. Hence, it is vital for exploring methods to mine the data from the exercises to better forecast students' future performance. This paper is a review that aims to discuss recent improvements in KT models on mining information existing in exercises. This paper will briefly discuss DKT, a popular approach to knowledge tracing, and its limitation on mining information from exercises. And then, this review will introduce four exercise-enhanced knowledge tracing models including EERNN, EKT, HGKT and Concept-Aware Deep Knowledge Tracing. EERNN and EKT can track students knowledge acquisition by using vectors to represent knowledge concepts. HGKT goes a step further by examining the interconnectedness among different exercises based on EERNN and EKT. Concept-Aware DKT is an improvement of DKVMN, the other approach of KT, by considering the influence of knowledge concepts and corresponding exercises in more detail. Finally, the applications of exercise-enhanced KT models will be covered.

Evaluating the Effectiveness of Bayesian Knowledge Tracing Model-Based Explainable Recommender

Explainable recommendation, which provides an explanation about why a quiz is recommended, helps to improve transparency, persuasiveness, and trustworthiness. However, little research examined the effectiveness of the explainable recommender, especially on academic performance. To survey its effectiveness, the authors evaluate the math academic performance among middle school students (n=115) by giving pre- and post-test questions based evaluation techniques. During the pre- and post-test periods, students were encouraged to use the Bayesian Knowledge Tracing model based explainable recommendation system. To evaluate how well the students were able to do what they could not do, the authors defined growth rate and found recommended quiz clicked counts had a positive effect on the total number of solved quizzes (R=0.343, P=0.005) and growth rate (R=0.297, P=0.017) despite no correlation between the total number of solved quizzes and growth rate. The results suggest that the use of an explainable recommendation system that learns efficiently will enable students to do what they could not do before.

DGEKT: A Dual Graph Ensemble Learning Method for Knowledge Tracing

Knowledge tracing aims to trace students’ evolving knowledge states by predicting their future performance on concept-related exercises. Recently, some graph-based models have been developed to incorporate the relationships between exercises to improve knowledge tracing, but only a single type of relationship information is generally explored. In this article, we present a novel Dual Graph Ensemble learning method for Knowledge Tracing (DGEKT), which establishes a dual graph structure of students’ learning interactions to capture the heterogeneous exercise–concept associations and interaction transitions by hypergraph modeling and directed graph modeling, respectively. To combine the dual graph models, we introduce the technique of online knowledge distillation. This choice arises from the observation that, while the knowledge tracing model is designed to predict students’ responses to the exercises related to different concepts, it is optimized merely with respect to the prediction accuracy on a single exercise at each step. With online knowledge distillation, the dual graph models are adaptively combined to form a stronger ensemble teacher model, which provides its predictions on all exercises as extra supervision for better modeling ability. In the experiments, we compare DGEKT against eight knowledge tracing baselines on three benchmark datasets, and the results demonstrate that DGEKT achieves state-of-the-art performance.

Improving the performance and explainability of knowledge tracing via Markov blanket

Knowledge tracing predicts student knowledge acquisition states during learning. Traditional knowledge tracing methods suffer from poor prediction performance; however, recent studies have significantly improved prediction performance through the incorporation of deep neural networks. However, prediction results generated from deep knowledge tracing methods are typically difficult to explain. To solve this issue, a knowledge tracing model using Markov blankets was proposed to improve the interpretability of knowledge tracing. The proposed method uses the Markov blanket of the target variable as a subset of features and applies interpretable machine learning techniques to knowledge tracing. The results from the ablation experiments demonstrate that the feature subspace created by the Markov blanket is substantially effective for prediction. The proposed model also performs better than several other knowledge tracing models on two widely used datasets, i.e., Junyi and ASSISTments. Furthermore, the use of Markov blanket-based features provides high interpretability for predicting knowledge mastery states, elucidating the impact of these features on student knowledge acquisition. Moreover, this enables the use of previously considered low-correlation features, which may possess important latent causal relationships.

Dual-Channel Adaptive Scale Hypergraph Encoders With Cross-View Contrastive Learning for Knowledge Tracing.

Knowledge tracing (KT) refers to predicting learners' performance in the future according to their historical responses, which has become an essential task in intelligent tutoring systems. Most deep learning-based methods usually model the learners' knowledge states via recurrent neural networks (RNNs) or attention mechanisms. Recently emerging graph neural networks (GNNs) assist the KT model to capture the relationships such as question-skill and question-learner. However, non-pairwise and complex higher-order information among responses is ignored. In addition, a single-channel encoded hidden vector struggles to represent multigranularity knowledge states. To tackle the above problems, we propose a novel KT model named dual-channel adaptive scale hypergraph encoders with cross-view contrastive learning (HyperKT). Specifically, we design an adaptive scale hyperedge distillation component for generating knowledge-aware hyperedges and pattern-aware hyperedges that reflect non-pairwise higher-order features among responses. Then, we propose dual-channel hypergraph encoders to capture multigranularity knowledge states from global and local state hypergraphs. The encoders consist of a simplified hypergraph convolution network and a collaborative hypergraph convolution network. To enhance the supervisory signal in the state hypergraphs, we introduce the cross-view contrastive learning mechanism, which performs among state hypergraph views and their transformed line graph views. Extensive experiments on three real-world datasets demonstrate the superior performance of our HyperKT over the state-of-the-art (SOTA).

Design and Evaluation of Trustworthy Knowledge Tracing Model for Intelligent Tutoring System

Design and Evaluation of Trustworthy Knowledge Tracing Model for Intelligent Tutoring System

Knowledge relation rank enhanced heterogeneous learning interaction modeling for neural graph forgetting knowledge tracing.

Knowledge tracing models have gained prominence in educational data mining, with applications like the Self-Attention Knowledge Tracing model, which captures the exercise-knowledge relationship. However, conventional knowledge tracing models focus solely on static question-knowledge and knowledge-knowledge relationships, treating them with equal significance. This simplistic approach often succumbs to subjective labeling bias and lacks the depth to capture nuanced exercise-knowledge connections. In this study, we propose a novel knowledge tracing model called Knowledge Relation Rank Enhanced Heterogeneous Learning Interaction Modeling for Neural Graph Forgetting Knowledge Tracing. Our model mitigates the impact of subjective labeling by fine-tuning the skill relation matrix and Q-matrix. Additionally, we employ Graph Convolutional Networks (GCNs) to capture intricate interactions between students, exercises, and skills. Specifically, the Knowledge Relation Importance Rank Calibration method is employed to generate the skill relation matrix and Q-matrix. These calibrated matrices, alongside heterogeneous interactions, serve as input for the GCN to compute exercise and skill embeddings. Subsequently, exercise embeddings, skill embeddings, item difficulty, and contingency tables collectively contribute to an exercise relation matrix, which is then fed into an attention mechanism for predictions. Experimental evaluations on two publicly available educational datasets demonstrate the superiority of our proposed model over baseline models, evidenced by enhanced performance across three evaluation metrics.

HiTSKT: A hierarchical transformer model for session-aware knowledge tracing

Knowledge tracing (KT) aims to leverage students’ learning histories to estimate their mastery levels on a set of pre-defined skills, based on which the corresponding future performance can be accurately predicted. In practice, a student’s learning history comprises answers to sets of massed questions, each known as a session, rather than merely being a sequence of independent answers. Theoretically, within and across these sessions, students’ learning dynamics can be very different. Therefore, how to effectively model the dynamics of students’ knowledge states within and across the sessions is crucial for handling the KT problem. Most existing KT models treat student’s learning records as a single continuing sequence, without capturing the sessional shift of students’ knowledge state. To address the above issue, we propose a novel hierarchical transformer model, named HiTSKT, comprises an interaction(-level) encoder to capture the knowledge a student acquires within a session, and a session(-level) encoder to summarize acquired knowledge across the past sessions. To predict an interaction in the current session, a knowledge retriever integrates the summarized past-session knowledge with the previous interactions’ information into proper knowledge representations. These representations are then used to compute the student’s current knowledge state. Additionally, to model the student’s long-term forgetting behaviour across the sessions, a power-law-decay attention mechanism is designed and deployed in the session encoder, allowing it to emphasize more on the recent sessions. Extensive experiments on four public datasets demonstrate that HiTSKT achieves new state-of-the-art performance on all the datasets compared with seven state-of-the-art KT models.

Towards more accurate and interpretable model: Fusing multiple knowledge relations into deep knowledge tracing

With the rapid growth of online education, Knowledge tracing (KT) has become a well established problem, which evaluates the knowledge states of students and predicts their performance on new exercises. Recently, more and more works have noticed the importance of relations among knowledge points and proposed to introduce the knowledge relations into KT. However, how to precisely learn the representation of different types of knowledge relations and effectively fuse multiple relations into KT is still challenging. To address this issue, we propose a novel KT model, called Deep Knowledge Tracing with Multiple Relations (DKTMR), which can simultaneously fuse the directed relation and undirected relation into KT. More specifically, casting the knowledge relations as a graph, DKTMR designs to utilize two types of Generative Adversarial Networks (GANs) to learn the representation of knowledge point with different relations via graph representation learning. Then, the Gated Recurrent Unit (GRU) is used to update the students’ knowledge states. Furthermore, to consider the different contribution for each type of relation to the final prediction, an attention-based fusion method is proposed to learn the coefficients for different relations. Compared with several state-of-the-art baselines, the extensive experiments on four real-world datasets demonstrate the effectiveness and interpretability of DKTMR.

Heterogeneous graph-based knowledge tracing with spatiotemporal evolution

Knowledge tracing (KT), in which the future performance of students is estimated by tracing their knowledge states based on their responses to exercises, is widely applied in the field of intelligent education. However, existing mainstream KT models explore the importance of knowledge relations but ignore the key role of cognitive factors. According to the knowledge construction theory, the human cognitive system performs both spatial accommodation and temporal assimilation to internalize knowledge. In this paper, we propose an innovative heterogeneous graph-based Knowledge tracing method with spatiotemporal evolution (TSKT), in which knowledge state evolution is traced along both temporal and spatial dimensions. We construct a heterogeneous graph with multiple exercise attributes, including content, concepts, and difficulty, to obtain a knowledge space with richer exercise representations through hierarchical aggregation. We design a spatial updating module in which each interaction updates the current node’s state of the knowledge space and transfers its influence to its neighbors. We also design a temporal updating module to further update the knowledge state through short-term memory enhancing and long-term memory forgetting. Finally, we stack these modules to obtain deeper features by using alternate spatiotemporal updating. Extensive experiments on three datasets reveal the superiority of the proposed method and its variants in terms of future performance prediction.

Sim-GAIL: A generative adversarial imitation learning approach of student modelling for intelligent tutoring systems

The continuous application of artificial intelligence (AI) technologies in online education has led to significant progress, especially in the field of Intelligent Tutoring Systems (ITS), online courses and learning management systems (LMS). An important research direction of the field is to provide students with customised learning trajectories via student modelling. Previous studies have shown that customisation of learning trajectories could effectively improve students’ learning experiences and outcomes. However, training an ITS that can customise students’ learning trajectories suffers from cold-start, time-consumption, human labour-intensity, and cost problems. One feasible approach is to simulate real students’ behaviour trajectories through algorithms, to generate data that could be used to train the ITS. Nonetheless, implementing high-accuracy student modelling methods that effectively address these issues remains an ongoing challenge. Traditional simulation methods, in particular, encounter difficulties in ensuring the quality and diversity of the generated data, thereby limiting their capacity to provide intelligent tutoring systems (ITS) with high-fidelity and diverse training data. We thus propose Sim-GAIL, a novel student modelling method based on generative adversarial imitation learning (GAIL). To the best of our knowledge, it is the first method using GAIL to address the challenge of lacking training data, resulting from the issues mentioned above. We analyse and compare the performance of Sim-GAIL with two traditional Reinforcement Learning-based and Imitation Learning-based methods using action distribution evaluation, cumulative reward evaluation, and offline-policy evaluation. The experiments demonstrate that our method outperforms traditional ones on most metrics. Moreover, we apply our method to a domain plagued by the cold-start problem, knowledge tracing (KT), and the results show that our novel method could effectively improve the KT model’s prediction accuracy in a cold-start scenario.

SFBKT: A Synthetically Forgetting Behavior Method for Knowledge Tracing

Knowledge tracing (KT) aims to model students’ knowledge levels based on their historical learning records and predict their future learning performance, which constitutes an essential component of intelligent education. Learning and forgetting are closely related, and forgetting can often interfere with the learning process. Prior research has employed diverse techniques to address the issue of interference caused by forgetting factors in predictions, yet many of these methods fail to fully leverage the forgetting information contained within learning records. This paper proposes a synthetically forgetting behavior knowledge tracing (SFBKT) model that comprehensively models a student’s knowledge level by considering both individual forgetting factors and group status. Specifically, the model initially extracts forgetting information from exercise records in the input module, then updates the student’s knowledge state through an improved continuous-time long short-term memory network (CTLSTM), and finally combines the individual state with the group state using collaborative filtering to predict the student’s ability to correctly answer the next exercise. Our predictive model has been evaluated using four public education datasets. The experimental results indicate that our model’s predictions are effective and outperform other existing methods.

Improving Interpretability of Deep Sequential Knowledge Tracing Models with Question-centric Cognitive Representations

Knowledge tracing (KT) is a crucial technique to predict students’ future performance by observing their historical learning processes. Due to the powerful representation ability of deep neural networks, remarkable progress has been made by using deep learning techniques to solve the KT problem. The majority of existing approaches rely on the homogeneous question assumption that questions have equivalent contributions if they share the same set of knowledge components. Unfortunately, this assumption is inaccurate in real-world educational scenarios. Furthermore, it is very challenging to interpret the prediction results from the existing deep learning based KT models. Therefore, in this paper, we present QIKT, a question-centric interpretable KT model to address the above challenges. The proposed QIKT approach explicitly models students’ knowledge state variations at a fine-grained level with question-sensitive cognitive representations that are jointly learned from a question-centric knowledge acquisition module and a question-centric problem solving module. Meanwhile, the QIKT utilizes an item response theory based prediction layer to generate interpretable prediction results. The proposed QIKT model is evaluated on three public real-world educational datasets. The results demonstrate that our approach is superior on the KT prediction task, and it outperforms a wide range of deep learning based KT models in terms of prediction accuracy with better model interpretability. To encourage reproducible results, we have provided all the datasets and code at https://pykt.org/.

Improving knowledge tracing via a heterogeneous information network enhanced by student interactions

Knowledge tracing (KT) aims to evaluate the knowledge state of students based on their coursework interactions in an intelligent tutoring system (ITS). It is the most fundamental and challenging task in the system. Existing KT models have achieved promising results. However, considerable interactions between the student entity and the other two types of entities (i.e., questions and skills) in ITS have not been well utilized, which restricts the performance of these models. To this end, we propose HIN-KT, a novel heterogeneous information network (HIN)-based pre-processing model, to further enhance the performance of existing KT models. An HIN is first built to model student–question–skill interactions. Subsequently, HIN-KT employs the interaction information modeled by HIN to pre-train embeddings for questions and then adopts pre-trained question embeddings to enhance the performance of recent deep KT models. Experimental results over three public KT datasets demonstrate that large gains in knowledge tracing can be achieved when the proposed HIN-KT model is used to pre-train question embeddings for state-of-the-art deep KT models. In particular, the two state-of-the-art deep KT models, deep knowledge tracing (DKT) and convolutional knowledge tracing (CKT) successfully improved their performance by an average of 7.88% with the help of the HIN-KT model, which is significant progress made in the knowledge tracing domain.

Improving Knowledge Tracing via Considering Two Types of Actual Differences From Exercises and Prior Knowledge

For offering adaptive learning to learners in intelligent tutoring systems, one of the fundamental tasks is knowledge tracing (KT), which aims to assess learners' learning states and make prediction for future performance. However, there are two crucial issues in deep learning-based KT models. First, the knowledge concepts are used to predict learners' performance instead of exercises. This choice ignores the actual difference among exercises, which leads to the KT model's inability to explore the rich information at the exercise level, thus weakening the KT model's performance. Second, the actual difference in prior knowledge is neglected. Therefore, in-depth research of these two types of the actual differences offers the possibility to improve the prediction performance of KT further. To this end, we propose a Fine-Grained Knowledge Tracing model (FGKT) <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">${}^{1}$</tex-math></inline-formula> to capture these two types of actual differences. We first obtain the exercise representations by considering the corresponding knowledge concepts and actual difference among exercises. Then, an effective attention mechanism is designed to capture the relevance between assessment exercises and historical interactions to acquire individual prior knowledge. Finally, we evaluate the proposed FGKT model on several available benchmark datasets. The experiment results show that our model surpasses benchmark KT models by up to 7% in prediction performance on the latest ASSISTments dataset.

Precise modeling of learning process based on multiple behavioral features for knowledge tracing

With the increase in needs for personalized learning of online students, knowledge tracing (KT), a technique aimed at tracing the state of a student’s knowledge mastery and predicting performance in future exercises, has become a hot topic in personalized learning research. The behavioral features exhibited during students’ learning process bear information that impacts the state of a student’s knowledge mastery. To study the influence of learning behaviors on students’ knowledge mastery state in the learning process, we propose a Precise Modeling of Learning Process based on Multiple Behavioral Features for Knowledge Tracing model (MBFKT), which models a student’s learning process by making use of these behavioral features. MBFKT initially processes these features through multi-head attention networks, memory networks, and recurrent neural networks to model students’ learning process into three memory links: memory decline link, memory enhancement link, and memory update link. Various update strategies are designed for each memory link, and the performance of numerous possible combinations of behavioral features in the memory links is compared, for the rules of learning and forgetting to be explained. Furthermore, we also study the contribution and degree of influence of different behavioral features on a student’s knowledge mastery state, by which MBFKT is improved, thus enhancing the accuracy of prediction. Through experiments on real online education datasets and comparison with existing benchmark methods, it is observed that MBFKT has evident advantages in predicting performance with good interpretability.

Knowledge Tracing Model and Student Profile Based on Clustering-Neural-Network

Knowledge tracing models based on deep neural networks are currently widely studied to enhance personalized learning. However, to ensure the practical deployment of DNN-based KT models, prediction accuracy, training efficiency, and interpretability should be greatly improved. In this paper, we observe that the prediction accuracy of KT models can be improved by clustering the features of both students and questions. Based on this observation, a distributed KT scheme is proposed: (1) it classifies both students and questions based on clustering technology to reduce the interaction between different feature data to improve the prediction accuracy; (2) models for different classifications are trained in parallel in this distributed deployment architecture to improve the training efficiency; (3) the combination of a students’ knowledge state matrix and an RPa-LLM model is designed to display the knowledge status of students in the learning process, which can be used to build students’ portraits, thus improving the interpretability of the model. Real educational data are collected to conduct experiments. The results show that the proposed scheme improves both prediction accuracy and training efficiency by 4.08% and 67.28%, respectively, compared to the baseline methods. Furthermore, the proposed method maintains the interpretability of KT models, making it suitable for practical deployment.

Deep knowledge tracing with learning curves.

Knowledge tracing (KT) models students' mastery level of knowledge concepts based on their responses to the questions in the past and predicts the probability that they correctly answer subsequent questions in the future. Recent KT models are mostly developed with deep neural networks and have demonstrated superior performance over traditional approaches. However, they ignore the explicit modeling of the learning curve theory, which generally says that more practices on the same knowledge concept enhance one's mastery level of the concept. Based on this theory, we propose a Convolution-Augmented Knowledge Tracing (CAKT) model and a Capsule-Enhanced CAKT (CECAKT) model to enable learning curve modeling. In particular, when predicting a student's response to the next question associated with a specific knowledge concept, CAKT uses a module built with three-dimensional convolutional neural networks to learn the student's recent experience on that concept, and CECAKT improves CAKT by replacing the global average pooling layer with capsule networks to prevent information loss. Moreover, the two models employ LSTM networks to learn the overall knowledge state, which is fused with the feature learned by the convolutional/capsule module. As such, the two models can learn the student's overall knowledge state as well as the knowledge state of the concept in the next question. Experimental results on four real-life datasets show that CAKT and CECAKT both achieve better performance compared to existing deep KT models.

Learning data teaching strategies via knowledge tracing

Teaching plays a fundamental role in human learning. Typically, a human teaching strategy involves assessing a student’s knowledge progress for tailoring the teaching materials to enhance the learning progress. A human teacher can achieve this by tracing a student’s knowledge over essential learning concepts in a task. Albeit, such a teaching strategy is not well exploited yet in machine learning as current machine teaching methods tend to directly assess the progress of individual training samples without paying attention to the underlying learning concepts in a learning task. In this paper, we propose a novel method, called Knowledge Augmented Data Teaching (KADT), which can optimize a data teaching strategy for a student model by tracing its knowledge progress over multiple learning concepts in a learning task. Specifically, the KADT method incorporates a knowledge tracing model to dynamically capture the knowledge progress of a student model in terms of latent learning concepts. We further develop an attention-pooling mechanism to distill knowledge representations of a student model with respect to class labels, which enables to develop a data teaching strategy on critical training samples. We have evaluated the performance of the KADT method on four different machine learning tasks, including knowledge tracing, sentiment analysis, movie recommendation, and image classification. The KADT method consistently outperforms the state-of-the-art methods on all these tasks.