Bayesian Knowledge Tracing Research Articles

Bayesian Knowledge Tracing Implemented in a Telecommunications Serious Game

The University of Johannesburg has integrated serious games into its teaching, exemplified by Codebreakers, a 2D game teaching information theory. While successful, Codebreakers lacked personalisation and used a criticised assessment method based on answer streaks. Knowledge tracing algorithms, known for their effectiveness in intelligent tutoring systems, were considered to address these limitations. This led to the research question: "Can a new serious game be designed, incorporating knowledge tracing algorithms to deliver personalised learning experiences in telecommunications education?" In response, an escape-themed serious game was developed, integrating Bayesian Knowledge Tracing as a statistical student model for personalised learning. This innovative approach combines free-roam gameplay with tailored educational content, significantly advancing serious game design. While primarily aimed at enhancing Codebreakers, this new game contributes substantially to serious game theory by successfully implementing personalised learning within an engaging format. The project showcases the potential of knowledge tracing algorithms in creating adaptive, student-centered learning experiences within the context of educational games.

Time-dependant Bayesian knowledge tracing-Robots that model user skills over time.

Creating an accurate model of a user's skills is an essential task for Intelligent Tutoring Systems (ITS) and robotic tutoring systems. This allows the system to provide personalized help based on the user's knowledge state. Most user skill modeling systems have focused on simpler tasks such as arithmetic or multiple-choice questions, where the user's model is only updated upon task completion. These tasks have a single correct answer and they generate an unambiguous observation of the user's answer. This is not the case for more complex tasks such as programming or engineering tasks, where the user completing the task creates a succession of noisy user observations as they work on different parts of the task. We create an algorithm called Time-Dependant Bayesian Knowledge Tracing (TD-BKT) that tracks users' skills throughout these more complex tasks. We show in simulation that it has a more accurate model of the user's skills and, therefore, can select better teaching actions than previous algorithms. Lastly, we show that a robot can use TD-BKT to model a user and teach electronic circuit tasks to participants during a user study. Our results show that participants significantly improved their skills when modeled using TD-BKT.

Enhancement of Prediction Model for Students’ Performance in Intelligent Tutoring System

Adapting Artificial Intelligence to Intelligent Tutoring System (ITS) has made teaching and learning more effective. Prediction of students’ performance has gained more interest among researchers to know whether the students master their learning before moving to another topic. For the research scope, we have analyzed numerous Bayesian Knowledge Tracing (BKT) variations in methodology and found that the most precise way to forecast students’ success is through Individualized Bayesian Knowledge Tracing (iBKT). Although iBKT makes a good prediction, iBKT does not consider other knowledge-related elements, such as learning and guess rate, and only uses students’ prior knowledge as the parameters. Due to issues concerning uncertainties in students’ interactions, this study proposes to enhance the prediction function of the iBKT using a feature relating to students’ confidence levels. Thus, this new confidence parameter is defined as P(C), assumed to improve prediction accuracy when forecasting student achievement. The prediction accuracy is tested using the attributes of the ASSISTment and Knowledge Discovery and Data Mining (KDD) datasets as input. In addition, Root Mean Square Error (RMSE) is applied to calculate the performance accuracy of iBKT and enhanced iBKT with the confidence parameter. As a result, the RMSE performance accuracy of iBKT with the confidence parameter shows a low RMSE score for both datasets. The ASSISTment dataset provides a higher prediction when applying the confidence parameter, 0.21190. Therefore, it is concluded that enhancing the confidence parameter is an effective method with accuracy improvement for predicting students’ success in ITS.

Tracing Knowledge States through Student Assessment in a Blended Learning Environment

Blended learning has recently acquired popularity in a variety of educational settings. This approach has the advantage of being able to autonomously monitor students' knowledge states using the collected learning data. Moodle is the most widely used learning management system in blended learning environments. Students can access Moodle to obtain supplementary materials, exercises, and assessments to complement their face-to-face meetings. However, its performance can be improved by more effectively tailoring students' skills and pace of learning. Several studies have been conducted on knowledge tracing; however, we have not discovered any studies that particularly investigate knowledge tracing in a blended learning setting with Moodle as a component. This study proposes a scheme for assessment using the features of the Moodle quiz platform. The assessment data is used to conduct knowledge tracing with the Bayesian Knowledge Tracing (BKT) model, which improves interpretability. The aforementioned data were collected from information engineering undergraduate students who completed 88 exercises that assessed 23 knowledge components within the course. We measure RMSE and MAE to evaluate the performance of the BKT model on our dataset. Furthermore, we compare the knowledge tracing performance to other well-known datasets. Our results show that the BKT model performed better with our dataset, with an RMSE of 0.314 and an MAE of 0.197. Moreover, the BKT model can be used to assess student performance and determine the level of mastery for each knowledge component. Thus, the outcomes can be applied to personalized learning in the future.

Evaluating the Effectiveness of Bayesian Knowledge Tracing Model-Based Explainable Recommender

Explainable recommendation, which provides an explanation about why a quiz is recommended, helps to improve transparency, persuasiveness, and trustworthiness. However, little research examined the effectiveness of the explainable recommender, especially on academic performance. To survey its effectiveness, the authors evaluate the math academic performance among middle school students (n=115) by giving pre- and post-test questions based evaluation techniques. During the pre- and post-test periods, students were encouraged to use the Bayesian Knowledge Tracing model based explainable recommendation system. To evaluate how well the students were able to do what they could not do, the authors defined growth rate and found recommended quiz clicked counts had a positive effect on the total number of solved quizzes (R=0.343, P=0.005) and growth rate (R=0.297, P=0.017) despite no correlation between the total number of solved quizzes and growth rate. The results suggest that the use of an explainable recommendation system that learns efficiently will enable students to do what they could not do before.

Twenty-Five Years of Bayesian knowledge tracing: a systematic review

Twenty-Five Years of Bayesian knowledge tracing: a systematic review

AI Adaptivity in a Mixed-Reality System Improves Learning

AbstractAdaptivity in advanced learning technologies offer the possibility to adapt to different student backgrounds, which is difficult to do in a traditional classroom setting. However, there are mixed results on the effectiveness of adaptivity based on different implementations and contexts. In this paper, we introduce AI adaptivity in the context of a new genre of Intelligent Science Stations that bring intelligent tutoring into the physical world. Intelligent Science Stations are mixed-reality systems that bridge the physical and virtual worlds to improve children’s inquiry-based STEM learning. Automated reactive guidance is made possible by a specialized AI computer vision algorithm, providing personalized interactive feedback to children as they experiment and make discoveries in their physical environment. We report on a randomized controlled experiment where we compare learning outcomes of children interacting with the Intelligent Science Station that has task-loop adaptivity incorporated, compared to another version that provides tasks randomly without adaptivity. Our results show that adaptivity using Bayesian Knowledge Tracing in the context of a mixed-reality system leads to better learning of scientific principles, without sacrificing enjoyment. These results demonstrate benefits of adaptivity in a mixed-reality setting to improve children’s science learning.

Adapting Knowledge Inference Algorithms to Measure Geometry Competencies through a Puzzle Game

The rapid technological evolution of the last years has motivated students to develop capabilities that will prepare them for an unknown future in the 21st century. In this context, many teachers intend to optimise the learning process, making it more dynamic and exciting through the introduction of gamification. Thus, this article focuses on a data-driven assessment of geometry competencies, which are essential for developing problem-solving and higher-order thinking skills. Our main goal is to adapt, evaluate and compare Bayesian Knowledge Tracing (BKT), Performance Factor Analysis (PFA), Elo, and Deep Knowledge Tracing (DKT) algorithms applied to the data of a geometry game named Shadowspect, in order to predict students’ performance by means of several classifier metrics. We analysed two algorithmic configurations, with and without prioritisation of Knowledge Components (KCs) – the skills needed to complete a puzzle successfully, and we found Elo to be the algorithm with the best prediction power with the ability to model the real knowledge of students. However, the best results are achieved without KCs because it is a challenging task to differentiate between KCs effectively in game environments. Our results prove that the above-mentioned algorithms can be applied in formal education to improve teaching, learning, and organisational efficiency.

An Introduction to Bayesian Knowledge Tracing with pyBKT

This study aims to introduce Bayesian Knowledge Tracing (BKT), a probabilistic model used in educational data mining to estimate learners’ knowledge states over time. It also provides a practical guide to estimating BKT models using the pyBKT library available in Python. The first section presents an overview of BKT by explaining its theoretical foundations and advantages in modeling individual learning processes. In the second section, we describe different variants of the standard BKT model based on item response theory (IRT). Next, we demonstrate the estimation of BKT with the pyBKT library in Python, outlining data pre-processing steps, parameter estimation, and model evaluation. Different cases of knowledge tracing tasks illustrate how BKT estimates learners’ knowledge states and evaluates prediction accuracy. The results highlight the utility of BKT in capturing learners’ knowledge states dynamically. We also show that the model parameters of BKT resemble the parameters from logistic IRT models.

Dynamic Cognitive Diagnosis: An Educational Priors-Enhanced Deep Knowledge Tracing Perspective

To provide personalized support on educational platforms, it is crucial to model the evolution of students' knowledge states. Knowledge tracing is one of the most popular technologies for this purpose, and deep learning-based methods have achieved state-of-the-art performance. Compared to classical models, such as Bayesian knowledge tracing, which track students' knowledge proficiencies, deep learning-based knowledge tracing is usually modeled to predict students' performances on questions, while ignoring the interpretability of students' knowledge states. However, for many practical applications such as learning resource recommendation, it would be more helpful if we could explicitly track the students' abilities or knowledge proficiencies separately from performance prediction. Researchers in psychometric area already designed cognitive diagnosis solutions to quantify the knowledge states of students in static conditions (e.g., examination), where the educational priors (i.e., factors related to students' learning process) were proved beneficial for student modeling. Inspired by this, we propose <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Dynamic Cognitive Diagnosis</i> , which integrates the interpretability of educational priors from cognitive diagnosis into deep learning-based knowledge tracing methods. We first discuss and provide evidence of which educational priors can be integrated, including question attributes and interaction function. Then we show the effects of using the educational priors in deep learning-based knowledge tracing from two aspects, i.e., interpretability and accuracy. Through extensive experiments and analyses, we prove that properly chosen priors can enable deep learning-based methods to evaluate students' knowledge states in a manner that is consistent with domain knowledge or human experience. Moreover, educational priors also improve the accuracy of student performance prediction.

Interpretable Knowledge Tracing: Simple and Efficient Student Modeling with Causal Relations

Intelligent Tutoring Systems have become critically important in future learning environments. Knowledge Tracing (KT) is a crucial part of that system. It is about inferring the skill mastery of students and predicting their performance to adjust the curriculum accordingly. Deep Learning based models like Deep Knowledge Tracing (DKT) and Dynamic Key-Value Memory Network (DKVMN) have shown significant predictive performance compared with traditional models like Bayesian Knowledge Tracing (BKT) and Performance Factors Analysis (PFA). However, it is difficult to extract psychologically meaningful explanations from the tens of thousands of parameters in neural networks, that would relate to cognitive theory. There are several ways to achieve high accuracy in student performance prediction but diagnostic and prognostic reasonings are more critical in learning science. In this work, we present Interpretable Knowledge Tracing (IKT), a simple model that relies on three meaningful features: individual skill mastery, ability profile (learning transfer across skills) and problem difficulty by using data mining techniques. IKT’s prediction of future student performance is made using a Tree Augmented Naive Bayes Classifier (TAN), therefore its predictions are easier to explain than deep learning based student models. IKT also shows better student performance prediction than deep learning based student models without requiring a huge amount of parameters. We conduct ablation studies on each feature to examine their contribution to student performance prediction. Thus, IKT has great potential for providing adaptive and personalized instructions with causal reasoning in real-world educational systems.

Ensemble Knowledge Tracing: Modeling interactions in learning process

Knowledge Tracing (KT) aims to continuously estimate students’ evolving knowledge state during their learning process, which has attracted much research attention due to its potential for delivering personalized and optimal experiences to students in intelligent learning systems. The learning process is essentially the pairwise interactions of Students, Concepts, and Questions (S–C, S–Q, C–Q for short). Modeling all these interactions will improve the performance of KT. However, existing KT methods hardly exploit all the interactions in a single model. Specifically, Bayesian Knowledge Tracing (BKT) and most of its variants neglect C–Q; Deep Knowledge Tracing (DKT) and other deep neural network approaches mostly neglect S–Q and C–Q. We propose the Ensemble Knowledge Tracing (EnKT), which models all three types of interactions. The base model of EnKT is a hybrid of BKT and DKT. We also present an ensemble algorithm Recurrent Boosting (RB), which extends AdaBoost to deal with KT sequential data. Inspired by BKT, EnKT represents S–C and S–Q using learning and performance parameters, respectively. Besides, EnKT defines C–Q as the correlation complexity among the concepts involved in a question. Experiments show EnKT significantly outperforms state-of-the-art methods (by up to 6% in AUC in some cases) on four real-world benchmark datasets and illustrate better interpretability by several typical case studies.

Detecting threshold concepts through Bayesian knowledge tracing: examining research skill development in biological sciences at the doctoral level

Detecting threshold concepts through Bayesian knowledge tracing: examining research skill development in biological sciences at the doctoral level

An Extensible Heterogeneous Network Embedding Framework for Knowledge Tracing

Knowledge tracing is of great significance for providing better personalized learning guidance and has thus attracted extensive research attention in recent years. The task of knowledge tracing is to model students' learning process on the basis of historical exercise records and trace students' knowledge proficiency, thereby predicting students' performance on future exercises or recommending exercises for better proficiency. Existing methods focus on either the skill level or the exercise level, ignoring the relationships among exercises and Knowledge Components (KCs). The classical single-factor models include the Deep Knowledge Tracing (DKT) {model} and the Dynamic Key-Value Memory Network (DKVMN) model. Although a few models, such as the Bayesian Knowledge Tracing (BKT) model and the Knowledge Proficiency Tracing (KPT) model, utilize the Q-matrix to improve model performance, most of them ignore the interaction among KCs, not to mention models that do not use the Q-matrix. Inspired by the recent success of network embedding, this paper presents a heterogeneous network embedding framework for knowledge tracing called HNEKT that takes both exercises and KCs into account. To adapt to the application of knowledge tracing, this paper also proposes several meta-paths to generate meaningful node sequences for network embedding. Besides, it explores other side information as well to improve the extensibility and effectiveness of the proposed model. Extensive experiments on three real-world datasets demonstrate the effectiveness of the HNEKT model.

A Literature Review of Knowledge Tracing for Student Modeling : Research Trends, Models, Datasets, and Challenges

Modeling students' knowledge is a fundamental part of online learning platforms. Knowledge tracing is an application of student modeling which renowned for its ability to trace students' knowledge. Knowledge tracing ability can be used in online learning platforms for predicting learning performance and providing adaptive learning. Due to the wide uses of knowledge tracing in student modeling, this study aims to understand the state-of-the-art and future research of knowledge tracing. This study focused on reviewing 24 studies published between 2017 to the third quarter of 2021 in four digital databases. The selected studies have been filtered using inclusion and exclusion criteria. Several previous studies have shown that there are two approaches used in knowledge tracing, including probabilistic and deep learning. Bayesian Knowledge Tracing model is the most widely used in the probabilistic approach, while the Deep Knowledge Tracing model is the most popular model in the deep learning approach. Meanwhile, ASSISTments 2009–2010 is the most frequently tested dataset for probabilistic and deep learning approaches. In the future, additional studies are required to explore several models which have been developed previously. Therefore this study provides direction for future research of each existing approach.

Immersive Intelligent Tutoring System for Remedial Learning Using Virtual Learning Environment

Intelligent Tutoring System (ITS) has been widely used in supporting personal learning. However, there is an aspects that have not become focus in ITS, namely immersive. This research proposes an Immersive Intelligent Tutoring (IIT) model based on Bayesian Knowledge Tracing (BKT) for determining the learner’s characteristics and learning content delivery strategies using genetic algorithms. The model uses remedial learning with a faded worked-out example. This study uses a 3-Dimensional Virtual Learning Environment (3DMUVLE) that implements immersive features to increase intrinsic motivation. This model was built using a client / server architecture. The server side component uses the MOODLE, the client side component uses OpenSim and its viewers, and the middleware component uses the Simulation Linked Object Oriented Dynamic Learning Environment (SLOODLE). Model testing is performed on user acceptance using a combination of Technology Acceptance Model (TAM) and Hedonic-Motivation System Adoption Model (HMSAM) and the impact of the model in learning using statistical test. The results showed 83% of the learners felt happy with the learning. Meanwhile, the evaluation of the impact on learning outcomes shows that the use of this model is significantly different from traditional learning.

Fuzzy Bayesian Knowledge Tracing

Online education promotes the sharing of learning resources. Knowledge tracing (KT) is aimed at tracking the cognition function of students according to their performance on various exercises at different times and has attracted considerable attention. Existing KT models primarily use bisection representations for the performance and cognitive states of students, thus limiting the application scope of these models and the accuracy of the evaluation of student cognitive performance in learning processes. Therefore, fuzzy Bayesian KT models (namely, FBKT and T2FBKT) are proposed to address continuous score scenarios (e.g., subjective examinations) so that the applicability of KT models may be broadened. Moreover, fine-grained cognitive states can be discerned. In particular, referring to type-2 fuzzy theory, T2FBKT mitigates the model uncertainty of FBKT induced by uncertain parameters. Finally, extensive experiments demonstrate the effectiveness of the proposed fuzzy KT models.

Enhancing skill prediction through generalising Bayesian knowledge tracing

Learning Analytics (LA) have been widely investigated and applied to understand and optimise the learning process and environment. Among a number of LA tools, Bayesian Knowledge Tracing (BKT) was d...

Bayesian Knowledge Tracing for Navigation through Marzano’s Taxonomy

Bayesian Knowledge Tracing for Navigation through Marzano’s Taxonomy

Enhancing skill prediction through generalising Bayesian knowledge tracing

Enhancing skill prediction through generalising Bayesian knowledge tracing