Adaptive Instructional Systems Research Articles

How to Optimize Student Learning Using Student Models That Adapt Rapidly to Individual Differences

An important component of many Adaptive Instructional Systems (AIS) is a ‘Learner Model’ intended to track student learning and predict future performance. Predictions from learner models are frequently used in combination with mastery criterion decision rules to make pedagogical decisions. Important aspects of learner models, such as learning rate and item difficulty, can be estimated from prior data. A critical function of AIS is to have students practice new content once the AIS predicts that they have ‘mastered’ current content or learned it to some criterion. For making this prediction, individual student parameters (e.g., for learning rate) are frequently unavailable due to having no prior data about a student, and thus population-level parameters or rules-of-thumb are typically applied instead. In this paper, we will argue and demonstrate via simulation and data analysis that even in best-case scenarios, learner models assuming equal learning rates for students will inevitably lead to systematic errors that result in suboptimal pedagogical decisions for most learners. This finding leads us to conclude that systematic errors should be expected, and mechanisms to adjust predictions to account for them should be included in AIS. We introduce two solutions that can adjust for student differences “online” in a running system: one that tracks systemic errors of the learner model (not the student) and adjusts accordingly, and a student-level performance adaptive feature. We demonstrate these solutions’ efficacy and practicality on six large educational datasets and show that these features improved model accuracy in all tested datasets.

Student Modeling and Analysis in Adaptive Instructional Systems

There is a growing interest in developing and implementing adaptive instructional systems to improve, automate, and personalize student education. A necessary part of any such adaptive instructional system is a student model used to predict or analyze learner behavior and inform adaptation. To help inform researchers in this area, this paper presents a state-of-the-art review of 11 years of research (2010-2021) in student modeling, focusing on learner characteristics, learning indicators, and foundational aspects of dissimilar models. We mainly emphasize increased prediction accuracy when using multidimensional learner data to create multimodal models in real-world adaptive instructional systems. In addition, we discuss challenges inherent in real-world multimodal modeling, such as uncontrolled data collection environments leading to noisy data and data sync issues. Finally, we reinforce our findings and conclusions through an industry case study of an adaptive instructional system. In our study, we verify that adding multiple data modalities increases our model prediction accuracy from 53.3% to 69%. At the same time, the challenges encountered with our real-world case study, including uncontrolled data collection environment with inevitably noisy data, calls for synchronization and noise control strategies for data quality and usability.

Questioning the Scope of AI Standardization in Learning, Education, and Training

Well-defined terminology and scope are essential in formal standardization work. In the broad domain of Information and Communications Technology (ICT) the necessity is even more so due to proliferation and appropriation of terms from other fields and public discourse – the term ‘smart’ is a classic example; as is ‘deep learning’. In reviewing the emerging impact of Artificial Intelligence (AI) on the field of Information Technology for Learning, Education, and Training (ITLET), this paper highlights several questions that might assist in developing scope statements of new work items.While learners and teachers are very much foregrounded in past and present standardization efforts in ITLET, little attention has been placed until recently on whether these learners and teachers are necessarily human. Now that AI is a hot spot of innovation it is receiving considerable attention from standardization bodies such as ISO/IEC, IEEE and pan-European initiatives such as the Next Generation Internet. Thus, terminology such as ‘blended learning’ necessarily now spans not just humans in a mix of online and offline learning, but also mixed reality and AI paradigms, developed to assist human learners in environments such as Adaptive Instructional Systems (AIS) that extend the scope and design of a learning experience where a symbiosis is formed between humans and AI. Although the fields of LET and AI may utilize similar terms, the language of AI is mathematics and terms can mean different things in each field. Nonetheless, in ‘symbiotic learning’ contexts where an AIS at times replaces a human teacher, a symbiosis between the human learner and the AIS occurs in such a way where both can exist as teacher and learner. While human ethics and values are preeminent in this new symbiosis, a shift towards a new ‘intelligence nexus’ is signalled where ethics and values can also apply to AI in learning, education, and training (LET) contexts. In making sense of the scope of standardization efforts in the context of LET based AI, issues for the human-computer interface become more complex than simply appropriating terminology such as ‘smart’ in the next era of standardization. Framed by ITLET perspectives, this paper focuses on detailing the implications for standardization and key questions arising from developments in Artificial Intelligence. At a high level, we need to ask: do the scopes of current LET related Standards Committees still apply and if not, what scope changes are needed?

An agent-based intelligent tutoring systems review

This paper presents the current state of research in the fields of intelligent tutoring systems and intelligent agents. The architecture of intelligent tutoring systems typically consists of four major components: the domain knowledge module, the learner diagnosis module, the tutoring module, and the communication module. An agent-based approach to intelligent tutoring systems research and development is applied as the set of agents that constitute the general architecture, and as animated characters - pedagogical agents. Therefore, the paper focuses on the components of intelligent tutoring systems that can be implemented as agents, as well as on functions that these intelligent agents can perform in each intelligent tutoring system component. Also, we present the recent endeavours in the intelligent tutoring systems community that moves from standalone systems focused on a single instructional domain to adaptive instructional systems that comprise learners, intelligent tutoring systems, and external (non-adaptive) environment.

Adaptivity in educational systems for language learning: a review

ABSTRACTAdaptive and intelligent instructional systems are used to deal with the issue of learning personalisation in contexts where human instructors are not immediately available, so their role is transferred entirely or in part onto the computer. Even though such systems are mostly developed for well-defined domains that have a rather straightforward acquisition order, such as mathematics or computer programming, they found their application in ill-defined domains as well. Natural language learning is one such domain, and developing adaptive instructional systems for this specific purpose is notoriously complex and challenging due to the nature of language systems. The paper at hand examines the theoretical background of adaptivity and intelligence in instructional systems, and discusses their application in learning and teaching of natural languages. Moreover, the paper reviews adaptive and intelligent language learning systems in existence and identifies their characteristics in a systematic manner. The discussion section offers a more detailed view of selected systems for language learning that exhibit interesting and innovative implementation solutions. The paper concludes by suggesting possible developmental directions and future work in the field.

A Meta-Analysis of the Effects of Feedback in Computer-Based Instruction

A quantitative research synthesis (meta-analysis) was conducted on the literature concerning the effects of feedback on learning from computer-based instruction (CBI). Despite the widespread acceptance of feedback in computerized instruction, empirical support for particular types of feedback information has been inconsistent and contradictory. Effect size calculations from twenty-two studies involving the administration of immediate achievement posttests resulted in a weighted mean effect size of .80. Also, a mean weighted effect size of .35 was obtained from nine studies involving delayed posttest administration. Feedback effects on learning and retention were found to vary with CBI typology, format of unit content and access to supplemental materials. Results indicate that the diagnostic and prescriptive management strategies of computer-based adaptive instructional systems provide the most effective feedback. The implementation of effective feedback in computerized instruction involves the computer's ability to verify the correctness of the learner's answer and the underlying causes of error.

Levels and types of mediation in instructional systems: an individual differences approach

Thirty-eight university students were tested for field-dependence/-independence using Riding's computer-administered Cognitive Styles Analysis (CSA). They also learned using computerized versions of Pask and Scott's teaching materials designed to suit holist and serialist learning strategies. It was found that (a) students' holist and serialist competence could be predicted using CSA scores, (b) learning in matched conditions (using instructional materials structured to suit their learning styles) was significantly superior for both holists and serialists than in mismatched conditions, and (c) serialist instructional materials resulted in overall better learning performance and efficiency than did holist materials. Possible reasons for the lack of positive correlations reported in previous studies, along with implications for the development of user models to support the development of adaptive instructional systems, are discussed.