Conversational Intelligent Tutoring Systems Research Articles

On improving conversational interfaces in educational systems

Conversational Intelligent Tutoring Systems (CITS) have drawn increasing interest in education because of their capacity to tailor learning experiences, improve user engagement, and contribute to the effective transfer of knowledge. Conversational agents employ advanced natural language techniques to engage in a convincing human-like tutorial conversation. In solving math word problems, a significant challenge arises in enabling the system to understand user utterances and accurately map extracted entities to the essential problem quantities required for problem-solving, despite the inherent ambiguity of human natural language. In this study, we propose two possible approaches to enhance the performance of a particular CITS designed to teach learners to solve arithmetic-algebraic word problems. Firstly, we propose an ensemble approach to intent classification and entity extraction, which combines the predictions made by two distinct individual models that use constraints defined by human experts. This approach leverages the intertwined nature of the intents and entities to yield a comprehensive understanding of the user’s utterance, ultimately aiming to enhance semantic accuracy. Secondly, we introduce an adapted Term Frequency-Inverse Document Frequency technique to associate entities with problem quantity descriptions. The evaluation was conducted on the AWPS and MATH-HINTS datasets, containing conversational data and a collection of arithmetical and algebraic math problems, respectively. The results demonstrate that the proposed ensemble approach outperforms individual models, and the proposed method for entity-quantity matching surpasses the performance of typical text semantic embedding models.

Automated Assessment of Initial Answers to Questions in Conversational Intelligent Tutoring Systems: Are Contextual Embedding Models Really Better?

This paper assesses the ability of semantic text models to assess student responses to electronics questions compared with that of expert human judges. Recent interest in text similarity has led to a proliferation of models that can potentially be used for assessing student responses. However, it is unclear whether these models perform as well as early models of distributional semantics. We assessed 5166 response pairings of 219 participants across 118 electronics questions and scored each with 13 different computational text models, including models that use Regular Expressions, distributional semantics, embeddings, contextual embeddings, and combinations of these features. Regular Expressions performed the best out of the stand-alone models. We show other semantic text models performing comparably to the Latent Semantic Analysis model that was originally used for the current task, and in a small number of cases outperforming the model. Models trained on a domain-specific electronics corpus for the task performed better than models trained on general language or Newtonian physics. Furthermore, semantic text models combined with RegEx outperformed stand-alone models in agreement with human judges. Tuning the performance of these recent models in Automatic Short Answer Grading tasks for conversational intelligent tutoring systems requires empirical analysis, especially in domain-specific areas such as electronics. Therefore, the question arises as to how well recent contextual embedding models compare with earlier distributional semantic language models on this task of answering questions about electronics. These results shed light on the selection of appropriate computational techniques for text modeling to improve the accuracy, recall, weighted agreement, and ultimately the effectiveness of automatic scoring in conversational ITSs.

Multiple Agent Designs in Conversational Intelligent Tutoring Systems

This article describes designs that use multiple conversational agents within the framework of intelligent tutoring systems. Agents in this case are computerized talking heads or embodied animated avatars that help students learn by performing actions and holding conversations with them in natural language. The earliest conversational intelligent tutoring systems were limited to a single agent that interacted with a student in the role of a teacher or expert. Technological advances have since made possible systems in which multiple agents interact with the learner and each other to model ideal behavior, strategies, reflections, and social interactions. Though still an emerging technology, multi-agent intelligent tutoring systems afford pedagogical benefits that go beyond the capabilities of the single-agent system and have facilitated learning gains on a variety of subject matters and skills, including science, technology, engineering, mathematics, research methods, metacognition, and language comprehension. The present work describes some common multi-agent designs that may be used to achieve a variety of pedagogical goals. We provide examples of how these designs have been implemented in educational or experimental settings and anticipate future use within the field of artificial intelligence.

On predicting learning styles in conversational intelligent tutoring systems using fuzzy decision trees

Abstract Intelligent Tutoring Systems personalise learning for students with different backgrounds, abilities, behaviours and knowledge. One way to personalise learning is through consideration of individual differences in preferred learning style. OSCAR is the name of a Conversational Intelligent Tutoring System that models a person's learning style using natural language dialogue during tutoring in order to dynamically predict, and personalise, their tutoring session. Prediction of learning style is undertaken by capturing independent behaviour variables during the tutoring conversation with the highest value variable determining the student's learning style. A weakness of this approach is that it does not take into consideration the interactions between behaviour variables and, due to the uncertainty inherently present in modelling learning styles, small differences in behaviour can lead to incorrect predictions. Consequently, the learner is presented with tutoring material not suited to their learning style. This paper proposes a new method that uses fuzzy decision trees to build a series of fuzzy predictive models combining these variables for all dimensions of the Felder Silverman Learning Styles model. Results using live data show the fuzzy models have increased the predictive accuracy of OSCAR-CITS across four learning style dimensions and facilitated the discovery of some interesting relationships amongst behaviour variables.

Non-intrusive assessment of learners’ prior knowledge in dialogue-based intelligent tutoring systems

This article describes a study whose goal was to assess students’ prior knowledge level with respect to a target domain based solely on characteristics of the natural language interaction between students and conversational Intelligent Tutoring Systems (ITSs). We report results on data collected from two conversational ITSs: a micro-adaptive-only ITS and a fully-adaptive (micro- and macro-adaptive) ITS. These two ITSs are in fact different versions of the state-of-the-art conversational ITS DeepTutor ( http://www.deeptutor.org ). Our models rely on both dialogue and session interaction features including time on task, student generated content features (e.g., vocabulary size or domain specific concept use), and pedagogy-related features (e.g., level of scaffolding measured as number of hints). Linear regression models were explored based on these features in order to predict students’ knowledge level, as measured with a multiple-choice pre-test, and yielded in the best cases an r=0.949 and adjusted r-square =0.833. We discuss implications of our findings for the development of future ITSs.

An adaptation algorithm for an intelligent natural language tutoring system

The focus of computerised learning has shifted from content delivery towards personalised online learning with Intelligent Tutoring Systems (ITS). Oscar Conversational ITS (CITS) is a sophisticated ITS that uses a natural language interface to enable learners to construct their own knowledge through discussion. Oscar CITS aims to mimic a human tutor by dynamically detecting and adapting to an individual's learning styles whilst directing the conversational tutorial. Oscar CITS is currently live and being successfully used to support learning by university students. The major contribution of this paper is the development of the novel Oscar CITS adaptation algorithm and its application to the Felder–Silverman learning styles model. The generic Oscar CITS adaptation algorithm uniquely combines the strength of an individual's learning style preference with the available adaptive tutoring material for each tutorial question to decide the best fitting adaptation. A case study is described, where Oscar CITS is implemented to deliver an adaptive SQL tutorial. Two experiments are reported which empirically test the Oscar CITS adaptation algorithm with students in a real teaching/learning environment. The results show that learners experiencing a conversational tutorial personalised to their learning styles performed significantly better during the tutorial than those with an unmatched tutorial.