Dialogue-based Intelligent Tutoring System Research Articles

DIRECT: Toward Dialogue-Based Reading Comprehension Tutoring

A major challenge in education is to provide students with a personalized learning experience. This study aims to address this by developing a dialogue-based intelligent tutoring system (ITS) that imitates human expert tutors. The ITS asks questions, assesses student answers, provides hints, and even chats to encourage student engagement. We constructed the Dialogue-based Reading Comprehension Tutoring (DIRECT) dataset to simulate real-world pedagogical scenarios with the assessment labels and key sentences to support tutoring. The DIRECT dataset is based on RACE, which is a large-scale English reading comprehension dataset. In addition, we propose a neural pipeline approach to model the tutoring tasks and conduct a comprehensive analysis on the results, including a human evaluation. The results show that our model performs well in generating questions, assessing answers, and chatting, showing high potential although some challenges remain. The proposed model provides a good basis for further development of dialogue-based ITSs.

CQACD: A Concept Question-Answering System for Intelligent Tutoring Using a Domain Ontology With Rich Semantics

In this study, a Concept Question Answering system applied to the Computer Domain (CQACD) for intelligent tutoring is proposed. This system is a dialogue-based Intelligent Tutoring System (ITS) that allows the tutor and student with mixed-initiative and natural language to ask each other questions concerning the basic computer knowledge in the <i>Computer Basics</i> course. CQACD is based on constructivist principles and encourages the learner to construct knowledge rather than merely receiving knowledge, which has the following characteristics: (a) this system employs a domain ontology with rich semantic relationships to model the basic computer knowledge and build up a concept-centric knowledge model, (b) uses a limited number of 80 input templates with description logics to acquire the intention of questions posed by students, (c) a textual entailment algorithm with semantic technologies is proposed to match the input template and assess the student&#x2019;s contribution to improve the flexibility of the system, and (d) an ontology-driven dialogue management mechanism is proposed, which can quickly form the conversational content and conversational sequence. The experimental results show that CQACD can replace the teachers&#x2019; tutoring in large classes and can promote the learning of poor students in large classes better than teachers can. The paper reveals that the domain ontology with rich semantic relationships plays an important role in the Concept Question Answer System (CQAS). It can model CQAS&#x2019;s discipline knowledge, provide structured domain knowledge for student model, template design and matching, and provide basic architectural architecture for dialogue management.

Is it a good move? Mining effective tutoring strategies from human–human tutorial dialogues

To construct dialogue-based Intelligent Tutoring Systems (ITS) with sufficient pedagogical expertise, a trendy research method is to mine large-scale data collected by existing dialogue-based ITS or generated between human tutors and students to discover effective tutoring strategies. However, most of the existing research has mainly focused on the analysis of successful tutorial dialogue. We argue that, to better inform the design of dialogue-based ITS, it is also important to analyse unsuccessful tutorial dialogues and gain a better understanding of the reasons behind those failures. Therefore, our study aimed to identify effective tutoring strategies by mining a large-scale dataset of both successful and unsuccessful human–human online tutorial dialogues, and further used these tutoring strategies for predicting students’ problem-solving performance. Specifically, the study adopted a widely-used educational dialogue act scheme to describe the action behind utterances made by a tutor/student in the broader context of a tutorial dialogue (e.g., asking/answering a question, providing hints). Frequent dialogue acts were identified and analysed by taking into account the prior progress that a student had made before the start of a tutorial session and the problem-solving performance the student achieved after the end of the session. Besides, we performed a sequence analysis on the inferred actions to identify prominent patterns that were closely related to students’ problem-solving performance. These prominent patterns could shed light on the frequent strategies used by tutors. Lastly, we measured the power of these tutorial actions in predicting students’ problem-solving performance by applying a well-established machine learning method, Gradient Tree Boosting (GTB). Through extensive analysis and evaluations, we identified a set of different action patterns that were pertinent to tutors and students across dialogues of different traits, e.g., students without prior progress in solving problems, compared to those with prior progress, were likely to receive more thought-provoking questions from their tutors. More importantly, we demonstrated that the actions taken by students and tutors during a tutorial process could not adequately predict student performance and should be considered together with other relevant factors (e.g., the informativeness of the utterances).

Automated Data-Driven Generation of Personalized Pedagogical Interventions in Intelligent Tutoring Systems

Intelligent tutoring systems (ITS) have been shown to be highly effective at promoting learning as compared to other computer-based instructional approaches. However, many ITS rely heavily on expert design and hand-crafted rules. This makes them difficult to build and transfer across domains and limits their potential efficacy. In this paper, we investigate how feedback in a large-scale ITS can be automatically generated in a data-driven way, and more specifically how personalization of feedback can lead to improvements in student performance outcomes. First, in this paper we propose a machine learning approach to generate personalized feedback in an automated way, which takes individual needs of students into account, while alleviating the need of expert intervention and design of hand-crafted rules. We leverage state-of-the-art machine learning and natural language processing techniques to provide students with personalized feedback using hints and Wikipedia-based explanations. Second, we demonstrate that personalized feedback leads to improved success rates at solving exercises in practice: our personalized feedback model is used in Korbit, a large-scale dialogue-based ITS with around 20,000 students launched in 2019. We present the results of experiments with students and show that the automated, data-driven, personalized feedback leads to a significant overall improvement of 22.95% in student performance outcomes and substantial improvements in the subjective evaluation of the feedback.

Deep Discourse Analysis for Generating Personalized Feedback in Intelligent Tutor Systems

We explore creating automated, personalized feedback in an intelligent tutoring system (ITS). Our goal is to pinpoint correct and incorrect concepts in student answers in order to achieve better student learning gains. Although automatic methods for providing personalized feedback exist, they do not explicitly inform students about which concepts in their answers are correct or incorrect. Our approach involves decomposing students answers using neural discourse segmentation and classification techniques. This decomposition yields a relational graph over all discourse units covered by the reference solutions and student answers. We use this inferred relational graph structure and a neural classifier to match student answers with reference solutions and generate personalized feedback. Although the process is completely automated and data-driven, the personalized feedback generated is highly contextual, domain-aware and effectively targets each student's misconceptions and knowledge gaps. We test our method in a dialogue-based ITS and demonstrate that our approach results in high-quality feedback and significantly improved student learning gains.

An application of Chinese dialogue-based intelligent tutoring system in remedial instruction for mathematics learning

The present study aims to examine the pedagogical effectiveness of a Chinese mathematical dialogue-based intelligent tutoring system used for teaching mathematics. The mathematical unit ‘multiplication and division of time expressions’ was taught to 134 fifth-grade students in three types of instruction conditions: the intelligent tutoring system (ITS), conventional teacher instruction and material reading. The results show that student performance was comparable between the proposed mathematical ITS and conventional instruction conditions but was significantly poorer when no teaching method was used. The questionnaire survey showed that the ITS method not only improved maths learning, but also increased motivation among the fifth graders. The effectiveness of ITS remedial instruction in mathematics is discussed.

Non-intrusive assessment of learners’ prior knowledge in dialogue-based intelligent tutoring systems

This article describes a study whose goal was to assess students’ prior knowledge level with respect to a target domain based solely on characteristics of the natural language interaction between students and conversational Intelligent Tutoring Systems (ITSs). We report results on data collected from two conversational ITSs: a micro-adaptive-only ITS and a fully-adaptive (micro- and macro-adaptive) ITS. These two ITSs are in fact different versions of the state-of-the-art conversational ITS DeepTutor ( http://www.deeptutor.org ). Our models rely on both dialogue and session interaction features including time on task, student generated content features (e.g., vocabulary size or domain specific concept use), and pedagogy-related features (e.g., level of scaffolding measured as number of hints). Linear regression models were explored based on these features in order to predict students’ knowledge level, as measured with a multiple-choice pre-test, and yielded in the best cases an r=0.949 and adjusted r-square =0.833. We discuss implications of our findings for the development of future ITSs.

Improving the efficiency of dialogue in tutoring

The current studies investigated the efficient use of dialogue in intelligent tutoring systems that use natural language interaction. Such dialogues can be relatively time-consuming. This work addresses the question of how much dialogue is needed to produce significant learning gains. In Experiment 1, a full dialogue condition and a read-only control condition were compared with a mixed dialogue condition in which students engaged in full dialogue for half the problems followed by problems requiring only a limited engagement. We found that the mixed dialogue condition produced results as impressive as the full dialogue condition and took less time. Experiment 2 replicated these findings and further examined issues of time engaged in learning, quality of instruction, and learning gains. Overall, these results show that dialogue-based intelligent tutoring systems could be designed in a more efficient manner to maximize learning and minimize the cost of time-on-task.

Research Methods Tutor: Evaluation of a dialogue-based tutoring system in the classroom

Research Methods Tutor (RMT) is a dialogue-based intelligent tutoring system for use in conjunction with undergraduate psychology research methods courses. RMT includes five topics that correspond to the curriculum of introductory research methods courses: ethics, variables, reliability, validity, and experimental design. We evaluated the effectiveness of the RMT system in the classroom using a nonequivalent control group design. Students in three classes (n = 83) used RMT, and students in two classes (n = 53) did not use RMT. Results indicated that the use of RMT yieldedstrong learning gains of 0.75 standard deviations above classroom instruction alone. Further, the dialogue-based tutoring condition of the system resulted in higher gains than did the textbook-style condition (CAI version) of the system. Future directions for RMT include the addition of new topics and tutoring elements.

Annotation of Tutorial Dialogue Goals for Natural Language Generation

We annotated transcripts of human tutoring dialogue for the purpose of constructing a dialogue-based intelligent tutoring system, CIRCSIM-Tutor. The tutors were professors of physiology who were also expert tutors. The students were 1st year medical students who communicated with the tutors using typed communication from separate rooms. The tutors made use of a rich variety of strategies, some specific to particular content areas and others more general, such as showing that the student holds contradictory beliefs about the domain. In this article, we describe our model of hierarchical goal structure for tutorial dialogues. We catalog each major pedagogical method we found in the dialogues, showing its structure and illustrating the features needed to represent each subgoal in its correct narrative and interpersonal context. We compare our goal structure with other analyses of tutorial dialogues.

Teaching the tacit knowledge of programming to noviceswith natural language tutoring

For beginning programmers, inadequate problem solving and planning skills are among the most salient of their weaknesses. In this paper, we test the efficacy of natural language tutoring to teach and scaffold acquisition of these skills. We describe ProPL (Pro-PELL), a dialogue-based intelligent tutoring system that elicits goal decompositions and program plans from students in natural language. The system uses a variety of tutoring tactics that leverage students' intuitive understandings of the problem, how it might be solved, and the underlying concepts of programming. We report the results of a small-scale evaluation comparing students who used ProPL with a control group who read the same content. Our primary findings are that students who received tutoring from ProPL seem to have developed an improved ability to solve the composition problem and displayed behaviors that suggest they were able to think at greater levels of abstraction than students in the read-only group.

Coached program planning

Coached program planning is a dialogue-based style of tutoring aimed at helping novices during the early stages of program writing. The intent is to help novices understand and solve problems in their own words through the construction of natural-language style pseudocode as the first step in solving a programming problem. We have designed an environment supporting coached program planning and have used it in a human-to-human, computer-mediated evaluation of 16 novice programmers enrolled in a pre-CS1 programming course at the University of Pittsburgh. The results show that students who underwent coached program planning, compared to those who did not, were more prolific with comments in their programs, committed fewer structural mistakes, and exhibited less erratic programming behavior during their implementation. The dialogues collected from this experiment followed a clear 4-step pattern. Starting with this observation, we are developing a dialogue-based intelligent tutoring system called the Pseudocode Tutor to support coached program planning.