Tutorial Dialogue Research Articles

Learning from watching dialog and monolog videos in online STEM courses

BackgroundInstructional videos have been increasingly critical to afford effective learning experiences in online courses, but most videos follow a lecturing monolog delivery format. This format tends to lead students to observe the videos passively. Previous laboratory studies have indicated that observing dialog videos of an instructor tutoring a tutee enhances student learning more than watching monolog videos of the same instructor lecturing. This paper describes two empirical studies that replicated the laboratory findings in large-scale college-level online STEM courses.ResultsThe results show that observing dialog videos of tutoring led to superior learning outcomes compared to monolog videos of lecturing, regardless of whether students observe the videos individually or collaboratively, as long as they engaged generatively with the materials. This finding was confirmed across two tested STEM domains.ConclusionsThe findings of these classroom studies suggest that observing dialog tutoring videos is a novel and robust online instructional format that can be generalized across STEM domains. The benefit of overhearing tutorial dialogs requires students to engage in Constructive and Interactive behaviors, as defined by the ICAP framework, rather than exposing students to long didactic lectures.

On the role of politeness in online human–human tutoring

AbstractResearchers have demonstrated that dialogue‐based intelligent tutoring systems (ITS) can be effective in assisting students in learning. However, little research has attempted to explore the necessity of equipping dialogue‐based ITS with one of the most important capabilities of human tutors, that is, maintaining polite interactions with students, which is essential to provide students with a pleasant learning experience. In this study, we examined the role of politeness by analysing a large‐scale real‐world dataset consisting of over 14K online human–human tutorial dialogues. Specifically, we employed linguistic theories of politeness to characterise the politeness levels of tutor–student‐generated utterances, investigated the correlation between the politeness levels of tutors' utterances and students' problem‐solving performance and quantified the power of politeness in predicting students' problem‐solving performance by applying Gradient Tree Boosting. The study results showed that: (i) in the effective tutorial sessions (ie, sessions in which students successfully solved problems), tutors tended to be very polite at the start of a tutorial session and become more direct to guide students as the session progressed; (ii) students with better performance in solving problems tended to be more polite at the beginning and the end of a tutorial session than their counterparts who failed to solve problems; (iii) the correlation between tutors' polite expressions and students' performance was not evident in non‐instructional communication; and (iv) politeness alone cannot adequately reveal students' problem‐solving performance, and thus other factors (eg, sentiment contained in utterances) should also be taken into account. Practitioner notesWhat is already known about this topic Human–human tutoring is acknowledged as an effective instructional method. Polite expression can help strengthen the relationship between tutors and students. Polite expression can promote students' learning achievements in many educational contexts. What this paper adds By considering the students' prior progress on a problem‐based learning task, we demonstrated the extent to which tutors and students display politeness in tutoring dialogues. Tutors' polite expressions might not correlate with students' problem‐solving performance in online human–human tutoring dialogues. Politeness alone was insufficient to predict the students' performance. Implications for practice Tutors might consider using words with positive sentiment values to express politeness to students with prior progress, which might encourage those students to make a further effort. The polite strategy of expressing indirect requests could help tutors mitigate the sense of directness, but this strategy should be carefully used in delivering instructional hints, especially for students without prior progress. To better assist students without prior progress, tutors might consider using more direct expression to explicitly guide students.

Is it a good move? Mining effective tutoring strategies from human–human tutorial dialogues

To construct dialogue-based Intelligent Tutoring Systems (ITS) with sufficient pedagogical expertise, a trendy research method is to mine large-scale data collected by existing dialogue-based ITS or generated between human tutors and students to discover effective tutoring strategies. However, most of the existing research has mainly focused on the analysis of successful tutorial dialogue. We argue that, to better inform the design of dialogue-based ITS, it is also important to analyse unsuccessful tutorial dialogues and gain a better understanding of the reasons behind those failures. Therefore, our study aimed to identify effective tutoring strategies by mining a large-scale dataset of both successful and unsuccessful human–human online tutorial dialogues, and further used these tutoring strategies for predicting students’ problem-solving performance. Specifically, the study adopted a widely-used educational dialogue act scheme to describe the action behind utterances made by a tutor/student in the broader context of a tutorial dialogue (e.g., asking/answering a question, providing hints). Frequent dialogue acts were identified and analysed by taking into account the prior progress that a student had made before the start of a tutorial session and the problem-solving performance the student achieved after the end of the session. Besides, we performed a sequence analysis on the inferred actions to identify prominent patterns that were closely related to students’ problem-solving performance. These prominent patterns could shed light on the frequent strategies used by tutors. Lastly, we measured the power of these tutorial actions in predicting students’ problem-solving performance by applying a well-established machine learning method, Gradient Tree Boosting (GTB). Through extensive analysis and evaluations, we identified a set of different action patterns that were pertinent to tutors and students across dialogues of different traits, e.g., students without prior progress in solving problems, compared to those with prior progress, were likely to receive more thought-provoking questions from their tutors. More importantly, we demonstrated that the actions taken by students and tutors during a tutorial process could not adequately predict student performance and should be considered together with other relevant factors (e.g., the informativeness of the utterances).

Linking Dialogue with Student Modelling to Create an Adaptive Tutoring System for Conceptual Physics

Jim Greer and his colleagues argued that student modelling is essential to provide adaptive instruction in tutoring systems and showed that effective modelling is possible, despite being enormously challenging. Student modelling plays a prominent role in many intelligent tutoring systems (ITSs) that address problem-solving domains. However, considerably less attention has been paid to using a student model to personalize instruction in tutorial dialogue systems (TDSs)—ITSs that engage students in natural-language, conceptual discussions. This paper describes Rimac, a TDS that tightly couples student modelling with tutorial dialogues about conceptual physics. Rimac is distinct from other TDSs insofar as it dynamically builds a persistent student model that guides reactive and proactive decision making in order to provide adaptive instruction. An initial pilot study set in high school physics classrooms compared a control version of Rimac without a student model with an experimental version that implemented a “poor man’s student model”; that is, the model was initialized using students’ pretest scores but not updated further. Both low and high prior knowledge students showed significant pretest to posttest learning gains. However, high prior knowledge students who used the experimental version of Rimac learned more efficiently than high prior knowledge students who used the control version. Specifically, high prior knowledge students who used the student model driven tutor took less time to complete the intervention but learned a similar amount as students who used the control version. A subsequent study found that both high and low prior knowledge students learned more efficiently from a version of the tutor that dynamically updates its student model during dialogues than from a control version that included the static “poor man’s student model.” We discuss future work needed to improve the performance of Rimac’s student model and to integrate TDSs in the classroom.

Learning by watching others learn: the use of videoed tutorials in undergraduate business education

ABSTRACT The Fourth Industrial Age (4IA) is likely to be accompanied simultaneously by an increase in technology-mediated learning and an urgent need for people to learn rapidly, effectively and collaboratively. This study investigates the potential of vicarious learning from videoed tutorials as a pedagogical tool suitable for the challenges of 4IA. Undergraduate Business students observed videos of student tutees responding to tutor prompts as they tackled open-ended and conceptually challenging problems. The results revealed that student observers self-reported: gains in their conceptual understanding from watching videoed tutorials; preferences for watching tutorial dialogues over alternative learning methods; and that watching videoed tutorials had positive impacts on their affect and access to additional learning-related information. The study concludes that vicarious learning from videoed tutorials is an accessible technology-mediated pedagogy that is achievable by mainstream educators and is effective in developing conceptual understanding, engaging students and providing access to additional learning-related information.

A Method for Automatically Analyzing Intelligent Tutoring System Dialogues with Coh-Metrix


 
 
 We developed a method for using Coh-Metrix to automatically analyze tutorial dialogues. Coh-Metrix, a web-based tool for automatically evaluating text, is freely available to researchers. We applied the method to 190 tutorial dialogues between women and BRCA Gist from two experiments. BRCA Gist is an intelligent tutoring system (ITS) to help women make decisions about genetic testing for breast cancer risk. Tutorial dialogues scored high on measures of textual cohesion (deep cohesion, referential cohesion, and the composite variable formality). They also scored high on measures of the situation model (LSA verb overlap and causal verb and causal particle). However, there was mixed support for the hypothesis that higher scores on Coh-Metrix variables would predict subsequent comprehension. A Coh-Metrix principle is that the observable cohesion of a text is a reliable guide to the coherence of the reader’s mental representation of that text. Thus it appears that interacting with BRCA Gist helped people form coherent mental representations of complex medical materials. We conclude that Coh-Metrix can be used to reliably assess tutorial dialogues and make inferences about the mental representations of people engaged in conversation with an ITS based on observable characteristics of the statements people make.
 
 

한국어 수업 대화의 화행 분석과 화행 자동분류를 위한 언어학적 기반연구

화행이란 의사소통 과정에서 발화자가 가지는 발화 의도를 말한다. 성공적인 의사소통을 위해서는 발화자의 화행을 정확하게 파악하는 것이 매우 중요하다. 본 논문에서는 한국어 대화체 문장의 화행자동분류를 위해, 화행을 결정짓는 요인이 무엇인지 언어학적으로 분석하고자 하였다. 한국어 수업 대화를 분석하여 화행 분류 체계를 자체 정립하였고, 언어학적 근거를 바탕으로 13개의 화행 분류 자질을 제안하였다. 또한, 제안하는 화행 분류 자질을 검증하고자 웨카(Weka)를 이용하여 정확률 실험을 진행하였다. 본 연구에서 제안한 다양한 언어학적 자질을 이용하여 발화의 화행을 자동분류한 경우 70.03%의 정확률을 보였다. 이는 유니그램과 바이그램만으로 화행을 자동분류한 베이스라인에 비해 약 30%p의 정확률이 향상된 결과이다.

IEL-based Formal Dialogue System for Tutorials

FFormal dialogue system for tutorials DL(IEL) T is introduced. The system allows for modelling certain behaviours related to questioning agendas observed in tutorial dialogues. Inferential Erotetic Logic is the underlying logic of questions used here. Tutorial dialogues retrieved from the Basic Electricity and Electronics Corpus are presented and analysed with the use of DL(IEL) T .

Pumps and Prompts for Gist Explanations in Tutorial Dialogues About Breast Cancer

Fuzzy-Trace Theory (FTT) generalizes research on discourse to predict how health messages can be better understood and remembered, thereby influencing decision making. Applying FTT, BRCA Gist delivers messages interactively through tutorial dialogues and is the first Intelligent Tutoring System designed to help laypeople make sound medical decisions. Previous research indicates that BRCA Gist helps people form useful “gist explanations,” which leads to improved knowledge, comprehension, and risk assessment. The present research examined the effectiveness of different BRCA Gist dialogue moves, including general pumps for information and prompts for specific information. Participants were randomly assigned to a control group or one of four BRCA Gist conditions evoking gist or verbatim representations crossed with general pumps or specific information prompts. Gist-evoking pumps by themselves produced significant gains in knowledge and risk assessment. Specific verbatim prompts increased knowledge without affecting risk assessment. Results are explained in light of memory research and FTT principles.

Active engagement in a web-based tutorial to prevent obesity grounded in Fuzzy-Trace Theory predicts higher knowledge and gist comprehension.

We used Sharable Knowledge Objects (SKOs) to create an Intelligent Tutoring System (ITS) grounded in Fuzzy-Trace Theory to teach women about obesity prevention: GistFit, getting the gist of healthy eating and exercise. The theory predicts that reliance on gist mental representations (as opposed to verbatim) is more effective in reducing health risks and improving decision making. Technical information was translated into decision-relevant gist representations and gist principles (i.e., healthy values). The SKO was hypothesized to facilitate extracting these gist representations and principles by engaging women in dialogue, "understanding" their responses, and replying appropriately to prompt additional engagement. Participants were randomly assigned to either the obesity prevention tutorial (GistFit) or a control tutorial containing different content using the same technology. Participants were administered assessments of knowledge about nutrition and exercise, gist comprehension, gist principles, behavioral intentions and self-reported behavior. An analysis of engagement in tutorial dialogues and responses to multiple-choice questions to check understanding throughout the tutorial revealed significant correlations between these conversations and scores on subsequent knowledge tests and gist comprehension. Knowledge and comprehension measures correlated with healthier behavior and greater intentions to perform healthy behavior. Differences between GistFit and control tutorials were greater for participants who engaged more fully. Thus, results are consistent with the hypothesis that active engagement with a new gist-based ITS, rather than a passive memorization of verbatim details, was associated with an array of known psychosocial mediators of preventive health decisions, such as knowledge acquisition, and gist comprehension.

Understanding genetic breast cancer risk: Processing loci of the BRCA Gist Intelligent Tutoring System

The BRCA Gist Intelligent Tutoring System helps women understand and make decisions about genetic testing for breast cancer risk. BRCA Gist is guided by Fuzzy-Trace Theory, (FTT) and built using AutoTutor LITE. It responds differently to participants depending on what they say. Seven tutorial dialogues requiring explanation and argumentation are guided by three FTT concepts: forming gist explanations in one's own words, emphasizing decision-relevant information, and deliberating the consequences of decision alternatives. Participants were randomly assigned to BRCA Gist, a control, or impoverished BRCA Gist conditions removing gist explanation dialogues, argumentation dialogues, or FTT images. All BRCA Gist conditions performed significantly better than controls on knowledge, comprehension, and risk assessment. Significant differences in knowledge, comprehension, and fine-grained dialogue analyses demonstrate the efficacy of gist explanation dialogues. FTT images significantly increased knowledge. Providing more elements in arguments against testing correlated with increased knowledge and comprehension.

Technology Support for Discussion Based Learning: From Computer Supported Collaborative Learning to the Future of Massive Open Online Courses

This article offers a vision for technology supported collaborative and discussion-based learning at scale. It begins with historical work in the area of tutorial dialogue systems. It traces the history of that area of the field of Artificial Intelligence in Education as it has made an impact on the field of Computer-Supported Collaborative Learning through the creation of forms of dynamic support for collaborative learning, and makes an argument for the importance of advances in the field of Language Technologies for this work. In particular, this support has been enabled by an integration of text mining and conversational agents to form a novel type of micro-script support for productive discussion processes. This research from the early part of the century has paved the way for emerging technologies that support discussion-based learning at scale in Massive Open Online Courses (MOOCs). In the next 25 years, we expect to see this early, emerging work in MOOC contexts grow into ubiquitously available social learning approaches in free online learning environments like MOOCs, or what comes next in the online learning space. These ambitious social learning approaches include Problem Based Learning, Team Project Based Learning, and Collaborative Reflection. We expect to see the capability of drawing in and effectively supporting learners of all walks of life, especially impacting currently under-served learners. To that end, we describe the current exploratory efforts to deploy technology supported collaborative and discussion-based learning in MOOCs and offer a vision for work going forward into the next decade, where we envision learning communities and open collaborative work communities coming together as persistent technology supported and enhanced communities of practice.

Do You Think You Can? The Influence of Student Self-Efficacy on the Effectiveness of Tutorial Dialogue for Computer Science

In recent years, significant advances have been made in intelligent tutoring systems, and these advances hold great promise for adaptively supporting computer science (CS) learning. In particular, tutorial dialogue systems that engage students in natural language dialogue can create rich, adaptive interactions. A promising approach to increasing the effectiveness of these systems is to adapt not only to problem-solving performance, but also to a student’s characteristics. Self-efficacy refers to a student’s view of her ability to complete learning objectives and to achieve goals; this characteristic may be particularly influential during tutorial dialogue for computer science education. This article examines a corpus of effective human tutoring for computer science to discover the extent to which considering self-efficacy as measured within a pre-survey, coupled with dialogue and task events during tutoring, improves models that predict the student’s self-reported frustration and learning gains after tutoring. The analysis reveals that students with high and low self-efficacy benefit differently from tutorial dialogue. Student control, social dialogue, and tutor moves to increase efficiency, may be particularly helpful for high self-efficacy students, while for low self-efficacy students, guided experimentation may foster greater learning while at the same time potentially increasing frustration. It is hoped that this line of research will enable tutoring systems for computer science to tailor their tutorial interactions more effectively.

Education: Expectation and the Unexpected

This paper considers concepts of expectation and responsibility, and how these drive dialogic interactions between tutor and student in an age of marketised Higher Education. In thinking about such interactions in terms of different forms of exchange, the paper considers the philosophy of Martin Buber and Emmanuel Levinas on dialogic intersubjectivity, and an ethics of responsibility. This enables a richer understanding of the tutorial dialogue in particular, as both teaching and encounter. This has significant implications for education and for the idea of the tutorial as a space for encounter with the other through language. The paper argues that the university tutorial, rather than being considered a place only for the meeting of expectations, might be envisioned as a space for encounter with the unexpected. In considering the nature of the educational encounter in relation to an example from the film adaptation of Alan Bennett’s ‘The History Boys’, the paper concludes that the tutorial opens up the possibility for a mutual encounter with otherness. This positions the tutorial as a space of educational otherness—a Foucauldian heterotopia which rejects the expectation-bound economy of exchange, and which offers instead the possibility of an education marked by an economy of excess.

Investigating student interactions with tutorial dialogues in EER-Tutor.

Eye-movement tracking and student-system interaction logs provide different types of information which can be used as a potential source of real-time adaptation in learning environments. By analysing student interactions with an intelligent tutoring system (ITS), we can identify sub-optimal behaviour such as not paying attention to important interface components. On the basis of such findings, ITSs can be enhanced to be proactive, rather than reactive, to users’ actions. Tutorial dialogues are one of the teaching strategies used in ITSs which has been shown empirically to significantly improve learning. Enhanced entity-relationship (EER)-Tutor is a constraint-based ITS that teaches conceptual database design. This paper presents the preliminary results of a project that investigates how students interact with the tutorial dialogues in EER-Tutor using both eye-gaze data and student-system interaction logs. Our findings indicate that advanced students are selective of the interface areas they visually focus on, whereas novices waste time by paying attention to interface areas that are inappropriate for the task at hand. Novices are also unaware that they require help with the tutorial dialogues. Furthermore, we have demonstrated that the student’s prior knowledge, the problem complexity and the percentage of the dialogue’s prompts that are answered correctly are factors that can be used to predict future errors. The findings from our study can be used to further enhance EER-Tutor in order to support learning better, including real-time classification of students into novices and advanced students in order to adapt system feedback and interventions.

Applying Clustering Approach to Analyze Reflective Dialogues and Students’ Problem Solving Ability

In the context of one-on-one instruction, reflective dialogues help students advance their learning and improve their problem solving ability. The effectiveness of one-on-one instruction with respect to learning through dialogue is highlighted by researchers and educators. However, little if any, is known about how reflective dialogues may lead to learning improvement and predict students’ problem solving ability. This information can be extracted from large educational datasets using data mining techniques. Consequently, this study aims at mining USNA physics data set applying a two-level clustering approach to find patterns in the data and identify how reflective dialogues predict students’ problem solving ability. The results indicated that reflective group performed better on the hourly exams. Control subjects took lower average count of steps during problem solving activity and the average of duration was longer in control group. Also, higher average of correct answers and average count of attempts was found in reflective condition. Yet, control group had a higher level of incorrect answers as compared to reflective group. During the intervention, reflective subjects had higher level of average count of attempts and lesser average count of deletion. Reflective group asked for less hints, had lesser count of problem and requested for calculator less than control subjects. The results of the analysis help educators plan more effective tutorial dialogues in Intelligent Tutoring Systems (ITSs).

Faculty of Computer Science and Information Systems, University of Malaya

In the context of one-on-one instruction, reflective dialogues help students advance their learning and improve their problem solving ability. The effectiveness of one-on-one instruction with respect to learning through dialogue is highlighted by researchers and educators. However, little if any, is known about how reflective dialogues may lead to learning improvement and predict students’ problem solving ability. This information can be extracted from large educational datasets using data mining techniques. Consequently, this study aims at mining USNA physics data set applying a two-level clustering approach to find patterns in the data and identify how reflective dialogues predict students’ problem solving ability. The results indicated that reflective group performed better on the hourly exams. Control subjects took lower average count of steps during problem solving activity and the average of duration was longer in control group. Also, higher average of correct answers and average count of attempts was found in reflective condition. Yet, control group had a higher level of incorrect answers as compared to reflective group. During the intervention, reflective subjects had higher level of average count of attempts and lesser average count of deletion. Reflective group asked for less hints, had lesser count of problem and requested for calculator less than control subjects. The results of the analysis help educators plan more effective tutorial dialogues in Intelligent Tutoring Systems (ITSs).

Tutorial dialogues and gist explanations of genetic breast cancer risk.

The intelligent tutoring system (ITS) BRCA Gist is a Web-based tutor developed using the Shareable Knowledge Objects (SKO) platform that uses latent semantic analysis to engage women in natural-language dialogues to teach about breast cancer risk. BRCA Gist appears to be the first ITS designed to assist patients' health decision making. Two studies provide fine-grained analyses of the verbal interactions between BRCA Gist and women responding to five questions pertaining to breast cancer and genetic risk. We examined how "gist explanations" generated by participants during natural-language dialogues related to outcomes. Using reliable rubrics, scripts of the participants' verbal interactions with BRCA Gist were rated for content and for the appropriateness of the tutor's responses. Human researchers' scores for the content covered by the participants were strongly correlated with the coverage scores generated by BRCA Gist, indicating that BRCA Gist accurately assesses the extent to which people respond appropriately. In Study 1, participants' performance during the dialogues was consistently associated with learning outcomes about breast cancer risk. Study 2 was a field study with a more diverse population. Participants with an undergraduate degree or less education who were randomly assigned to BRCA Gist scored higher on tests of knowledge than those assigned to the National Cancer Institute website or than a control group. We replicated findings that the more expected content that participants included in their gist explanations, the better they performed on outcome measures. As fuzzy-trace theory suggests, encouraging people to develop and elaborate upon gist explanations appears to improve learning, comprehension, and decision making.

Still to learn from vicarious learning

The term ‘vicarious learning’ was introduced in the 1960s by Bandura, who demonstrated how learning can occur through observing the behaviour of others. Such social learning is effective without the need for the observer to experience feedback directly. More than twenty years later a series of studies on vicarious learning was undertaken in the context of technology-enhanced learning in mass higher education. These studies employed the idea in a rather narrower sense, defining it as learning through observing others in the act of learning, focusing in particular on the kind of conceptual learning that is typical in higher education and is best observed in tutorial settings. The key proposal was that multimedia recorded versions of tutorial dialogues could be chosen for their effectiveness in revealing the way concepts are grasped, questions posed and problems solved, from the perspective of real learners rather than from that imagined by teachers. These recorded dialogues could be linked to the primary subject matter and, it was hoped, would constitute a powerful resource for new students who would otherwise not have the opportunity to benefit from a teaching method which, if restricted to the original participants, simply does not scale. This paper reviews the subsequent research that has continued to explore this idea. Overall, the potential importance of vicarious learning in various kinds of subject matter has been confirmed. Research on vicarious learning continues to resonate with fundamental issues of the design of higher education in a digital world.

Learning by Communicating in Natural Language With Conversational Agents

Learning is facilitated by conversational interactions both with human tutors and with computer agents that simulate human tutoring and ideal pedagogical strategies. In this article, we describe some intelligent tutoring systems (e.g., AutoTutor) in which agents interact with students in natural language while being sensitive to their cognitive and emotional states. These systems include one-on-one tutorial dialogues, conversational trialogues in which two agents (a tutor and a “peer”) interact with a human student, and other conversational ensembles in which agents take on different roles. Tutorial conversations with agents have also been incorporated into educational games. These learning environments have been developed for different populations (elementary through high school students, college students, adults with reading difficulties) and different subjects spanning science, technology, engineering, mathematics, reading, writing, and reasoning. This article identifies some of the conversation patterns that are implemented in the dialogues and trialogues.