Pedagogic Conversational Agents Research Articles

A First Approach to Co-Design a Multimodal Pedagogic Conversational Agent with Pre-Service Teachers to Teach Programming in Primary Education

Pedagogic Conversational Agents (PCAs) are interactive systems that engage the student in a dialogue to teach some domain. They can have the roles of a teacher, student, or companion, and adopt several shapes. In our previous work, a significant increase of students’ performance when learning programming was found when using PCAs in the teacher role. However, it is not common to find PCAs used in classrooms. In this paper, it is explored whether pre-service teachers would accept PCAs to teach programming better if they were co-designed with them. Pre-service teachers are chosen because they are still in training, so they can be taught what PCAs are and how this technology could be helpful. Moreover, pre-service teachers can choose whether they integrate PCAs in the teaching activities that they carry out as part of their degree’s course. An experiment with 35 pre-service primary education teachers was carried out during the 2021/2022 academic year to co-design a robotic PCA to teach programming. The experience validates the idea that involving pre-service teachers in the design of a PCA facilitates their involvement to integrate this technology in their classrooms. In total, 97% of the pre-service teachers that stated in a survey that they believed robot PCA could help children to learn programming, and 80% answered that they would like to use them in their classrooms.

Knowledge-Based and Generative-AI-Driven Pedagogical Conversational Agents: A Comparative Study of Grice’s Cooperative Principles and Trust

The emergence of generative language models (GLMs), such as OpenAI’s ChatGPT, is changing the way we communicate with computers and has a major impact on the educational landscape. While GLMs have great potential to support education, their use is not unproblematic, as they suffer from hallucinations and misinformation. In this paper, we investigate how a very limited amount of domain-specific data, from lecture slides and transcripts, can be used to build knowledge-based and generative educational chatbots. We found that knowledge-based chatbots allow full control over the system’s response but lack the verbosity and flexibility of GLMs. The answers provided by GLMs are more trustworthy and offer greater flexibility, but their correctness cannot be guaranteed. Adapting GLMs to domain-specific data trades flexibility for correctness.

Modeling Synchronization for Detecting Collaborative Learning Process Using a Pedagogical Conversational Agent: Investigation Using Recurrent Indicators of Gaze, Language, and Facial Expression

AbstractConstructive interactions and knowledge integration activities are methods commonly used for learning; however, establishing successful coordination becomes a hurdle in computer-mediated collaborations. The development of systems to facilitate communication activities in such situations has been attempted, but models are still required for capturing learners’ interactions and detecting their quality. This study explored several types of verbal and nonverbal behaviors of learners that can be implemented while designing tutoring systems to effectively capture their interaction processes in scenarios where learners engage in collaborative learning mediated by a pedagogical conversational agent (PCA). This study focused on the degree of behavior recurrence of each speaker, which is considered suitable for observing levels of effectiveness. Specifically, this study focused on three indicators—gaze synchronization, language conformance, and emotional matching through facial expression—to establish a system-based index for measuring learners’ collaborative processes such as synchronization. This study experimentally examined the relationship between these indicators and the performance and process of collaborative learning among 44 learners while using PCA for facilitation. Subsequently, numerous dependent variables in the collaborative learning process were predicted using the three proposed indicators. However, no significant correlation was established between learning performance and the indicators used. These findings show that the recurrence of indicators is useful for estimating the collaborative learning process and that these indicators can be used in the development of learning support systems to trace learners’ achievements in successful interactions.

Empathic pedagogical conversational agents: A systematic literature review

AbstractArtificial intelligence (AI) and natural language processing technologies have fuelled the growth of Pedagogical Conversational Agents (PCAs) with empathic conversational capabilities. However, no systematic literature review has explored the intersection between conversational agents, education and emotion. Therefore, this study aimed to outline the key aspects of designing, implementing and evaluating these agents. The data sources were empirical studies, including peer‐reviewed conference papers and journal articles, and the most recent publications, from the ACM Digital Library, IEEE Xplore, ProQuest, ScienceDirect, Scopus, SpringerLink, Taylor & Francis Online, Web of Science and Wiley Online Library. The remaining papers underwent a rigorous quality assessment. A filter study meeting the objective was based on keywords. Comparative analysis and synthesis of results were used to handle data and combine study outcomes. Out of 1162 search results, 13 studies were selected. The results indicate that agents promote dialogic learning, proficiency in knowledge domains, personalized feedback and empathic abilities as essential design principles. Most implementations employ a quantitative approach, and two variables are used for evaluation. Feedback types play a vital role in achieving positive results in learning performance and student perceptions. The main limitations and gaps are the time range for literature selection, the level of integration of the empathic field and the lack of a detailed development stage report. Moreover, future directions are the ethical implications of agents operating beyond scheduled learning times and the adoption of Responsible AI principles. In conclusion, this review provides a comprehensive framework of empathic PCAs, mostly in their evaluation. The systematic review registration number is osf.io/3xk6a.Practitioner notesWhat is already known about this topic Emotions play a pivotal role in shaping the interaction process, making it essential to consider them when designing methodological strategies or learning tools. Empathic Pedagogical Conversational Agents (PCAs) have emerged as a crucial approach for enhancing and personalizing the learning experience (24/7) for pupils and supporting human teachers in their teaching process. Despite the creation of numerous empathic PCAs, there is a scarcity of Systematic Literature Reviews (SLRs) on their application in the educational field, particularly concerning the integration of emotional abilities in combination with the competencies of each subject. What this paper adds It offers new insights into the design principles underlying the integration of the empathic field. It reviews different approaches for incorporating students' prior knowledge in real time. It provides a comprehensive and up‐to‐date overview of the research designs used for implementation, including quantitative, qualitative and mixed methods. It examines the factors that influence the effectiveness of empathic PCA in teaching and learning. It evaluates the types of feedback that enhance the impact of the empathic field on learning outcomes. Implications for practice and/or policy It is crucial to grasp the topics that this paper introduces in order to effectively integrate new learning tools into any context. Techno‐pedagogical designers seeking to gain insights into empathic PCAs will find immense value in this SLR, as it comprehensively covers each stage of the process. For future research endeavours, this study offers a wealth of ideas to draw upon, enabling researchers to address the challenges outlined and explore new avenues of investigation.

What is the impact of a multi‐modal pedagogical conversational AI system on parents' concerns about technology use by young children?

AbstractPrevious research showed that the parents acknowledged the technology's benefits for their young children's learning, however, they are still worried about the extended screen time, lack of physical activity and lack of social interactions. To address these concerns, we developed Kid Space to enable pedagogically appropriate technology use for children in early childhood education by combining various sensing technologies with a multi‐modal conversational artificial intelligence system that can interact with children, understand individual progress and provide personalised learning experiences. To understand the impact of Kid Space on the parents' initial concerns about technology use by their young children, we conducted a multi‐method user study: (1) a quasi‐experimental design and (2) formative research method using an exploratory case study with a set of children and their parents experiencing Kid Space in their homes. The results show that after experiencing Kid Space with their children, the parents felt significantly less concerned about screen time, social interactions and physical activity and reported positive perceptions towards pedagogical value of Kid Space. Detailed analysis on the multi‐modal data quantitatively and qualitatively validated why Kid Space alleviated these concerns. Future research is needed to validate long‐term educational value of Kid Space and generate insights for improvement for next iterations. Practitioner notesWhat is already known about this topic Play‐based learning is critical for young children's education, but digital games create major concerns around extended screen time, lack of physical activity and lack of social interactions. Blending digital and physical spaces could support pedagogically appropriate technology use for young children. Towards this end, there are some exemplary studies in the state–of‐the‐art reporting positive educational outcomes as an effect of utilising such spaces. However, none of these studies supported children's most natural mode of communication in their interactions with the systems—speaking. Pedagogical conversational agents (PCAs) are promising, but they are tricky when it comes to young children's speech because of unique technical challenges resulted from how children use language and communicate with digital systems. What this paper adds To our best knowledge, Kid Space is one of the earliest implementations of a PCA with a multi‐modal artificial intelligence (AI) system utilising physical and digital learning manipulatives for maths learning with a focus on early childhood education. The key contributions of this paper are (1) the design and development of an end‐to‐end multi‐modal system enabling Wizard‐of‐Oz experimentation for initial evaluations with users, (2) the creation of a multi‐modal, in‐the‐wild labelled dataset with children–agent, children–parent and children–physical/digital space interactions enabling advancements for AI components for later evaluations with users and (3) the generation of rich insights from an initial research study on user perceptions and engagement as well as actionable findings to improve Kid Space experiences for next iterations and inform key design features for similar systems. Implications for practice and/or policy The results of the study implied a set of areas for improvement—or design features—for Kid Space and other similar pedagogical conversational systems developed for children's home usages: (1) easier setup and usage with optimised setup size addressing diverse space limitations at homes, (2) minimised latency between Oscar (the conversational pedagogical agent) and child interactions (eg, adding multimodal dialogue system to reduce the need for a human wizard), (3) more advanced personalisation, social (including more verbal interactions) and pedagogical skills for Oscar with increased contextual awareness (eg, sending children's engagement), (4) scalability and higher visual quality of content with diverse games and learning outcomes, (5) parental control features over Kid Space platform and Oscar (eg, time limit, content, etc.) and (6) accessibility features (eg, captions turned on for multilingual children) and support for neurodiversity.

Game-inspired Pedagogical Conversational Agents: A Systematic Literature Review

Game-inspired Pedagogical Conversational Agents: A Systematic Literature Review

On the Experimental Process in Evaluations of Brazilian Conversational Agents in Education

Conversational agents represent applications that interact in a natural language with humans. Due to their ability to process the human language, they have been explored in diverse areas, e.g., e-commerce, health, entertainment, and education. Particularly in the education field, they have been used as mechanisms that support the teaching of a second language, that recommend educational resources, and that resolve students’ doubts, as a learning partner. Brazil is particularly interested in the educational technology community studying such software systems. Despite the strong interests in their development, few studies have addressed the impact such systems exert when they are used by students. Experimental studies have enabled a better understanding of the impact of technology on education based on evidence. This article presents an overview of empirical research on conversational agents in education through a mapping of the Brazilian community, identifying the way researchers systematically evaluate such agents produced in their research work. The results show a lack of experimental methodologies and taxonomies that support the planning, development, and reporting of experimental studies. However, a set of variables commonly employed has been recognized. An initial set of guidelines has been developed towards guiding experiments in this context and supporting the systematization of studies on experimental evaluations of pedagogical conversational agents.

A Review of the Practical Applications of Pedagogic Conversational Agents to Be Used in School and University Classrooms

Pedagogic Conversational Agents (PCAs) can be defined as autonomous characters that cohabit learning environments with students to create rich learning interactions. Currently, there are many agents reported in the literature of this fast-evolving field. In this paper, several designs of PCAs used as instructors, students, or companions are reviewed using a taxonomy to analyze the possibilities that PCAs can bring into the classrooms. Finally, a discussion as to whether this technology could become the future of education depending on the design trends identified is open for any educational technology practitioner, researcher, teacher, or manager involved in 21st century education.

Gaze awareness and metacognitive suggestions by a pedagogical conversational agent: an experimental investigation on interventions to support collaborative learning process and performance

Research on collaborative learning has revealed that peer-collaboration explanation activities facilitate reflection and metacognition and that establishing common ground and successful coordination are keys to realizing effective knowledge-sharing in collaborative learning tasks. Studies on computer-supported collaborative learning have investigated how awareness tools can facilitate coordination within a group and how the use of external facilitation scripts can elicit elaborated knowledge during collaboration. However, the separate and joint effects of these tools on the nature of the collaborative process and performance have rarely been investigated. This study investigates how two facilitation methods—coordination support via learner gaze-awareness feedback and metacognitive suggestion provision via a pedagogical conversational agent (PCA)—are able to enhance the learning process and learning gains. Eighty participants, organized into dyads, were enrolled in a 2 × 2 between-subject study. The first and second factors were the presence of real-time gaze feedback (no vs. visible gaze) and that of a suggestion-providing PCA (no vs. visible agent), respectively. Two evaluation methods were used: namely, dialog analysis of the collaborative process and evaluation of learning gains. The real-time gaze feedback and PCA suggestions facilitated the coordination process, while gaze was relatively more effective in improving the learning gains. Learners in the Gaze-feedback condition achieved superior learning gains upon receiving PCA suggestions. A successful coordination/high learning performance correlation was noted solely for learners receiving visible gaze feedback and PCA suggestions simultaneously (visible gaze/visible agent). This finding has the potential to yield improved collaborative processes and learning gains through integration of these two methods as well as contributing towards design principles for collaborative-learning support systems more generally.

Predicting self-condence in a web-based explanation activity with a pedagogical conversational agent: Investigation on individual characteristics and task work activity

Predicting self-condence in a web-based explanation activity with a pedagogical conversational agent: Investigation on individual characteristics and task work activity

Multiple pedagogical conversational agents to support learner-learner collaborative learning: Effects of splitting suggestion types

This study experimentally investigated the design of effective interactions using pedagogical conversational agents (PCAs) in a learner-learner collaborative learning activity. While dyads engaged in a concept explanation task (explaining the mechanism of computer processing), PCAs served as facilitators and provided metacognitive suggestions to better improve learning performance. Previous studies have shown that learners who received several types of suggestions from multiple PCAs were motivated to produce effective explanations; this study then further explored the effects of using multiple PCAs in different roles, providing different types of facilitation. It was predicted that by using two different PCAs to offer suggestions with a delay, learners may be able to process information more efficiently, for example by paying closer attention to each type of suggestion. To investigate this possibility, two types of facilitating content, namely provision of metacognitive suggestions and advice on effective coordination, were each implemented into two role-playing PCAs, named the “explanation adviser” and the “communication adviser” respectively. The results show that when learners used PCAs playing different roles and offering suggestions corresponding to these roles, learners generated explanations related to the suggestions and improved performance (efficacy of explanations) in several areas, including learning performance, for example better understanding the concept and becoming able to explain it using a greater range of technical words. This study shows empirically how multiple PCAs can be effectively designed to implement roles yielding different types of suggestions. The advantages of using such methods and implementing such functions of PCAs are further discussed.

A Procedure to Engage Children in the Morphological and Syntax Analysis of Pedagogic Conversational Agent-Generated Sentences to Study Language

Pedagogic Conversational Agents are computer applications that interact with the students in natural language. They usually focus the dialogue on a certain topic under study. In this paper, the authors propose the possibility of children studying morphology and syntax by using a Pedagogic Conversational Agent. The main benefit is that the agent is able to generate an infinite number of sentences and, it automatically generates the morphological and syntactical analysis from a given grammar. That way, students can practise with all the sentences they need, receive immediate feedback with automatic evaluation, at their own rhythm and, the level of difficulty can be adapted to their particular competence of analysis. Given the originality of this new computer assisted learning initiative, the authors have devised a procedure to engage the students in the dialogue with the agent to carry out the morphological and syntax analysis at five different levels of difficulty, and test the validity of the approach with a limited number of users according to the principles of User-Centered Design. The results gathered provide evidence of the goodness of the procedure and, they encourage us to keep working on this promising field of using pedagogic agents as computer teaching language assistants.

Developing multimodal conversational agents for an enhanced e-learning experience

Conversational agents have become a strong alternative to enhance educational systems with intelligent communicative capabilities, provide motivation and engagement, and increment significant learning and helping in the acquisition of meta-cognitive skills. In this paper, we present Geranium, a multimodal conversational agent that helps children to appreciate and protect their environment. The system, which integrates an interactive chatbot, has been developed by means of a modular and scalable framework that eases building pedagogic conversational agents that can interact with the students using speech and natural language.

A Procedure to Create a Pedagogic Conversational Agent in Secondary Physics and Chemistry Education

Pedagogic Conversational Agents are computer applications that can interact with students in natural language. They have been used with satisfactory results on the instruction of several domains. The authors believe that they could also be useful for the instruction of Secondary Physics and Chemistry Education. Therefore, in this paper, the authors present a procedure to create an agent for that domain. First, teachers have to introduce the exercises with their correct answers. Secondly, students will be presented the exercises, and if the students know the answer, and if it is correct, more difficult exercises will be presented. Otherwise, step-by-step natural language support will be provided to guide the student towards the solution. It is the authors’ hypothesis that this innovative teaching method will be satisfactory and useful for teachers and students, and that by following the procedure more computer programmers can be encouraged to develop agents for other domains to be used by teachers and students at class.

An exploratory study on how children interact with pedagogic conversational agents

A pedagogic conversational agent (PCA) can be defined as a computer system that interacts with the student in natural language assuming the role of the instructor, a student or a companion. It can have a personality and can generate different sentences according to the agent or the student mood. Empathy with the students' feelings seems to increase their motivation to study. However, the influence of the agent personality and role as well as the students' opinion is still unclear. Therefore, in this article, it is explored with the help of a field experiment, for the first time, how these factors can affect the interaction of children with PCAs, and their opinions according to an anonymous and voluntary opinion questionnaire and some personal interviews.

Análisis de la experiencia de uso de un agente de comprensión lectora con niños en edad escolar

El papel de las tecnologías en la sociedad es fundamental y en continuo crecimiento. En el contexto educativo, se pueden identificar en las últimas décadas un incremento de nuevas tecnologías educativas, entre las que se encuentran el uso de Agentes Conversacionales Pedagógicos. Esto es, sistemas informáticos que interaccionan con los estudiantes en lenguaje natural, hablado o escrito, y que pueden adoptar el rol de profesor, estudiante o bien compañero de estudios. En el curso 2011/2012, aplicamos el Agente Conversacional Pedagógico llamado Dr. Roland, para resolver problemas de matemáticas, en un aula con 38 estudiantes de entre 12-13 años de edad y 3 profesores. En este artículo, se presenta el análisis de la experiencia y cómo se pueden integrar estas nuevas tecnologías en el aula. También se proporcionan, a partir del análisis realizado, diez recomendaciones que se proponen como guía para docentes, investigadores y profesionales interesados en usar este tipo de agentes. Algunas de estas recomendaciones son combinar el uso del ordenador, con otros dispositivos como tabletas que pueden facilitar el manejo del sistema por varios estudiantes, incluir a los padres para fomentar el uso del agente desde casa, y validar cada uno de los avances realizados con los profesores, padres y estudiantes para adaptarse a sus necesidades.

Using Conceptual Models to Implement Natural Language Pedagogic Agent-Student Conversations

The last several decades have seen a growing trend in incorporating pedagogic conversational agents in interactive learning environments. Software systems have increasingly integrated intelligent virtual agents that can interact with students in natural language to fulfill specific tasks such as reviewing content or providing tutor training. The use of an agent-based approach in education has shown many benefits. However, certain design and development issues are still unresolved. This article focuses on the potentials of employing conceptual models to generate agent-student dialog and introduces a new mixed-initiative general domain agent called JARO. The authors report on the procedure for creating the initial conceptual model and discuss its use in guiding agent-student conversations adapted to students' individual learning needs. The stages of implementation of the model as well as the model's viability tested in a proof-of-concept experiment are addressed.

Social Attitude Recognition in Multimodal Interaction with a Pedagogical Agent

Conversational agents have been widely used in pedagogical contexts. They have the advantage of offering to users not only a task-oriented support, but also the possibility to relate with the system at social level. Therefore, besides endowing the conversational agent with knowledge necessary to fulfill pedagogical goals, it is important to provide the agent with social intelligence. To do so the agent should be able to recognize the social attitude of the user during the interaction in order to accommodate the conversational strategy. In this paper we illustrate how we defined and applied a model for recognizing the social attitude of the student in natural interaction with a Pedagogical Conversational Agent (PCA) starting from the linguistic, acoustic and gestural analysis of the communicative act.