Dialogue Module Research Articles

An e-learning platform for clinical reasoning in cardiovascular diseases: a study reporting on learner and tutor satisfaction

BackgroundMedical students gain essential skills through hospital training and internships, which complement their theoretical education. However, virtual patient platforms have been shown to effectively promote clinical reasoning and enhance learning outcomes. This study evaluates a web-based platform designed for learning clinical reasoning in cardiovascular diseases, detailing its functionalities and user satisfaction.MethodsThe Virtual Patient platform presents medical students with clinically valid scenarios, encompassing stages such as patient description, anamnesis, objective examination, presumptive diagnosis, health investigations, treatment planning, complications, differential and final diagnoses, and prognosis. Scenarios are generated either automatically or manually by professors, based on labeled and annotated clinical data. The Virtual Patient contains two types of medical cases: simple scenarios describing patients with one pathology, and complex scenarios describing patients with several related pathologies. The platform was evaluated by a total of 210 users: 178 medical students, 7 professors, and 25 engineering students, using questionnaires adjusted for each evaluation round to assess satisfaction and gather feedback. The evaluation by medical students was performed in four rounds, each round corresponding to successive enhancements of the platform functionalities and addition of new cases, with a total number of 1,098 evaluation sessions.ResultsThe platform was evaluated at different implementation stages, involving simple and complex scenarios for various heart diseases. The majority of students found the platform very useful (82.58%), with significant appreciation for its features and functionalities, for example the dialogue module supporting natural language interactions in Romanian and English or the feed-back obtained during interaction. Professors highly valued the platform’s flexibility in scenario generation, real-time feedback provision, and data management capabilities. They appreciated the possibility to provide feedback and score student performance in real-time or after the session, though some professors suggested improving the explainability of the scores.ConclusionsThe Virtual Patient platform enables medical students to virtually replicate hospital interactions, diagnose patients, and plan treatments in clinically valid scenarios for cardiovascular diseases. User evaluations demonstrated high satisfaction and appreciation for the platform’s features. Future work will focus on expanding medical cases, enhancing the dialogue module, improving scenario generation for complex cases, and extending the synthetic data generation component to produce additional types of medical investigations.

Evaluating Recent Advances in Affective Intelligent Tutoring Systems: A Scoping Review of Educational Impacts and Future Prospects

Affective intelligent tutoring systems (ATSs) are gaining recognition for their role in personalized learning through adaptive automated education based on students’ affective states. This scoping review evaluates recent advancements and the educational impact of ATSs, following PRISMA guidelines for article selection and analysis. A structured search of the Web of Science (WoS) and Scopus databases resulted in 30 studies covering 27 distinct ATSs. These studies assess the effectiveness of ATSs in meeting learners’ emotional and cognitive needs. This review examines the technical and pedagogical aspects of ATSs, focusing on how emotional recognition technologies are used to customize educational content and feedback, enhancing learning experiences. The primary characteristics of the selected studies are described, emphasizing key technical features and their implications for educational outcomes. The discussion highlights the importance of emotional intelligence in educational environments and the potential of ATSs to improve learning processes. This review identifies gaps in the current research and suggests future directions, including broader implementation across diverse educational settings and deeper integration of affective data to refine system responsiveness. Future research should investigate the integration of advanced natural dialogue modules and generative AI to create more sophisticated interfaces, underscoring the role of affective adaptation in educational technology.

Intelligent Customer Service System Optimization Based on Artificial Intelligence

To elevate the intelligence of customer service dialogue systems, this article proposes an intelligent customer service system comprising chat dialogue subsystems, task-oriented multi-turn dialogue subsystems, single-turn dialogue subsystems, and an integration model. Firstly, to enhance diversity of responses and improve user experience, particularly in casual chat scenarios, this article presents a Seq2Seq-based approach for multi-answer responses, allowing for more expressive emotional expression in responses. Secondly, to address situations where customers cannot articulate their needs in a single sentence during multi-turn dialogues, this article designs a task-oriented multi-turn dialogue module. It employs intent recognition and slot filling to maintain contextual information throughout the conversation, aiding customers in problem resolution. Lastly, to overcome the current limitation of intelligent customer service models providing relatively one-dimensional answers in specific domains.

Enhancing Sustainable Urban Landscapes through AI-Driven Low-Carbon Plant Selection: A Novel Approach

In the face of global environmental challenges, sustainable urban landscapes play a pivotal role in mitigating carbon emissions and fostering healthier cities. This paper presents a groundbreaking approach to low-carbon plant selection for landscape architecture, leveraging AI technology to address the limitations of existing language models in the field. Through the integration of professional databases, the introduction of judgment modules, low-carbon plant landscape recommendation modules, general dialogue modules, the use of prompt engineering technology, fine-tuning technology, our system offers precise, ecologically sound recommendations for low-carbon landscape designs.

Adaptation of a robotic dialog system for medication reminder in elderly care

Social robots can assist older adults in their daily life. Verbal conversation is a natural and convenient way for older adults to interact with social robots. However, most of the existing conversation-based robot services, such as medication reminders, are rule-based systems. These systems require many hand-crafted rules and a significant amount of expert knowledge, therefore they cannot adapt to older adults’ characteristics and dialog history. There are many reinforcement learning (RL) based methods for task-oriented dialogues, but they mainly focus on completing the tasks through text-based conversations. Those methods cannot be directly used for elderly care applications involving human–robot interactions (HRI). Considering the above shortcomings, we proposed a dialog system adaptation method (DSAM) for social robots. The DSAM is based on reinforcement learning which considers the characteristics of older adults, the dialog history and user preference to adapt the dialog policy and improve the dialog module. We implemented DSAM in our custom-made ASCCBot social robot. To evaluate DSAM, we firstly tested the dialog agent which was trained by a user simulator with different settings. The results show that the obtained agent achieves a good result with the desired dialog flow compared to the baseline agent. Based on the obtained dialog policy, the adaptation process is evaluated. The results show that with a good success rate, the number of dialog turns is decreased and the NLU module performance is improved by the adaptation process, which proves the effectiveness of DSAM. We also tested DSAM with human subjects. The results show that the average adaptation success rate is 94.7% and the preference distance reaches 0 after 6 rounds of adaptation while creating reminders successfully with a limited amount of user feedback. • A robotic dialog adaptation system for medication reminder in elderly care. • Reinforcement learning based methods for task-oriented dialogues. • Characteristics of older adults, dialog history and user preference considered in adaptation. • User simulator and user modeling combined to adapt the dialog agent.

A semantic relation-aware deep neural network model for end-to-end conversational recommendation

Conversational recommendation system (CRS) aims at recommending appropriate items to the user through a multi-turn conversation. The end-to-end CRS is a type of CRS that models the recommendation task and the conversation task simultaneously which has attracted more and more attention in recent years. At the same time, knowledge graph and Transformer are incorporated into the end-to-end CRS to generate better recommendations and better responses to the user, which makes the CRS have state-of-the-art performance. It is known that there exist semantic relations in a conversation. However, we observe that existing end-to-end CRSs in general ignore the semantic relations in the conversation and therefore would likely hinder the performance of CRSs. Motivated by this, we propose a gated cross- and self-attention based CRS utilizing semantic relation information (ASR) model, which can explicitly model and utilize the semantic relations in a conversation. To the best of our knowledge, we are the first to advocate for modelling and utilizing the semantic relations in the end-to-end CRS, which could help to improve the performance of the CRS. Furthermore, to mitigate the class-imbalance problem that most end-to-end CRSs face, we propose a new negative sampling method which could make the proposed CRS learn better. Moreover, we design a Transformer-based dialogue module integrating the semantic relations in a conversation to generate more diversified and precise responses. Extensive experiments on widely used benchmark datasets demonstrate that the proposed ASR model achieves state-of-the-art results in both recommendation and conversation tasks.

Self-Supervised learning for Conversational Recommendation

Conversational recommender system (CRS) aims to model user preference through interactive conversations. Although there are some works, they still have two drawbacks: (1) they rely on large amounts of training data and suffer from data sparsity problem; and (2) they do not fully leverage different types of knowledge extracted from dialogues. To address these issues in CRS, we explore the intrinsic correlations of different types of knowledge by self-supervised learning, and propose the model SSCR, which stands for Self-Supervised learning for Conversational Recommendation. The main idea is to jointly consider both the semantic and structural knowledge via three self-supervision signals in both recommendation and dialogue modules. First, we carefully design two auxiliary self-supervised objectives: token-level task and sentence-level task, to explore the semantic knowledge. Then, we extract the structural knowledge based on external knowledge graphs from user mentioned entities. Finally, we model the inter-information between the semantic and structural knowledge with the advantages of contrastive learning. As existing similarity functions fail to achieve this goal, we propose a novel similarity function based on negative log-likelihood loss. Comprehensive experimental results on two real-world CRS datasets (including both English and Chinese with about 10,000 dialogues) show the superiority of our proposed method. Concretely, in recommendation, SSCR gets an improvement about 5%∼15% compared with state-of-the-art baselines on hit rate, mean reciprocal rank and normalized discounted cumulative gain. In dialogue generation, SSCR outperforms baselines on both automatic evaluations (distinct n-gram, BLEU and perplexity) and human evaluations (fluency and informativeness).

Soloist: BuildingTask Bots at Scale with Transfer Learning and Machine Teaching

AbstractWe present a new method, Soloist,1 that uses transfer learning and machine teaching to build task bots at scale. We parameterize classical modular task-oriented dialog systems using a Transformer-based auto-regressive language model, which subsumes different dialog modules into a single neural model. We pre-train, on heterogeneous dialog corpora, a task-grounded response generation model, which can generate dialog responses grounded in user goals and real-world knowledge for task completion. The pre-trained model can be efficiently adapted to accomplish new tasks with a handful of task-specific dialogs via machine teaching, where training samples are generated by human teachers interacting with the system. Experiments show that (i)Soloist creates new state-of-the-art on well-studied task-oriented dialog benchmarks, including CamRest676 and MultiWOZ; (ii) in the few-shot fine-tuning settings, Soloist significantly outperforms existing methods; and (iii) the use of machine teaching substantially reduces the labeling cost of fine-tuning. The pre-trained models and codes are available at https://aka.ms/soloist.

Learning Dual Encoding Model for Adaptive Visual Understanding in Visual Dialogue.

Different from Visual Question Answering task that requires to answer only one question about an image, Visual Dialogue task involves multiple rounds of dialogues which cover a broad range of visual content that could be related to any objects, relationships or high-level semantics. Thus one of the key challenges in Visual Dialogue task is to learn a more comprehensive and semantic-rich image representation that can adaptively attend to the visual content referred by variant questions. In this paper, we first propose a novel scheme to depict an image from both visual and semantic views. Specifically, the visual view aims to capture the appearance-level information in an image, including objects and their visual relationships, while the semantic view enables the agent to understand high-level visual semantics from the whole image to the local regions. Furthermore, on top of such dual-view image representations, we propose a Dual Encoding Visual Dialogue (DualVD) module, which is able to adaptively select question-relevant information from the visual and semantic views in a hierarchical mode. To demonstrate the effectiveness of DualVD, we propose two novel visual dialogue models by applying it to the Late Fusion framework and Memory Network framework. The proposed models achieve state-of-the-art results on three benchmark datasets. A critical advantage of the DualVD module lies in its interpretability. We can analyze which modality (visual or semantic) has more contribution in answering the current question by explicitly visualizing the gate values. It gives us insights in understanding of information selection mode in the Visual Dialogue task. The code is available at https://github.com/JXZe/Learning_DualVD.

MOSS: End-to-End Dialog System Framework with Modular Supervision

A major bottleneck in training end-to-end task-oriented dialog system is the lack of data. To utilize limited training data more efficiently, we propose Modular Supervision Network (MOSS), an encoder-decoder training framework that could incorporate supervision from various intermediate dialog system modules including natural language understanding, dialog state tracking, dialog policy learning and natural language generation. With only 60% of the training data, MOSS-all (i.e., MOSS with supervision from all four dialog modules) outperforms state-of-the-art models on CamRest676. Moreover, introducing modular supervision has even bigger benefits when the dialog task has a more complex dialog state and action space. With only 40% of the training data, MOSS-all outperforms the state-of-the-art model on a complex laptop network trouble shooting dataset, LaptopNetwork, that we introduced. LaptopNetwork consists of conversations between real customers and customer service agents in Chinese. Moreover, MOSS framework can accommodate dialogs that have supervision from different dialog modules at both framework level and model level. Therefore, MOSS is extremely flexible to update in real-world deployment.

Module for Dialog Management in the Interaction System Between User and Mobile Robotic Guide

The paper presents dialogue management module for a mobile robot as a guide. The model of the dialog management is presented as a network of transitions between states depending on two factors: visual and communicative. Details of the dialogue management module are presented.

SYSTEM AND METHODS FOR DELIVERING ADVANCED NATURAL LANGUAGE INTERACTION APPLICATIONS

A system for delivering advanced natural language interaction applications, comprising a dialog interface module, a natural language interaction engine, a solution data repository component operating comprising at least one domain model, at least one language model, and a plurality of flow elements and rules for managing interactions with users, and an interface software module. Upon receipt of a request from a user via a network, the dialog interface module preprocesses the request and transmits it to the natural language interaction engine. The natural language interaction engine interprets the request using a plurality of language recognition rules stored in the solution data repository, and based at least determined semantic meaning or user intent, the natural language interaction engine forms an appropriate response and delivers the response to the user via the dialog module, or takes an appropriate action based on the request.

Humanoid Robot Interactions by Motion Description Language and Dialogue

This paper describes the development of a system for facilitating humanoid robots to acquire sequence of movements by motion description and dialogue. The unique feature of our system is that we define a descriptive language and framework that allows the specification of humanoid motions and the representation of motor skills. We subsequently developed and tested a virtual reality simulation system that can execute the motion descriptive language. We then extended the system to allow humanoid robots to acquire motion descriptions through conversing with human. We implemented and tested the proposed system on the NAO humanoid robot. Our implementation includes a dialogue module that continues to update the encoded motion descriptions as the conversation advances and an action module that retrieves the acquired motion descriptions and direct the robot to perform the specified tasks.

A Single-Chip Speech Dialogue Module and Its Evaluation on a Personal Robot, PaPeRo-Mini

This paper presents a single-chip speech dialogue module and its evaluation on a personal robot. This module is implemented on an application processor that was developed primarily for mobile phones to provide a compact size, low power-consumption, and low cost. It performs speech recognition with preprocessing functions such as direction-of-arrival (DOA) estimation, noise cancellation, beamforming with an array of microphones, and echo cancellation. Text-to-speech (TTS) conversion is also equipped with. Evaluation results obtained on a new personal robot, PaPeRo-mini, which is a scale-down version of PaPeRo, demonstrate an 85% correct rate in DOA estimation, and as much as 54% and 30% higher speech recognition rates in noisy environments and during robot utterances, respectively. These results are shown to be comparable to those obtained by PaPeRo.

Using Utterance and Semantic Level Confidence for Interactive Spoken Dialog Clarification

Spoken dialog tasks incur many errors including speech recognition errors, understanding errors, and even dialog management errors. These errors create a big gap between the user's intention and the system's understanding, which eventually results in a misinterpretation. To fill in the gap, people in human-to-human dialogs try to clarify the major causes of the misunderstanding to selectively correct them. This paper presents a method of clarification techniques to human-to-machine spoken dialog systems. We viewed the clarification dialog as a two-step problem-Belief confirmation and Clarification strategy establishment. To confirm the belief, we organized the clarification process into three systematic phases. In the belief confirmation phase, we consider the overall dialog system's processes including speech recognition, language understanding and semantic slot and value pairs for clarification dialog management. A clarification expert is developed for establishing clarification dialog strategy. In addition, we proposed a new design of plugging clarification dialog module in a given expert based dialog system. The experiment results demonstrate that the error verifiers effectively catch the word and utterance-level semantic errors and the clarification experts actually increase the dialog success rate and the dialog efficiency.

Optimization tool for short-term production planning and scheduling

A decision support system for short-term production planning and scheduling is presented. The system consists of three modules: an expert system module, an optimization module and a dialog module. The rules defined in the expert module are used to reduce the space searched by the multicriteria genetic algorithm implemented in the optimization module. A set of experiments showed that reducing the search space increases the efficiency of the applied heuristics. Finally, the dialog module supports the decision maker in choosing the best compromise solution from the set of potentially Pareto-optimal solutions generated by the optimization module.

Information Seeking Spoken Dialogue Systems— Part II: Multimodal Dialogue

For pt.1see ibid., vol. 9, p. 3 (2007). In this paper, the task and user interface modules of a multimodal dialogue system development platform are presented. The main goal of this work is to provide a simple, application-independent solution to the problem of multimodal dialogue design for information seeking applications. The proposed system architecture clearly separates the task and interface components of the system. A task manager is designed and implemented that consists of two main submodules: the electronic form module that handles the list of attributes that have to be instantiated by the user, and the agenda module that contains the sequence of user and system tasks. Both the electronic forms and the agenda can be dynamically updated by the user. Next a spoken dialogue module is designed that implements the speech interface for the task manager. The dialogue manager can handle complex error correction and clarification user input, building on the semantics and pragmatic modules presented in Part I of this paper. The spoken dialogue system is evaluated for a travel reservation task of the DARPA Communicator research program and shown to yield over 90% task completion and good performance for both objective and subjective evaluation metrics. Finally, a multimodal dialogue system which combines graphical and speech interfaces, is designed, implemented and evaluated. Minor modifications to the unimodal semantic and pragmatic modules were required to build the multimodal system. It is shown that the multimodal system significantly outperforms the unimodal speech-only system both in terms of efficiency (task success and time to completion) and user satisfaction for a travel reservation task

Presence and interaction in mixed reality environments

In this paper, we present a simple and robust mixed reality (MR) framework that allows for real-time interaction with virtual humans in mixed reality environments under consistent illumination. We will look at three crucial parts of this system: interaction, animation and global illumination of virtual humans for an integrated and enhanced presence. The interaction system comprises of a dialogue module, which is interfaced with a speech recognition and synthesis system. Next to speech output, the dialogue system generates face and body motions, which are in turn managed by the virtual human animation layer. Our fast animation engine can handle various types of motions, such as normal key-frame animations, or motions that are generated on-the-fly by adapting previously recorded clips. Real-time idle motions are an example of the latter category. All these different motions are generated and blended on-line, resulting in a flexible and realistic animation. Our robust rendering method operates in accordance with the previous animation layer, based on an extended for virtual humans precomputed radiance transfer (PRT) illumination model, resulting in a realistic rendition of such interactive virtual characters in mixed reality environments. Finally, we present a scenario that illustrates the interplay and application of our methods, glued under a unique framework for presence and interaction in MR.

지식진화형 지능공작기계-Part 1: M2M 환경에서의 Agent 표준 플랫폼 기반 Dialogue Module 설계

For the effective operation of manufacturing system, FMS(Flexible Manufacturing System) and CIM(Computer Integrated Manufacturing) system are developed. In these systems, a machine tool is the target of integration in last 3 decades. In nowadays, the conventional concept of machine tools is changing to the autonomous manufacturing device based on knowledge-evolution through applying advanced information technology in which open architecture controller, high speed network and internet technology are contained. In this environment, a machine tool is not the target of integration but the subject of cooperation. In the future, a machine tool will be more improved in the form of a knowledge-evolution based device. In order to develop the knowledge-evolution based machine tools, this paper proposes the structure of knowledge evolution in M2M(Machine To Machine) and the scheme of a dialogue agent among agent-based modules such as a sensory module, a dialogue module, and an expert system. The dialogue agent has a role of interfacing with another machine for cooperation. To design the dialogue agent module in M2M environment, FIPA-OS and ping agent based on FIPA-OS are analyzed in this study. Through this, it is expected that the dialogue agent module can be more efficiently designed and the knowledge-evolution based machine tools can be hereafter more easily implemented.

Homer II—man–machine interface to internet for blind and visually impaired people

HOMER II is a voice-driven text-to-speech system developed for blind or visually impaired persons for reading Slovenian texts. Users can obtain texts from the Internet site of the Association of Slovenian Blind and Visually Impaired Persons Societies from their Electronic Information System where they can find daily newspapers, some novels and other information. The system consists of four main modules. The first module enables Internet communication, retrieves text to a local disc and converts it to a standard form. The input interface manages the keyboard entry and/or speaker independent speech recognition. The output interface performs speech synthesis of a given text and in addition prints the same text magnified to the screen. The user dialog is responsible for the user friendly communication and controls other tasks of the system. Homer II was ported from Linux to the MS Windows 9x/ME/NT/2000 operating systems. For the best performance it uses multi-threading and other advantages of the 32-bit environment. Further versions of the HOMER system with even more advanced dialogue modules and some basic World Wide Web browsing functionality will represent an important tool in the distance learning and teaching process for the impaired persons using academic networks.