Dialogue Systems Research Articles

FinKENet: A Novel Financial Knowledge Enhanced Network for Financial Question Matching.

Question matching is the fundamental task in retrieval-based dialogue systems which assesses the similarity between Query and Question. Unfortunately, existing methods focus on improving the accuracy of text similarity in the general domain, without adaptation to the financial domain. Financial question matching has two critical issues: (1) How to accurately model the contextual representation of a financial sentence? (2) How to accurately represent financial key phrases in an utterance? To address these issues, this paper proposes a novel Financial Knowledge Enhanced Network (FinKENet) that significantly injects financial knowledge into contextual text. Specifically, we propose a multi-level encoder to extract both sentence-level features and financial phrase-level features, which can more accurately represent sentences and financial phrases. Furthermore, we propose a financial co-attention adapter to combine sentence features and financial keyword features. Finally, we design a multi-level similarity decoder to calculate the similarity between queries and questions. In addition, a cross-entropy-based loss function is presented for model optimization. Experimental results demonstrate the effectiveness of the proposed method on the Ant Financial question matching dataset. In particular, the Recall score improves from 73.21% to 74.90% (1.69% absolute).

Beyond Words: An Intelligent Human-Machine Dialogue System with Multimodal Generation and Emotional Comprehension

Intelligent service robots have become an indispensable aspect of modern-day society, playing a crucial role in various domains ranging from healthcare to hospitality. Among these robotic systems, human-machine dialogue systems are particularly noteworthy as they deliver both auditory and visual services to users, effectively bridging the communication gap between humans and machines. Despite their utility, the majority of existing approaches to these systems primarily concentrate on augmenting the logical coherence of the system’s responses, inadvertently neglecting the significance of user emotions in shaping a comprehensive communication experience. To tackle this shortcoming, we propose the development of an innovative human-machine dialogue system that is both intelligent and emotionally sensitive, employing multimodal generation techniques. This system is architecturally comprised of three components: (1) data collection and processing, responsible for gathering and preparing relevant information, (2) a dialogue engine, which generates contextually appropriate responses, and (3) an interaction module, responsible for facilitating the communication interface between users and the system. To validate our proposed approach, we have constructed a prototype system and conducted an evaluation of the performance of the core dialogue engine by utilizing an open dataset. The results of our study indicate that our system demonstrates a remarkable level of multimodal generation response, ultimately offering a more human-like dialogue experience.

A review of dialogue systems: current trends and future directions

A review of dialogue systems: current trends and future directions

Improving user's sense of participation in robot-driven dialogue

ABSTRACT In this study, we aimed to enhance the sense of dialogue participation (SDP), which reflects an individual's conscious engagement in dialogues, and investigated the multimodal dialogue elements that contribute to its improvement. To achieve this, we developed a dialogue system utilizing an android robot specifically for tour spot recommendations at a travel agency. The system was designed with various components aimed at enhancing SDP and was evaluated by 29 participants during the Dialogue Robot Competition 2022. After conceptually defining SDP, we employed the author's subjective evaluation to visualize the time-series changes in SDP at approximately 20-second intervals, using video recordings of the dialogues. Initially, we expected an overall decrease in SDP, as the robot had a speaking duration of around two minutes during the middle of the dialogue session when presenting the tour plan. However, the analysis of five participants' data revealed an increase in SDP during segments that involved ‘Dealing with content that increases the user's interest’ and ‘Multimodal response requests.’ Consequently, we can conclude that these elements have a discernible impact on enhancing SDP.

Conversational Breakdown Detector for a Motivational Interviewing Conversational Agent

A conversational breakdown in human-chatbot interaction refers to a disruption or failure in the communicative flow between the human user and the chatbot. To recover a disrupted conversation, the first step is to detect the breakdown. Researchers have proposed methods using supervised learning and semi-supervised learning in dialogue systems to achieve the goal of detecting conversational breakdown. However, few studies have focused on detecting breakdowns in automated therapeutic conversations, especially conversations led by motivational interviewing chatbots. The presence of conversational breakdowns has negative impacts on the human-chatbot interaction, such as frustration, dissatisfaction, or loss of trust. This gap suggests a need to build a robust and efficient conversational breakdown detector that recognizes interruptions during the conversation. Conversational breakdown detection paves the way for further action to recover conversations. In this paper, I develop a novel, unifying framework called “CIMIC” for characterizing the conversational breakdowns of “MIBot,” a motivational interviewing conversational agent for smoking cessation. I collect 200 pieces of conversational data through Prolific and annotate them using the CIMIC framework with a group of four trained annotators. The annotated dataset is then applied as the training set to fine-tune GPT-3 models to build a conversational breakdown detector for the MIBot.

Natural Language-Driven Dialogue Systems for Support in Physical Medicine and Rehabilitation

This paper presents a natural language-driven dialogue system designed to support healthcare professionals and students in the field of physical medicine and rehabilitation. The system seamlessly integrates concepts from intelligent information systems, data mining, ontologies, and human-computer interaction, employing at its core a rule-based dialogue mechanism. The system harnesses the power of ontology-based graph knowledge, underscoring its domain-specific efficacy. This article delves into the automated knowledge base formation, utilising Python scripts to translate EBSCO’s dataset of articles on physical medicine and rehabilitation into an OWL ontology. This methodology ensures adaptability to the ever-evolving landscape of medical insights. The system’s approach to natural language processing encompasses text preprocessing, semantic category discernment, andSPARQL query creation, providing 26 predefined categories. As an innovation in performance optimisation, the system integrates a strategy to cache precomputed responses using a PostgreSQL database, which aids in resource conservation and reduction in query execution latency. The system’s user engagement avenues are further detailed, showcasing a Telegram bot and an API, enhancing accessibility and user experience. In essence, this article illuminates an advanced, efficient dialogue system for physical medicine and rehabilitation, synthesising multiple computational paradigms, and standing as a beacon for healthcare practitioners and students alike.

A multi-message passing framework based on heterogeneous graphs in conversational emotion recognition

As an important development direction of natural language processing, emotion recognition in conversation (ERC) remains a challenge in sentiment analysis. Given the large-scale dialogue datasets and their wide application in the fields of recommendation systems and human–machine dialogue systems, researchers have begun to pay more attention to the issue of ERC. In recent research, the task of ERC has been largely based on the graph structure to model the speaker level. However, most existing studies simply splice multimodal features, and the heterogeneity of multimodal features tends to be overlooked. Hence, this paper proposes a multivariate messaging framework to embed heterogeneous information into multimodal relational graphs. In the process of aggregating graph node information, we take into account the homogeneity of nodes and assign different weights to different nodes so as to better aggregate semantic information. In order to improve the robustness of the model, we utilize the mechanism of sharing weights among neighbors to reduce the number of network parameters and improve the generalization ability of the model. In so doing, the node information is aggregated through the constructed graph network, and the final semantic vector representation is obtained. Experiments over two benchmark datasets for ERC show that our proposed model achieves improved performance in accuracy and F1 value.

Enhancing Privacy in Large Language Model with Homomorphic Encryption and Sparse Attention

In response to the challenges of personal privacy protection in the dialogue models of the information era, this study introduces an innovative privacy-preserving dialogue model framework. This framework seamlessly incorporates Fully Homomorphic Encryption (FHE) technology with dynamic sparse attention (DSA) mechanisms, aiming to enhance the response efficiency and accuracy of dialogue systems without compromising user privacy. Experimental comparative analyses have confirmed the advantages of the proposed framework in terms of precision, recall, accuracy, and latency, with values of 0.92, 0.91, 0.92, and 15 ms, respectively. In particular, the newly proposed DSA module, while ensuring data security, significantly improves performance by up to 100 times compared to traditional multi-head attention mechanisms.

A Natural Language Generation Algorithm for Greek by Using Hole Semantics and a Systemic Grammatical Formalism

This work is about the progress of previous related work based on an experiment to improve the intelligence of robotic systems, with the aim of achieving more linguistic communication capabilities between humans and robots. In this paper, the authors attempt an algorithmic approach to natural language generation through hole semantics and by applying the OMAS-III computational model as a grammatical formalism. In the original work, a technical language is used, while in the later works, this has been replaced by a limited Greek natural language dictionary. This particular effort was made to give the evolving system the ability to ask questions as well as the authors developed an initial dialogue system using these techniques. The results show that the use of these techniques the authors apply can give us a more sophisticated dialogue system in the future.

A Knowledge-Enhanced Hierarchical Reinforcement Learning-Based Dialogue System for Automatic Disease Diagnosis

Deep Reinforcement Learning is a key technology for the diagnosis-oriented medical dialogue system, determining the type of disease according to the patient’s utterances. The existing dialogue models for disease diagnosis cannot achieve good performance due to the large number of symptoms and diseases. In this paper, we propose a knowledge-enhanced hierarchical reinforcement learning model for strategy learning in the medical dialogue system for disease diagnosis. Our hierarchical strategy alleviates the problem of a large action space in reinforcement learning. In addition, the knowledge enhancement module integrates a learnable disease–symptom relationship matrix and medical knowledge graph into the hierarchical strategy for higher diagnosis success rate. Our proposed model has been proved to be effective on a medical dialogue dataset for automatic disease diagnosis.

Job sharing between human professionals and chatbots: How should ‘handovers’ happen?

Job sharing between humans and artificial intelligence (AI) systems is likely to become increasingly common in several domains of work. In this paper, we examine mechanisms for managing job sharing for one particular class of AI system: human–machine dialogue systems, or chatbots. This is a useful case to consider, as several mechanisms for managing job sharing are already emerging in these systems, and these mechanisms draw on those that human professionals already use to share work among themselves. A key concept in this domain is that of a ‘handover’, where a client is passed from one worker to another. We identify different types of handover, for human and AI workers, and discuss a range of issues that govern how these should take place for effective job sharing. We identify several questions that arise for engineers designing dialogue systems that support handover functionality, relating to timing and transparency. We conclude by arguing that handovers provide a useful way for structuring discussions of job sharing between humans and AI systems, in dialogue-based domains and perhaps beyond.

Graph Models for Contextual Intention Prediction in Dialog Systems

Graph Models for Contextual Intention Prediction in Dialog Systems

Working with roubles and failures in conversation between humans and robots: workshop report.

This paper summarizes the structure and findings from the first Workshop on Troubles and Failures in Conversations between Humans and Robots. The workshop was organized to bring together a small, interdisciplinary group of researchers working on miscommunication from two complementary perspectives. One group of technology-oriented researchers was made up of roboticists, Human-Robot Interaction (HRI) researchers and dialogue system experts. The second group involved experts from conversation analysis, cognitive science, and linguistics. Uniting both groups of researchers is the belief that communication failures between humans and machines need to be taken seriously and that a systematic analysis of such failures may open fruitful avenues in research beyond current practices to improve such systems, including both speech-centric and multimodal interfaces. This workshop represents a starting point for this endeavour. The aim of the workshop was threefold: Firstly, to establish an interdisciplinary network of researchers that share a common interest in investigating communicative failures with a particular view towards robotic speech interfaces; secondly, to gain a partial overview of the "failure landscape" as experienced by roboticists and HRI researchers; and thirdly, to determine the potential for creating a robotic benchmark scenario for testing future speech interfaces with respect to the identified failures. The present article summarizes both the "failure landscape" surveyed during the workshop as well as the outcomes of the attempt to define a benchmark scenario.

The beginning of ChatGPT – a systematic and bibliometric review of the literature

PurposeThis paper aims to present a systematic literature review and bibliometric analysis of research papers published on chat generative pre-trained transformer (ChatGPT), an OpenAI-developed large-scale generative language model. The study’s objective is to provide a comprehensive assessment of the present status of research on ChatGPT and identify current trends and themes in the literature.Design/methodology/approachA total of 328 research article data was extracted from Scopus for bibliometric analysis, to investigate publishing trends, productive countries and keyword analysis around the topic and 34 relevant research publications were selected for an in-depth systematic literature review.FindingsThe findings indicate that ChatGPT research is still in its early stages, with the current emphasis on applications such as natural language processing and understanding, dialogue systems, speech processing and recognition, learning systems, chatbots and response generation. The USA is at the forefront of publishing on this topic and new keywords, e.g. “patient care”, “medical”, “higher education” and so on are emerging themes around the topic.Research limitations/implicationsThese findings underscore the importance of ongoing research and development to address these limitations and ensure that ChatGPT is used responsibly and ethically. While systematic review research on ChatGPT heralds exciting opportunities, it also demands a careful understanding of its nuances to harness its potential effectively.Originality/valueOverall, this study provides a valuable resource for researchers and practitioners interested in ChatGPT at this early stage and helps to identify the grey areas around this topic.

PluDG: enhancing task-oriented dialogue system with knowledge graph plug-in module

Task-oriented dialogue systems continue to face significant challenges as they require not only an understanding of dialogue history but also domain-specific knowledge. However, knowledge is often dynamic, making it difficult to effectively integrate into the learning process. Existing large language model approaches primarily treat knowledge bases as textual resources, neglecting to capture the underlying relationships between facts within the knowledge base. To address this limitation, we propose a novel dialogue system called PluDG. We regard the knowledge as a knowledge graph and propose a knowledge extraction plug-in, Kg-Plug, to capture the features of the graph and generate prompt entities to assist the system’s dialogue generation. Besides, we propose Unified Memory Integration, a module that enhances the comprehension of the sentence’s internal structure and optimizes the knowledge base’s encoding location. We conduct experiments on three public datasets and compare PluDG with several state-of-the-art dialogue models. The experimental results indicate that PluDG achieves significant improvements in both accuracy and diversity, outperforming the current state-of-the-art dialogue system models and achieving state-of-the-art performance.

Emotion-and-knowledge grounded response generation in an open-domain dialogue setting

The neural-based interactive dialogue system focuses on engaging and retaining humans in long-lasting conversations. This has been explored for a variety of goal-oriented dialogue domains, such as education, health care, entertainment, sports, and politics. To develop an understanding and awareness of social and cultural norms, and to address specific social skills, we need to invent strategies for building interactive systems that take into account the user’s emotions and relevant-facts in a multi-turn conversation. In this paper, we propose a new neural generative model that combines step-wise co-attention with a self-attention-based transformer network along with an emotion classifier to jointly control emotion and knowledge transfer during response generation. Quantitative, qualitative, and human evaluation results on the benchmark Topical Chat and the CMU_DoG dataset show that the proposed models can generate natural and coherent sentences, capturing essential facts with considerable improvement over emotional content.

Coreference resolution helps visual dialogs to focus

Visual Dialog is a multi-modal task involving both computer vision and dialog systems. The goal is to answer multiple questions in conversation style, given an image as the context. Neural networks with attention modules are widely used for this task, because of their effectiveness in reasoning the relevance between the texts and images. In this work, we study how to further improve the quality of such reasoning, which is an open challenge. Our baseline is the Recursive Visual Attention (RVA) model, which refines the vision-text attention by iteratively visiting the dialog history. Building on top of that, we propose to improve the attention mechanism with contrastive learning. We train a Matching-Aware Attention Kernel (MAAK) by aligning the deep feature embeddings of an image and its caption, to provide better attention scores. Experiments show consistent improvements from MAAK. In addition, we study the effect of using Multimodal Compact Bilinear (MCB) pooling as a three-way feature fusion for the visual, textual and dialog history embeddings. We analyze the performance of both methods in the discussion section, and propose further ideas to resolve current limitations.

Exploring the Chinese Cultural Aphasia Phenomenon and Countermeasures in College English Teaching Based on Semantic Comprehension Models

Abstract The purpose of this paper is to study the phenomenon of Chinese cultural aphasia and analyze the reasons for the aphasia phenomenon. Secondly, the BERT model in the deep learning model is utilized to improve the effect of various natural language processing tasks, and the English dialogue system is set up to improve the students’ English oral expression ability when introducing Chinese culture. Then, a semantic comprehension model is constructed based on the role information, and coping strategies are proposed for the current English cultural aphasia phenomenon. Finally, the results of teaching evaluation in the university English classroom and the effects of learning Chinese culture in AI-assisted English teaching are analyzed, respectively. According to the study, the teaching process stage is where the most noticeable progress is made, with a score 18.66 points higher than the pre-test score. The average score of the learning effect of students who received AI-assisted means to learn Chinese culture was 3.603 points.

A dual-stream recurrence-attention network with global–local awareness for emotion recognition in textual dialog

In real-world dialog systems, the ability to understand the user’s emotions and interact anthropomorphically is of great significance. Emotion Recognition in Conversation (ERC) is one of the key ways to accomplish this goal and has attracted growing attention. How to model the context in a conversation is a central aspect and a major challenge of ERC tasks. Most existing approaches struggle to adequately incorporate both global and local contextual information, and their network structures are overly sophisticated. For this reason, we propose a simple and effective Dual-stream Recurrence-Attention Network (DualRAN), which is based on Recurrent Neural Network (RNN) and Multi-head ATtention network (MAT). DualRAN eschews the complex components of current methods and focuses on combining recurrence-based methods with attention-based ones. DualRAN is a dual-stream structure mainly consisting of local- and global-aware modules, modeling a conversation simultaneously from distinct perspectives. In addition, we develop two single-stream network variants for DualRAN, i.e., SingleRANv1 and SingleRANv2. According to the experimental findings, DualRAN boosts the weighted F1 scores by 1.43% and 0.64% on the IEMOCAP and MELD datasets, respectively, in comparison to the strongest baseline. On two other datasets (i.e., EmoryNLP and DailyDialog), our method also attains competitive results.

AI to Train AI: Using ChatGPT to Improve the Accuracy of a Therapeutic Dialogue System

In this work, we present the use of one artificial intelligence (AI) application (ChatGPT) to train another AI-based application. As the latter one, we show a dialogue system named Terabot, which was used in the therapy of psychiatric patients. Our study was motivated by the fact that for such a domain-specific system, it was difficult to acquire large real-life data samples to increase the training database: this would require recruiting more patients, which is both time-consuming and costly. To address this gap, we have employed a neural large language model: ChatGPT version 3.5, to generate data solely for training our dialogue system. During initial experiments, we identified intents that were most often misrecognized. Next, we fed ChatGPT with a series of prompts, which triggered the language model to generate numerous additional training entries, e.g., alternatives to the phrases that had been collected during initial experiments with healthy users. This way, we have enlarged the training dataset by 112%. In our case study, for testing, we used 2802 speech recordings originating from 32 psychiatric patients. As an evaluation metric, we used the accuracy of intent recognition. The speech samples were converted into text using automatic speech recognition (ASR). The analysis showed that the patients’ speech challenged the ASR module significantly, resulting in deteriorated speech recognition and, consequently, low accuracy of intent recognition. However, thanks to the augmentation of the training data with ChatGPT-generated data, the intent recognition accuracy increased by 13% relatively, reaching 86% in total. We also emulated the case of an error-free ASR and showed the impact of ASR misrecognitions on the intent recognition accuracy. Our study showcased the potential of using generative language models to develop other AI-based tools, such as dialogue systems.