User Utterances Research Articles

CIRG-SL: Commonsense Inductive Relation Graph framework with Soft Labels for Empathetic Response Generation

Empathy, the ability to understand and respond to others’ emotions, is critical in dialogue systems, particularly for applications such as psychological counselling and casual conversation. However, existing approaches often struggle to accurately capture emotional transitions in user utterances and dialogue contexts across multiple levels of granularity. Additionally, the orthogonality of one-hot emotion representations overlooks the subtle relationships between emotion categories. To overcome these limitations, we propose a novel framework: the commonsense inductive relation graph with fine emotional soft labels for empathetic response generation. This framework refines the representation of user utterances by incorporating commonsense knowledge to derive external emotional cues, which are integrated into an adaptive inductive relation graph. This graph explicitly captures and models dynamic emotional trajectories throughout the dialogue. Additionally, we propose a fine-grained soft label construction method that more accurately reflects the nuanced distribution of emotions, replacing the rigid one-hot encoding with a continuous representation that better captures inter-emotion relationships. Our approach achieves strong performance on the benchmark EmpatheticDialogues dataset, with a perplexity score of 27.85, Dist-1 of 1.81, Dist-2 of 6.58, and a dialogue emotion accuracy of 55.65%. These results highlight the effectiveness of our framework in capturing subtle emotions and generating contextually rich, empathetic responses.

Intent detection for task‐oriented conversational agents: A comparative study of recurrent neural networks and transformer models

AbstractConversational assistants (CAs) and Task‐oriented ones, in particular, are designed to interact with users in a natural language manner, assisting them in completing specific tasks or providing relevant information. These systems employ advanced natural language understanding (NLU) and dialogue management techniques to comprehend user inputs, infer their intentions, and generate appropriate responses or actions. Over time, the CAs have gradually diversified to today touch various fields such as e‐commerce, healthcare, tourism, fashion, travel, and many other sectors. NLU is fundamental in the natural language processing (NLP) field. Identifying user intents from natural language utterances is a sub‐task of NLU that is crucial for conversational systems. The diversity in user utterances makes intent detection (ID) even a challenging problem. Recently, with the emergence of Deep Neural Networks. New State of the Art (SOA) results have been achieved for different NLP tasks. Recurrent neural networks (RNNs) and Transformer architectures are two major players in those improvements. RNNs have significantly contributed to sequence modelling across various application areas. Conversely, Transformer models represent a newer architecture leveraging attention mechanisms, extensive training data sets, and computational power. This review paper begins with a detailed exploration of RNN and Transformer models. Subsequently, it conducts a comparative analysis of their performance in intent recognition for Task‐oriented (CAs). Finally, it concludes by addressing the main challenges and outlining future research directions.

Shift-Reduce Task-Oriented Semantic Parsing with Stack-Transformers

Intelligent voice assistants, such as Apple Siri and Amazon Alexa, are widely used nowadays. These task-oriented dialogue systems require a semantic parsing module in order to process user utterances and understand the action to be performed. This semantic parsing component was initially implemented by rule-based or statistical slot-filling approaches for processing simple queries; however, the appearance of more complex utterances demanded the application of shift-reduce parsers or sequence-to-sequence models. Although shift-reduce approaches were initially considered the most promising option, the emergence of sequence-to-sequence neural systems has propelled them to the forefront as the highest-performing method for this particular task. In this article, we advance the research on shift-reduce semantic parsing for task-oriented dialogue. We implement novel shift-reduce parsers that rely on Stack-Transformers. This framework allows to adequately model transition systems on the transformer neural architecture, notably boosting shift-reduce parsing performance. Furthermore, our approach goes beyond the conventional top-down algorithm: we incorporate alternative bottom-up and in-order transition systems derived from constituency parsing into the realm of task-oriented parsing. We extensively test our approach on multiple domains from the Facebook TOP benchmark, improving over existing shift-reduce parsers and state-of-the-art sequence-to-sequence models in both high-resource and low-resource settings. We also empirically prove that the in-order algorithm substantially outperforms the commonly used top-down strategy. Through the creation of innovative transition systems and harnessing the capabilities of a robust neural architecture, our study showcases the superiority of shift-reduce parsers over leading sequence-to-sequence methods on the main benchmark.

On improving conversational interfaces in educational systems

Conversational Intelligent Tutoring Systems (CITS) have drawn increasing interest in education because of their capacity to tailor learning experiences, improve user engagement, and contribute to the effective transfer of knowledge. Conversational agents employ advanced natural language techniques to engage in a convincing human-like tutorial conversation. In solving math word problems, a significant challenge arises in enabling the system to understand user utterances and accurately map extracted entities to the essential problem quantities required for problem-solving, despite the inherent ambiguity of human natural language. In this study, we propose two possible approaches to enhance the performance of a particular CITS designed to teach learners to solve arithmetic–algebraic word problems. Firstly, we propose an ensemble approach to intent classification and entity extraction, which combines the predictions made by two distinct individual models that use constraints defined by human experts. This approach leverages the intertwined nature of the intents and entities to yield a comprehensive understanding of the user’s utterance, ultimately aiming to enhance semantic accuracy. Secondly, we introduce an adapted Term Frequency-Inverse Document Frequency technique to associate entities with problem quantity descriptions. The evaluation was conducted on the AWPS and MATH-HINTS datasets, containing conversational data and a collection of arithmetical and algebraic math problems, respectively. The results demonstrate that the proposed ensemble approach outperforms individual models, and the proposed method for entity–quantity matching surpasses the performance of typical text semantic embedding models.

Saliency infused dialogue response generation: Improving task oriented text generation using feature attribution

Challenges persist in dialogue scenarios, particularly in multi-turn dialogues where response generation often disregards contextual information beyond the last user utterance, resulting in fluent yet inadequate responses. This paper addresses these issues by identifying and resolving common shortcomings in base model responses during response generation and proposes methods to enhance response quality in unannotated dialogue settings. Our approach involves augmenting information from multiple sources, including keywords, salient features, and knowledge graph triples. We compare the effectiveness of these methods against both the base model and human annotation, which includes dialogue acts and entities. Our findings demonstrate that appending extracted tokens significantly enhances response quality compared to annotated information. In task-oriented dialogue, models perform best when infused with saliency and knowledge graph triples, as shown in the MultiWOZ dataset. Conversely, focusing solely on saliency yields better results for open-domain dialogue, as demonstrated with the DailyDialog dataset. For contextual relevance, the information infusion could also approach the performance of the LLama2 model with only a tenth of the available parameters.

Combining statistical dialog management and intent recognition for enhanced response selection

Abstract Conversational interfaces are becoming ubiquitous in an increasing number of application domains as Artificial Intelligence, Natural Language Processing and Machine Learning methods associated with the recognition, understanding and generation of natural language advance by leaps and bounds. However, designing the dialog model of these systems is still a very demanding task requiring a great deal of effort given the number of information sources to be considered related to the analysis of user utterances, interaction context, information repositories, etc. In this paper, we present a general framework for increasing the quality of the system responses by combining a statistical dialog management technique and a deep learning-based intention recognizer that allow replacing the system responses initially selected by the statistical dialog model with other presumably better candidates. This approach is portable to different task-oriented domains, a diversity of methodologies for dialog management and intention estimation techniques. We have evaluated our two-step proposal using two conversational systems, assessed several intention recognition methodologies and used the developed modules to dynamically select the system responses. The results of the evaluation show that the proposed framework achieves satisfactory results by making it possible to reduce the number of non-coherent dialog responses by replacing them by more coherent alternatives.

DialGNN: Heterogeneous Graph Neural Networks for Dialogue Classification

Dialogue systems have attracted growing research interests due to its widespread applications in various domains. However, most research work focus on sentence-level intent recognition to interpret user utterances in dialogue systems, while the comprehension of the whole documents has not attracted sufficient attention. In this paper, we propose DialGNN, a heterogeneous graph neural network framework tailored for the problem of dialogue classification which takes the entire dialogue as input. Specifically, a heterogeneous graph is constructed with nodes in different levels of semantic granularity. The graph framework allows flexible integration of various pre-trained language representation models, such as BERT and its variants, which endows DialGNN with powerful text representational capabilities. DialGNN outperforms on CM and ECS datasets, which demonstrates robustness and the effectiveness. Specifically, our model achieves a notable enhancement in performance, optimizing the classification of document-level dialogue text. The implementation of DialGNN and related data are shared through https://github.com/821code/DialGNN.

Multi-Stage Prompting for Next Best Agent Recommendations in Adaptive Workflows

Traditional business processes such as loan processing, order processing, or procurement have a series of steps that are pre-defined at design and executed by enterprise systems. Recent advancements in new-age businesses, however, focus on having adaptive and ad-hoc processes by stitching together a set of functions or steps enabled through autonomous agents. Further, to enable business users to execute a flexible set of steps, there have been works on providing a conversational interface to interact and execute automation. Often, it is necessary to guide the user through the set of possible steps in the process (or workflow). Existing work on recommending the next agent to run relies on historical data. However, with changing workflows and new automation constantly getting added, it is important to provide recommendations without historical data. Additionally, hand-crafted recommendation rules do not scale. The adaptive workflow being a combination of structured and unstructured information, makes it harder to mine. Hence, in this work, we leverage Large Language Models (LLMs) to combine process knowledge with the meta-data of agents to discover NBAs specifically at cold-start. We propose a multi-stage approach that uses existing process knowledge and agent meta-data information to prompt LLM and recommend meaningful next best agent (NBA) based on user utterances.

Winnie: Task-Oriented Dialog System with Structure-Aware Contrastive Learning and Enhanced Policy Planning

Pre-trained encoder-decoder models are widely applied in Task-Oriented Dialog (TOD) systems on the session level, mainly focusing on modeling the dialog semantic information. Dialogs imply structural information indicating the interaction among user utterances, belief states, database search results, system acts and responses, which is also crucial for TOD systems. In addition, for the system acts, additional pre-training and datasets are considered to improve their accuracies, undoubtedly introducing a burden. Therefore, a novel end-to-end TOD system named Winnie is proposed in this paper to improve the TOD performance. First, to make full use of the intrinsic structural information, supervised contrastive learning is adopted to narrow the gap in the representation space between text representations of the same category and enlarge the overall continuous representation margin between text representations of different categories in dialog context. Then, a system act classification task is introduced for policy optimization during fine-tuning. Empirical results show that Winnie substantially improves the performance of the TOD system. By introducing the supervised contrastive and system act classification losses, Winnie achieves state-of-the-art results on benchmark datasets, including MultiWOZ2.2, In-Car, and Camrest676. Their end-to-end combined scores are improved by 3.2, 1.9, and 1.1 points, respectively.

DSTEA: Improving Dialogue State Tracking via Entity Adaptive pre-training

Dialogue State Tracking (DST) is critical for comprehensively interpreting user and system utterances, thereby forming the cornerstone of efficient dialogue systems. Despite past research efforts focused on enhancing DST performance through alterations to the model structure or integrating additional features like graph relations, they often require additional pre-training with external dialogue corpora. In this study, we propose DSTEA, improving Dialogue State Tracking via Entity Adaptive pre-training, which can enhance the encoder through by intensively training key entities in dialogue utterances. DSTEA identifies these pivotal entities from input dialogues utilizing four different methods: ontology information, named-entity recognition, the spaCy toolkit, and the flair library. Subsequently, it employs selective knowledge masking to train the model effectively. Remarkably, DSTEA only requires pre-training without the direct infusion of extra knowledge into the DST model. This approach results in substantial performance improvements of four robust DST models on MultiWOZ 2.0, 2.1, and 2.2, with joint goal accuracy witnessing an increase of up to 2.69% (from 52.41% to 55.10%). Comparative experiments considering various entity types and different entity adaptive pre-training configurations, such as masking strategy and masking rate, further validated the efficacy of DSTEA.

Conversational Localization

We propose a novel sensorless approach to indoor localization by leveraging natural language conversations with users, which we call conversational localization. To show the feasibility of conversational localization, we develop a proof-of-concept system that guides users to describe their surroundings in a chat and estimates their position based on the information they provide. We devised a modular architecture for our system with four modules. First, we construct an entity database with available image-based floor maps. Second, we enable the dynamic identification and scoring of information provided by users through our utterance processing module. Then, we implement a conversational agent that can intelligently strategize and guide the interaction to elicit localizationally valuable information from users. Finally, we employ visibility catchment area and line-of-sight heuristics to generate spatial estimates for the user's location. We conduct two user studies in designing and testing the system. We collect 800 natural language descriptions of unfamiliar indoor spaces in an online crowdsourcing study to learn the feasibility of extracting localizationally useful entities from user utterances. We then conduct a field study with 10 participants at 10 locations to evaluate the feasibility and performance of conversational localization. The results show that conversational localization can achieve within-10 meter localization accuracy at eight out of the ten study sites, showing the technique's utility for classes of indoor location-based services.

Language Artificial Intelligences' Communicative Performance Quantified Through the Gricean Conversation Theory.

This study pragmatically investigates an artificial intelligence (AI) speaker (AIS)'s verbal communicative performance based on real AI-human conversation data. Specifically, this study explores Grice's conversation theory, which enables the categorization of an AIS's mistaken utterances as violations of specific conversational maxims. Twenty native Korean-speaking participants recorded at least 50 conversations with Kakao Mini AISs, provided by Daum Kakao, Inc., in Korea. Each conversation, either for information sharing or as daily dialogue, was required to contain at least two turn-taking instances. A total of 1,026 recorded dialogues were decomposed into adjacency pairs based on turn-taking. The dialogues were arranged into 3,365 adjacency pairs, and each pair was then classified as a conversational success or failure based on whether the AIS answered the user's utterance appropriately. Language users' evaluations of the AIS's mistaken expressions were also quantified via an additional acceptability rating test with 1,024 adjacency pairs. The overall results indicate that Grice's "maxim of relation" is most frequently flouted by AISs and is considered to be the least natural to language users. These findings suggest that to improve AISs' natural communication capacity, more detailed AI algorithms that generate utterances relevant to either the partner's preceding utterance or a broader conversational context should be created. Although the verbal communicative capacities of the AIS we test are substantially overtaken by those of recent large language models, such as generative pretrained transformer, the pragmatic evaluation described in the current study will remain useful for more precise linguistic quantification of current/future language AI's communicative performance/competence.

Automated government form filling for aged and monolingual people using interactive tool

Purpose The Government of India offers various schemes for various classes of citizens. Most of the application forms of schemes to be filled are in English and it is observed that monolingual individuals find it difficult to access and fill the forms. This paper addresses the challenges faced by monolingual individuals in India, particularly the elderly, people with impairments, and those from marginalized communities. The proposed work is to create an interactive system called "Dhvani" voicebot, specifically designed for the Kannada language. It helps users in identifying suitable government schemes and fills forms in English. Materials and Methods The proposed system is developed using the RASA chatbot framework and NLP techniques to comprehend user utterances. RNN and SVM algorithms are employed to ensure smooth conversation flow and interaction with the users. To enhance scheme suggestion accuracy, a knowledge graph is created, containing relevant data on government schemes. Results The intent classification model achieves an accuracy of 97%, indicating its ability to accurately understand user intentions. The integration of a knowledge graph improves the accuracy of scheme identification and suggestion to users. The system automates the process of filling out government scheme forms based on user inputs. Conclusion Dhvani voicebot system presents a practical solution to address the challenges faced by monolingual individuals in accessing government schemes in India. The high accuracy of intent classification and the use of a knowledge graph contribute to the system’s effectiveness. The study suggests that this system can be extended to other languages.

Out-of-scope intent detection with intent-invariant data augmentation

In practical dialogue systems, it is crucial to avoid undesired responses and poor user experiences by detecting Out-Of-Scope (OOS) intents from user utterances. Currently, to detect OOS intents, limited-supervised methods are more potential due to using limited true OOS data to better model the OOS data distribution. However, existing limited-supervised methods often yield low-quality OOS augmentations, i.e., noisy OOS and long-distant OOS, due to the use of the retrieval-based data augmentation mechanism, which will damage the OOS intent detection performance. To tackle this problem, in this paper, we propose a novel OOS intent detection method by leveraging intent-invariant data augmentation, called InInOOS, which can generate high-quality pseudo-OOS utterances with invariant OOS intents but various slot values and different expressions to further enhance the OOS intent detection performance. Specifically, we first generate pseudo-OOS candidates using our pre-trained intent-invariant utterance generation model, and then the most beneficial candidates are selected to train an OOS intent detection model. Extensive experiments on two public datasets show that our method performs better than state-of-the-art baselines.

Effect of attentive listening robot on pleasure and arousal change in psychiatric daycare

In this paper, we investigate the usefulness of an attentive-listening robot in psychiatric daycare, an outpatient treatment program for the rehabilitation of psychiatric disorders. The robot was developed based on counseling techniques, such as repeating words that the user has said. It can also generate backchannels as a listening behavior during user utterance. Conversation experiments have been conducted to evaluate whether the robot can provide effective activities in this setting. The robot attentively listened to 18 daycare attendees talking about their recent memorable events for up to 3 min. The results showed that the conversation increased self-rated arousal. The impressions of the robot showed that talking with the robot was more conversable than with strangers and more useful as a talking partner than a friend. The subjects also had a positive impression about whether they would keep the robot in their homes. A linear regression analysis indicates that the frequency of the robot's assessment responses and backchannels positively affect pleasure improvement. The findings may pave the way for utilizing this kind of robots that people can talk to easily without hesitation or excessive consideration.

On Using Conversational Frameworks to Support Natural Language Interaction in Intelligent Tutoring Systems

In this paper, we analyse the potential of conversational frameworks to support the adaptation of existing tutoring systems to a natural language form of interaction. We have based our research on a pilot study, in which the open-source machine learning framework Rasa has been used to build a conversational agent that interacts with an existing Intelligent Tutoring System (ITS) called Hypergraph Based Problem Solver (HBPS). This agent has been seamlessly integrated into the ITS to replace the previously available button-based user interface and allow the user to interact with HBPS in natural language. Once appropriately trained, the conversational agent was capable of identifying the intention of a given user utterance and extracting the relevant entities related to the message content, with average weighted F1-scores of 0.965 and 0.989 for intents and entities, respectively. Methodological guidelines are provided to generate a realistic training set that enables the creation of the required Natural Language Understanding (NLU) model and also evaluate the resulting system. These guidelines can be easily exported to other ITS and contexts, to provide an enhanced interaction based on natural language processing methods.

Are You Depressed? Analyze User Utterances to Detect Depressive Emotions Using DistilBERT

This paper introduces the Are u Depressed (AuD) model, which aims to detect depressive emotional intensity and classify detailed depressive symptoms expressed in user utterances. The study includes the creation of a BWS dataset using a tool for the Best-Worst Scaling annotation task and a DSM-5 dataset containing nine types of depression annotations based on major depressive disorder (MDD) episodes in the Diagnostic and Statistical Manual of Mental Disorders (DSM-5). The proposed model employs the DistilBERT model for both tasks and demonstrates superior performance compared to other machine learning and deep learning models. We suggest using our model for real-time depressive emotion detection tasks that demand speed and accuracy. Overall, the AuD model significantly advances the accurate detection of depressive emotions in user utterances.

Better Data from AI Users: A Field Experiment on the Impacts of Robot Self-Disclosure on the Utterance of Child Users in Home Environment

Data are one of the important factors in artificial intelligence (AI). Moreover, in order for AI to understand the user and go beyond the role of a simple machine, the data contained in the user's self-disclosure is required. In this study, two types of robot self-disclosures (disclosing robot utterance, involving user utterance) are proposed to elicit higher self-disclosure from AI users. Additionally, this study examines the moderating effects of multi-robot conditions. In order to investigate these effects empirically and increase the implications of research, a field experiment with prototypes was conducted in the context of using smart speaker of children. The results indicate that both types of robot self-disclosures were effective in eliciting the self-disclosure of children. The interaction effect between disclosing robot and involving user was found to take a different direction depending on the sub-dimension of the user's self-disclosure. Multi-robot conditions partially moderate the effects of the two types of robot self-disclosures.

Multitask learning for multilingual intent detection and slot filling in dialogue systems

Dialogue systems are becoming an ubiquitous presence in our everyday lives having a huge impact on business and society. Spoken language understanding (SLU) is the critical component of every goal-oriented dialogue system or any conversational system. The understanding of the user utterance is crucial for assisting the user in achieving their desired objectives. Future-generation systems need to be able to handle the multilinguality issue. Hence, the development of conversational agents becomes challenging as it needs to understand the different languages along with the semantic meaning of the given utterance. In this work, we propose a multilingual multitask approach to fuse the two primary SLU tasks, namely, intent detection and slot filling for three different languages. While intent detection deals with identifying user’s goal or purpose, slot filling captures the appropriate user utterance information in the form of slots. As both of these tasks are highly correlated, we propose a multitask strategy to tackle these two tasks concurrently. We employ a transformer as a shared sentence encoder for the three languages, i.e., English, Hindi, and Bengali. Experimental results show that the proposed model achieves an improvement for all the languages for both the tasks of SLU. The multi-lingual multi-task (MLMT) framework shows an improvement of more than 2% in case of intent accuracy and 3% for slot F1 score in comparison to the single task models. Also, there is an increase of more than 1 point intent accuracy and 2 points slot F1 score in the MLMT model as opposed to the language specific frameworks.

MACR: Multi-information Augmented Conversational Recommender

Conversational recommender systems (CRS) aim to provide high-quality recommendations through fewer multi-turn conversations. However, because short conversation histories lack sufficient item information, CRSs not only struggle to make accurate recommendations but also lack diversity in the generated responses. Existing CRSs mainly alleviate these problems by introducing external information (e.g., reviews) while ignoring information inside the conversations (e.g., potential category preferences in user utterances). Besides, item introduction is a kind of external information that is more objective and contains more entities than reviews. Therefore, we propose a Multi-information Augmented Conversational Recommender (MACR), which improves the performance of recommendation and response generation by mining the underlying category preferences in users’ utterances and incorporating item introductions. Specifically, we enhance the category associations among entities by constructing a knowledge graph DBMG with category nodes, extracting and encoding the item categories that match the user preferences into the user representation. For item introductions, we extract the entities in them and fuse them into the conversation using an introduction-attentive encoder–decoder. Extensive experiments on the dataset REDIAL show that our MACR significantly outperforms previous state-of-the-art approaches. The source code will be available at https://github.com/zcy-cqut/MACR.