Dialogue Response Research Articles

Combining Natural Language Processing with Epistemic Network Analysis to Investigate Student Knowledge Integration within an AI Dialog

AbstractIn this study, we used Epistemic Network Analysis (ENA) to represent data generated by Natural Language Processing (NLP) analytics during an activity based on the Knowledge Integration (KI) framework. The activity features a web-based adaptive dialog about energy transfer in photosynthesis and cellular respiration. Students write an initial explanation, respond to two adaptive prompts in the dialog, and write a revised explanation. The NLP models score the KI level of the initial and revised explanations. They also detect the ideas in the explanations and the dialog responses. The dialog uses the detected ideas to prompt students to elaborate and refine their explanations. Participants were 196 8th-grade students at a public school in the Western United States. We used ENA to represent the idea networks at each KI score level for the revised explanations. We also used ENA to analyze the idea trajectories for the initial explanation, the two dialog responses, and the final explanation. Higher KI levels were associated with more links and increased frequency of mechanistic ideas in ENA representations. Representation of the trajectories suggests that the NLP adaptive dialog helped students who started with descriptive and macroscopic ideas to add more microscopic ideas. The dialog also helped students who started with partially linked ideas to keep linking the microscopic ideas to mechanistic ideas. We discuss implications for STEM teachers and researchers who are interested in how students build on their ideas to integrate their ideas.

A Novel Cognitive Behavioral Therapy-Based Generative AI Tool (Socrates 2.0) to Facilitate Socratic Dialogue: Protocol for a Mixed Methods Feasibility Study.

Digital mental health tools, designed to augment traditional mental health treatments, are becoming increasingly important due to a wide range of barriers to accessing mental health care, including a growing shortage of clinicians. Most existing tools use rule-based algorithms, often leading to interactions that feel unnatural compared with human therapists. Large language models (LLMs) offer a solution for the development of more natural, engaging digital tools. In this paper, we detail the development of Socrates 2.0, which was designed to engage users in Socratic dialogue surrounding unrealistic or unhelpful beliefs, a core technique in cognitive behavioral therapies. The multiagent LLM-based tool features an artificial intelligence (AI) therapist, Socrates, which receives automated feedback from an AI supervisor and an AI rater. The combination of multiple agents appeared to help address common LLM issues such as looping, and it improved the overall dialogue experience. Initial user feedback from individuals with lived experiences of mental health problems as well as cognitive behavioral therapists has been positive. Moreover, tests in approximately 500 scenarios showed that Socrates 2.0 engaged in harmful responses in under 1% of cases, with the AI supervisor promptly correcting the dialogue each time. However, formal feasibility studies with potential end users are needed. This mixed methods study examines the feasibility of Socrates 2.0. On the basis of the initial data, we devised a formal feasibility study of Socrates 2.0 to gather qualitative and quantitative data about users' and clinicians' experience of interacting with the tool. Using a mixed method approach, the goal is to gather feasibility and acceptability data from 100 users and 50 clinicians to inform the eventual implementation of generative AI tools, such as Socrates 2.0, in mental health treatment. We designed this study to better understand how users and clinicians interact with the tool, including the frequency, length, and time of interactions, users' satisfaction with the tool overall, quality of each dialogue and individual responses, as well as ways in which the tool should be improved before it is used in efficacy trials. Descriptive and inferential analyses will be performed on data from validated usability measures. Thematic analysis will be performed on the qualitative data. Recruitment will begin in February 2024 and is expected to conclude by February 2025. As of September 25, 2024, overall, 55 participants have been recruited. The development of Socrates 2.0 and the outlined feasibility study are important first steps in applying generative AI to mental health treatment delivery and lay the foundation for formal feasibility studies. DERR1-10.2196/58195.

Generative commonsense knowledge subgraph retrieval for open-domain dialogue response generation

Grounding on a commonsense knowledge subgraph can help the model generate more informative and diverse dialogue responses. Prior Traverse-based works explicitly retrieve a subgraph from the external knowledge base (eKB). Notably, the available knowledge is strictly restricted by the eKB. To break this restriction, Generative Retrieval methods externalize knowledge from the language model. However, they always generate boring knowledge due to their one-pass externalization procedure. This work proposes a novel TiLM Traverse in Language Model (TiLM), which uses three ‘Chain-of-Thought’ sub-tasks, i.e., Query Entity Production, Topic Entity Prediction, and Knowledge Subgraph Completion, to build a high-quality knowledge subgraph to ground the next Response Generation without explicitly accessing the eKB in inference. Experimental results on both Chinese and English datasets demonstrate TiLM’s outstanding performance even only with a small scale of parameters.

Studi Literatur Pemanfaatan Teknologi Chat GPT dalam Pendidikan

The aim of the research is to analyze the development of publications related to the use of GPT Chat technology in the field of education. The research method is a literature study with bibliometric analysis. A total of 919 articles were used from the Google Scholar database. The results of research publications on the use of GPT Chat technology in education have increased from year to year in the 2018-2022 period. The presentation of research development is known to be 2018, namely 1%, 2019 4%, 2020 17%, 2021 33%, and 2022 45%. There are 6 clusters namely cluster 1 (15 items), cluster 2 (14 items), cluster 3 (13 items), cluster 4 (12 items), cluster 5 (10 items), and cluster 6 (6 items). GPT Chat research network mapping in education topics that are often researched are GPT, System and Chatbot, while the topics that become new themes are Xlnet, Elmo, NLP, Deep Learning, and Dialog Response Generation, so that the keywords with the new theme can be used as a reference in do further research.

Evaluating emotional and subjective responses in synthetic art-related dialogues: A multi-stage framework with large language models

The appearance of Large Language Models (LLM) has implied a qualitative step forward in the performance of conversational agents, and even in the generation of creative texts. However, previous applications of these models in generating dialogues neglected the impact of ‘hallucinations’ in the context of generating synthetic dialogues, thus omitting this central aspect in their evaluations. For this reason, we propose an open-source and flexible framework called GenEvalGPT framework: a comprehensive multi-stage evaluation strategy utilizing diverse metrics. The objective is two-fold: first, the goal is to assess the extent to which synthetic dialogues between a chatbot and a human align with the specified commands, determining the successful creation of these dialogues based on the provided specifications; and second, to evaluate various aspects of emotional and subjective responses. Assuming that dialogues to be evaluated were synthetically produced from specific profiles, the first evaluation stage utilizes LLMs to reconstruct the original templates employed in dialogue creation. The success of this reconstruction is then assessed in a second stage using lexical and semantic objective metrics. On the other hand, crafting a chatbot’s behaviors demands careful consideration to encompass a diverse range of interactions it is meant to engage in. Synthetic dialogues play a pivotal role in this context, as they can be deliberately synthesized to emulate various behaviors. This is precisely the objective of the third stage: evaluating whether the generated dialogues adhere to the required aspects concerning emotional and subjective responses. To validate the capabilities of the proposed framework, we applied it to recognize whether the chatbot exhibited one of two distinct behaviors in the synthetically generated dialogues: being emotional and providing subjective responses, or remaining neutral. This evaluation will encompass traditional metrics and automatic metrics generated by the LLM. In our use case of art-related dialogues, our findings reveal that the capacity to recover templates or profiles is more effective for information or profile items that are objective and factual, in contrast to those related to mental states or subjective facts. For the emotional and subjective behavior assessment, rule-based metrics achieved a 79% of accuracy in detecting emotions or subjectivity (anthropic), and an 82% on the LLM automatic metrics. The combination of these metrics and stages could help to decide which of the generated dialogues should be maintained depending on the applied policy, which could vary from preserving between 57% to 93% of the initial dialogues.

Integrating discourse features and response assessment for advancing empathetic dialogue

Empathetic response generation is a crucial task in natural language processing, enabling emotionally resonant machine–human interactions. In this paper, we introduce the InfRa (Integrating Discourse Features and Response Assessment) model to address limitations in traditional methods for this task, such as the lack of deep dialogue comprehension and response control. InfRa integrates discourse features to augment structural dialogue understanding, with a novel edge pruning and mutual information learning module to further refine the representation. The model also employs a response evaluation module for dynamic optimization, ensuring emotional and semantic consistency between the generated response and its context. Our experiments demonstrate that InfRa outperforms existing baselines, reducing the Perplexity (PPL) score by approximately 9 points and excelling in all three fine-grained aspects of human evaluation. This research not only advances the development of empathetic chatbots but also provides valuable insights for broader text generation tasks.

The Influence of Response Time and Feedback Type on the User Experience of Voice Interaction among Older Adults

With the maturity and development of digital technology, voice interaction has been widely used in different devices or scenarios. However, older adults have low satisfaction and acceptance levels when using voice interaction. To improve older adults’ user experience of voice interaction, the current study explored the effect of response time and feedback type on the user experience of voice interaction among older adults. A Xiaodu smart speaker was used as the experimental device, and the response time was recorded using artificial breakpoints. Our study consisted of two experiments that explored the voice wake-up and voice dialogue stages under different combinations of response time and feedback type and older adults rated their satisfaction, comfort, anthropomorphism, and perceived speed level. The results showed that: (1) the optimal wake-up response time for older adults is 250–1000 ms, and the optimal dialogue response time is 600–1500 ms. (2) As response time increases, older adults’ satisfaction, comfort, and anthropomorphism toward smart speakers decrease. In the wake-up stage, the scores in the time period >2000 ms is significantly lower than those shorter response times. In the dialogue stage, the scores in the time periods >3000 ms are significantly lower than those shorter response times. (3) In the wake-up stage, the voice-based prompt “Hello, I’m Xiaodu” scores highest among all kinds of feedback types. In the dialogue stage, the social-oriented feedback scores higher in satisfaction and comfort than task-oriented feedback. The results of the current study could provide implications for age-friendly voice interaction design.

Customer Intent Mining from Service Inquiries with Newly Improved Deep Embedded Clustering

Discovering customer intents from text or speech data plays a vital role in text mining and automated dialogue response. It is challenging to process thousands of customer interactions daily. Deep Embedded Clustering (DEC) and Improved DEC (IDEC) with Kullback–Leibler loss handle a lot of data inefficiently due to the asymmetric nature of the loss. To address the challenge, an unsupervised learning approach to discover intents and automatically produce the labels from a collection of unlabeled utterances in the context of the banking domain is proposed. The proposed approach focuses on improving both architectures of DEC and IDEC by combining the Jensen–Shannon (JS) divergence to simultaneously learn feature representations and cluster assignments, and the Second-order Clipped Stochastic Optimization (Sophia). Then, a set of intent labels for each cluster is generated by using a dependency parser in the second stage. Experimental results showed that the proposed approach is capable of generating meaningful intent labels and short text clustering with high performance.

Enhancing Document Information Selection Through Multi-Granularity Responses for Dialogue Generation

Document information selection is an essential part of document-grounded dialogue tasks, and more accurate information selection results can provide more appropriate dialogue responses. Existing works have achieved excellent results by employing multi-granularity of dialogue history information, indicating the effectiveness of multi-level historical information. However, these works often focus on exploring the hierarchical information of dialogue history, while neglecting the multi-granularity utilization in response, important information that holds an impact on the decoding process. Therefore, this paper proposes a model for document information selection based on multi-granularity responses. By integrating the document selection results at the response word level and semantic unit level, the model enhances its capability in knowledge selection and produces better responses. For the division at the semantic unit level of the response, we propose two semantic unit division methods, static and dynamic. Experiments on two public datasets show that our models combining static or dynamic semantic unit levels significantly outperform baseline models.

Saliency infused dialogue response generation: Improving task oriented text generation using feature attribution

Challenges persist in dialogue scenarios, particularly in multi-turn dialogues where response generation often disregards contextual information beyond the last user utterance, resulting in fluent yet inadequate responses. This paper addresses these issues by identifying and resolving common shortcomings in base model responses during response generation and proposes methods to enhance response quality in unannotated dialogue settings. Our approach involves augmenting information from multiple sources, including keywords, salient features, and knowledge graph triples. We compare the effectiveness of these methods against both the base model and human annotation, which includes dialogue acts and entities. Our findings demonstrate that appending extracted tokens significantly enhances response quality compared to annotated information. In task-oriented dialogue, models perform best when infused with saliency and knowledge graph triples, as shown in the MultiWOZ dataset. Conversely, focusing solely on saliency yields better results for open-domain dialogue, as demonstrated with the DailyDialog dataset. For contextual relevance, the information infusion could also approach the performance of the LLama2 model with only a tenth of the available parameters.

Can ChatGPT provide a better support: a comparative analysis of ChatGPT and dataset responses in mental health dialogues

Can ChatGPT provide a better support: a comparative analysis of ChatGPT and dataset responses in mental health dialogues

Combining statistical dialog management and intent recognition for enhanced response selection

Abstract Conversational interfaces are becoming ubiquitous in an increasing number of application domains as Artificial Intelligence, Natural Language Processing and Machine Learning methods associated with the recognition, understanding and generation of natural language advance by leaps and bounds. However, designing the dialog model of these systems is still a very demanding task requiring a great deal of effort given the number of information sources to be considered related to the analysis of user utterances, interaction context, information repositories, etc. In this paper, we present a general framework for increasing the quality of the system responses by combining a statistical dialog management technique and a deep learning-based intention recognizer that allow replacing the system responses initially selected by the statistical dialog model with other presumably better candidates. This approach is portable to different task-oriented domains, a diversity of methodologies for dialog management and intention estimation techniques. We have evaluated our two-step proposal using two conversational systems, assessed several intention recognition methodologies and used the developed modules to dynamically select the system responses. The results of the evaluation show that the proposed framework achieves satisfactory results by making it possible to reduce the number of non-coherent dialog responses by replacing them by more coherent alternatives.

An Adaptive Contextual Relation Model for Improving Response Generation

An Adaptive Contextual Relation Model for Improving Response Generation

Yes, I am afraid of the sharks and also wild lions!: A multitask framework for enhancing dialogue generation via knowledge and emotion grounding

Current end-to-end neural conversation models inherently lack the capability to generate coherently engaging responses. Efforts to boost informativeness have an adversarial effect on emotional and factual accuracy, as validated by several sequence-based models. While these issues can be alleviated by access to emotion labels and background knowledge, there is no guarantee of relevance and informativeness in the generated responses. In real dialogue corpus, informative words like named entities, and words that carry specific emotions can often be infrequent and hard to model, and one primary challenge of the dialogue system is how to promote the model’s capability of generating high-quality responses with those informative words. Furthermore, earlier approaches depended on straightforward concatenation techniques that lacked robust representation capabilities in order to account for human emotions. To address this problem, we propose a novel multitask hierarchical encoder–decoder model, which can enhance the multi-turn dialogue response generation by incorporating external textual knowledge and relevant emotions. Experimental results on a benchmark dataset indicate that our model is superior over competitive baselines concerning both automatic and human evaluation.

Talk Funny! A Large-Scale Humor Response Dataset with Chain-of-Humor Interpretation

Humor is a crucial part of human communication. Understanding humor and generating humorous responses in dialogue can provide natural and empathic human-computer interactions. However, most existing pre-trained language models (PLMs) perform unsatisfactorily in humor generation. On one hand, the serious shortage of humor corpus and datasets pose challenges for constructing models that can understand and generate humorous expressions. On the other hand, humor generation relies on rich knowledge and commonsense, which is often tacit and unspoken. In this paper, we construct the largest Chinese Explainable Humor Response Dataset to date with chain-of-humor and humor mind map annotations, which can be used to comprehensively evaluate as well as improve the humorous response ability of PLMs. We further design humor-related auxiliary tasks to further enhance PLMs' humorous response performance. Extensive evaluations demonstrate that our proposed dataset and auxiliary tasks effectively help PLMs to generate humorous responses, laying the groundwork for future humor research.

Response Enhanced Semi-supervised Dialogue Query Generation

Leveraging vast and continually updated knowledge from the Internet has been considered an important ability for a dialogue system. Therefore, the dialogue query generation task is proposed for generating search queries from dialogue histories, which will be submitted to a search engine for retrieving relevant websites on the Internet. In this regard, previous efforts were devoted to collecting conversations with annotated queries and training a query producer (QP) via standard supervised learning. However, these studies still face the challenges of data scarcity and domain adaptation. To address these issues, in this paper, we propose a semi-supervised learning framework -- SemiDQG, to improve model performance with unlabeled conversations. Based on the observation that the search query is typically related to the topic of dialogue response, we train a response-augmented query producer (RA) to provide rich and effective training signals for QP. We first apply a similarity-based query selection strategy to select high-quality RA-generated pseudo queries, which are used to construct pseudo instances for training QP and RA. Then, we adopt the REINFORCE algorithm to further enhance QP, with RA-provided rewards as fine-grained training signals. Experimental results and in-depth analysis of three benchmarks show the effectiveness of our framework in cross-domain and low-resource scenarios. Particularly, SemiDQG significantly surpasses ChatGPT and competitive baselines. Our code is available at \url{https://github.com/DeepLearnXMU/SemiDQG}.

Improving Open-Domain Dialogue Response Generation with Multi-Source Multilingual Commonsense Knowledge

Knowledge-grounded Dialogue Response Generation (KRG) can facilitate informative and fidelity dialogues using external knowledge. Prior monolingual works can only use the knowledge of the corresponding native language. Thus, due to the prohibitive costs of collecting and constructing external knowledge bases, the limited scale of accessible external knowledge always constrains the ability of KRG, especially in low-resource language scenarios. To this end, we propose a new task, Multi-Source Multilingual Knowledge-Grounded Response Generation (MMKRG), which simultaneously uses multiple knowledge sources of different languages. We notice that simply combining knowledge of different languages is inefficient due to the Cross-Conflict issue and Cross-Repetition issue. Thus, we propose a novel approach MMK-BART, which uses a simple but elegant Estimate-Cluster-Penalize mechanism to overcome the mentioned issues and adopts the multilingual language model mBART as the backbone. Meanwhile, based on the recent multilingual corpus XDailyDialog, we propose an MMKRG dataset MMK-DailyDialog, which has been aligned to the large-scale multilingual commonsense knowledge base ConceptNet and supports four languages (English, Chinese, German, and Italian). Extensive experiments have verified the effectiveness of our dataset and approach in monolingual, cross-lingual, and multilingual scenarios.

China's Response To Indonesia's Policies In The North Natuna Sea

Problems: The North Natuna Sea has emerged as a point of contention between China and Indonesia. This research delves into the ambiguity and potential conflict within China's response to Indonesia's actions in the region. Purpose: This research aims to examine China's response to Indonesia's North Natuna Sea policies. And also Investigate Indonesia's policy regarding North Natuna Sea map issuance. Methodology: Qualitative research methods are employed, primarily involving an extensive review of existing literature. The analysis relies on a descriptive approach, drawing from foreign policy theory and geopolitical theory. Results/Findings: Indonesia's bold move in 2017, Indonesia officially inaugurated the North Natuna Sea Map. Objectives were clear, demarcate territorial boundaries, maximize resource utilization, and advance maritime ambitions. There is China's Ambiguous Response that took three forms. Coercive Response, China asserted dominance via the controversial Nine Dash Line and military force deployment within Indonesia's Exclusive Economic Zone (EEZ) in the North Natuna Sea. Persuasive Response, Diplomatically, China argued for South China Sea sovereignty, including the Nansha Islands and their waters. President Xi Jinping advocated mutual respect among neighboring countries. Dialogic Response, China engaged in diplomatic dialogues through forums like ASEAN-China Free Trade Area (ACFTA) and ASEAN-China Political Consultative Mechanism (PCM) to reduce regional tensions, seeking mutual agreements.Bottom of Form Paper Type: Geopolitical Issue Keywords: North Natuna Sea, UNLCOS, ZEE

Exploring Dense Retrieval for Dialogue Response Selection

Recent progress in deep learning has continuously improved the accuracy of dialogue response selection. However, in real-world scenarios, the high computation cost forces existing dialogue response selection models to rank only a small number of candidates, recalled by a coarse-grained model, precluding many high-quality candidates. To overcome this problem, we present a novel and efficient response selection model and a set of tailor-designed learning strategies to train it effectively. The proposed model consists of a dense retrieval module and an interaction layer, which could directly select the proper response from a large corpus. We conduct re-rank and full-rank evaluations on widely used benchmarks to evaluate our proposed model. Extensive experimental results demonstrate that our proposed model notably outperforms the state-of-the-art baselines on both re-rank and full-rank evaluations. Moreover, human evaluation results show that the response quality could be improved further by enlarging the candidate pool with nonparallel corpora. In addition, we also release high-quality benchmarks that are carefully annotated for more accurate dialogue response selection evaluation. All source codes, datasets, model parameters, and other related resources have been publicly available. 1

Семантика глаголов сквозь призму «пассивных» и «активных» словарей

The paper compares the meanings of the verbs zhit’ (to live), delat’ (to do), and idti (to go) as presented in dictionaries of different types. The dictionaries used include three types: (1) the traditional “passive” dictionaries by V. I. Dal’ and T. F. Efremova, (2) the “active” dictionary by Yu. D. Apresyan, and (3) the associative databases RAS (Russian Associative Dictionary) and SIBAS (Russian Regional Associative Database of Siberia and the Far East). The first and second types of dictionaries are based on textual materials. The third type reflects associative-verbal networks modeled on the data of free mass associative experiments conducted in the USSR and Russia in the 1990s (RAS) and 2000s (SIBAS). The assumptions are corroborated that the mental meaning of a word differs from its linguistic meaning and that associative thesauri reflect the language consciousness of an average Russian speaker. Thus, the order of meanings of the verb zhit’ given in the first and second types of dictionaries does not match the “ranks” of these meanings in the native speakers’ language consciousness depicted in RAS and SIBAS. Dialogic responses, traces of precedent phenomena, and visual images present in RAD and SIBAS supplement the image of active linguistic consciousness processing the speaker’s experience of interacting with reality. Conversely, the first and second types of dictionaries provide more information about different meanings and contexts that exclude ambiguity of interpretation of meanings. It is concluded that a comprehensive study of linguistic personality requires the complementary use of different types of dictionaries.