Dialogue Agent Research Articles

Natural language dialog system considering speaker’s emotion for open-ended conversation

To respond appropriately to an utterance, human-like communication system, should consider not only words in the utterance but also the speaker’s emotion. We thus proposed a natural language dialog system that can estimate the user’s emotion from utterances and respond on the basis of the estimated emotion. To estimate a speaker’s emotion (positive, negative, or neutral), 384 acoustic features extracted from an utterance are utilized by a Support Vector Machine (SVM). Artificial Intelligence Markup Language (AIML)-based response generating rules are expanded so that the speaker’s emotion can be considered as a condition of these rules. Two experiments were carried out to compare impressions of a dialog agent that considered emotion (proposed system) with those of an agent that did not (previous system). In the first experiment, 10 subjects evaluated the impressions after watch four conversation videos (no emotion estimation, correct emotion estimation, inadequate emotion estimation, and imperfect emotion estimation). In the second experiment, another 10 subjects evaluated the impressions after talk with both dialog agents. These experimental results and a demonstration of the proposed system will be shown in the presentation.

OK or not OK? Commitments in acknowledgments and corrections

While a semantics without differing “points of view” of different agents is a good first hypothesis for the analysis of the content of monologue, dialogues typically involve differing points of view from different agents. In particular one agent may not agree with what another agent asserts, or may have a different interpretation of an utterance from that of its author. An adequate semantics for dialogue should proceed by attributing to different dialogue agents separate views of the contents of their conversation. We model this, following others, by assigning each agent her own commitment slate. In this paper we bring out a complication with this approach that has gone so far unnoticed in formal semantics and the prior work we just mentioned, albeit it is well-known from epistemic game theory: commitment slates interact; agents typically commit to the fact that other agents make certain commitments. We thus formulate the semantics of dialogue moves and conversational goals in terms of nested, public commitments. We develop two semantics for nested commitments, one for a simple propositional language, the other for a full description language for the discourse structure of dialogues; and we show how one is an approximation of the other. We apply this formal setting to provide a unified account of different linguistic problems: the problem of ambiguity and the problem of acknowledgments and grounding. We also briefly discuss the problem of corrections and how to integrate them in our framework.

A Socratic epistemology for verbal emotional intelligence

We describe and experimentally validate a question-asking framework for machine-learned linguistic knowledge about human emotions. Using the Socratic method as a theoretical inspiration, we develop an experimental method and computational model for computers to learn subjective information about emotions by playing emotion twenty questions (EMO20Q), a game of twenty questions limited to words denoting emotions. Using human–human EMO20Q data we bootstrap a sequential Bayesian model that drives a generalized pushdown automaton-based dialog agent that further learns from 300 human–computer dialogs collected on Amazon Mechanical Turk. The human–human EMO20Q dialogs show the capability of humans to use a large, rich, subjective vocabulary of emotion words. Training on successive batches of human–computer EMO20Q dialogs shows that the automated agent is able to learn from subsequent human–computer interactions. Our results show that the training procedure enables the agent to learn a large set of emotion words. The fully trained agent successfully completes EMO20Q at 67% of human performance and 30% better than the bootstrapped agent. Even when the agent fails to guess the human opponent’s emotion word in the EMO20Q game, the agent’s behavior of searching for knowledge makes it appear human-like, which enables the agent to maintain user engagement and learn new, out-of-vocabulary words. These results lead us to conclude that the question-asking methodology and its implementation as a sequential Bayes pushdown automaton are a successful model for the cognitive abilities involved in learning, retrieving, and using emotion words by an automated agent in a dialog setting.

Real-time Language-Independent Algorithm for Dialogue Agents

Real-time Language-Independent Algorithm for Dialogue Agents

対話システムのパーソナリティを問う質問の大規模な収集と分析

対話システムのパーソナリティを問う質問の大規模な収集と分析

Redação jurídica objetiva: o juridiquês no banco dos réus

<p>Autores de docu­mentos jurídicos pecam repe­tidamente ao não fornecerem estrutura textual e vocabular satisfatória para o completo entendimento de seus textos, aceitando o perigo de serem mal compreendidos pelo leitor final. De fato, de um lado, o bacha­relismo produz verborragia e, de outro, minimaliza-se com o uso de siglas e expressões redu­zidas (jargão profissional). Na prática, essas estratégias não são eficazes, prejudicando uma comunicação clara, objetiva e, portanto, unívoca. Dentro desse contexto, parece prudente repensar o uso de certos hábitos linguísticos (formadores de juri­diquês), orientando-se à dimi­nuição de barreiras linguísticas entre os agentes de diálogo (operadores do Direito e público leigo). Apoiando essa causa, o presente artigo apresenta algumas sugestões, ancoradas em reflexões do movimento internacional <em>Plain Language, </em>que contribuem para uma redação jurídica mais objetiva.</p>

Shifting the Load: a Peer Dialogue Agent that Encourages its Human Collaborator to Contribute More to Problem Solving

Despite a growing need for educational tools that support students at the earliest phases of undergraduate Computer Science (CS) curricula, relatively few such tools exist–the majority being Intelligent Tutoring Systems. Since peer interactions more readily give rise to challenges and negotiations, another way in which students can become more interactive during problem solving, we created an artificial peer collaborator to determine its value for aiding CS students. Central to its development was the notion that it should monitor the student’s collaborative behavior and attempt to guide him/her towards more productive behavior. In prior work, we found that initiative shifts correlate with both Knowledge Co-Construction (KCC) and learning and are potentially easier to model as an indicator of productive collaboration in instructional software. In this paper, we describe a unique peer dialogue agent that we created to test the effects of tracking and reacting to initiative shifts. While our study did not find differences in learning gains when comparing agents that do and do not track and react to initiative shifts, we did find that students do learn when interacting with the agent and that attempting to influence initiative taking did make a difference. This suggests that by tracking initiative shifts, the agent was able to detect times when the student had been letting the agent do most of the “deep thinking” and that the agent’s tactics for encouraging the student to begin taking the initiative again were helpful.

Comprehension and engagement in survey interviews with virtual agents.

This study investigates how an onscreen virtual agent's dialog capability and facial animation affect survey respondents' comprehension and engagement in “face-to-face” interviews, using questions from US government surveys whose results have far-reaching impact on national policies. In the study, 73 laboratory participants were randomly assigned to respond in one of four interviewing conditions, in which the virtual agent had either high or low dialog capability (implemented through Wizard of Oz) and high or low facial animation, based on motion capture from a human interviewer. Respondents, whose faces were visible to the Wizard (and videorecorded) during the interviews, answered 12 questions about housing, employment, and purchases on the basis of fictional scenarios designed to allow measurement of comprehension accuracy, defined as the fit between responses and US government definitions. Respondents answered more accurately with the high-dialog-capability agents, requesting clarification more often particularly for ambiguous scenarios; and they generally treated the high-dialog-capability interviewers more socially, looking at the interviewer more and judging high-dialog-capability agents as more personal and less distant. Greater interviewer facial animation did not affect response accuracy, but it led to more displays of engagement—acknowledgments (verbal and visual) and smiles—and to the virtual interviewer's being rated as less natural. The pattern of results suggests that a virtual agent's dialog capability and facial animation differently affect survey respondents' experience of interviews, behavioral displays, and comprehension, and thus the accuracy of their responses. The pattern of results also suggests design considerations for building survey interviewing agents, which may differ depending on the kinds of survey questions (sensitive or not) that are asked.

Exploiting Psychological Factors for Interaction Style Recognition in Spoken Conversation

Determining how a speaker is engaged in a conversation is crucial for achieving harmonious interaction between computers and humans. In this study, a fusion approach was developed based on psychological factors to recognize Interaction Style ($IS$ ) in spoken conversation, which plays a key role in creating natural dialogue agents. The proposed Fused Cross-Correlation Model (FCCM) provides a unified probabilistic framework to model the relationships among the psychological factors of emotion, personality trait ($PT$), transient $IS$, and $IS$ history, for recognizing $IS$. An emotional arousal-dependent speech recognizer was used to obtain the recognized spoken text for extracting linguistic features to estimate transient $IS$ likelihood and recognize $PT$. A temporal course modeling approach and an emotional sub-state language model, based on the temporal phases of an emotional expression, were employed to obtain a better emotion recognition result. The experimental results indicate that the proposed FCCM yields satisfactory results in $IS$ recognition and also demonstrate that combining psychological factors effectively improves $IS$ recognition accuracy.

Korean anaphora recognition system to develop healthcare dialogue-type agent.

ObjectivesAnaphora recognition is a process to identify exactly which noun has been used previously and relates to a pronoun that is included in a specific sentence later. Therefore, anaphora recognition is an essential element of a dialogue agent system. In the current study, all the merits of rule-based, machine learning-based, semantic-based anaphora recognition systems were combined to design and realize a new hybrid-type anaphora recognition system with an optimum capacity.MethodsAnaphora recognition rules were encoded on the basis of the internal traits of referred expressions and adjacent contexts to realize a rule-based system and to serve as a baseline. A semantic database, related to predicate instances of sentences including referred expressions, was constructed to identify semantic co-relationships between the referent candidates (to which semantic tags were attached) and the semantic information of predicates. This approach would upgrade the anaphora recognition system by reducing the number of referent candidates. Additionally, to realize a machine learning-based system, an anaphora recognition model was developed on the basis of training data, which indicated referred expressions and referents. The three methods were further combined to develop a new single hybrid-based anaphora recognition system.ResultsThe precision rate of the rule-based systems was 54.9%. However, the precision rate of the hybrid-based system was 63.7%, proving it to be the most efficient method.ConclusionsThe hybrid-based method, developed by the combination of rule-based and machine learning-based methods, represents a new system with enhanced functional capabilities as compared to other pre-existing individual methods.

Special Issue on Intelligent Support for Learning in Groups

Advances in Intelligent Tutoring Systems (ITS) over the past two decades have enabled the development and deployment of highly effective learning technologies that support learners across a wide-range of domains and age-groups. On the other hand, research in collaborative learning has informed instructional approaches that leverage the pedagogical benefits of learning in groups (including dyads, small groups and teams). The use of these approaches in digital environments is being actively studied in the field of computer supported collaborative learning (CSCL). In addition to the increased use of computing devices in traditional classrooms, technological advancements such as mass access to revolutionary communication technologies have prompted the adoption of web-based tools for educational purposes. Educational service providers are deploying web-based platforms that allow students to access resources that supplement classroom instruction and allow students to share their learning experience with their peers. Hundreds of hours of high quality learning content and thousands of educational games that enrich learning interactions are accessible through mobile/tablet devices. Harmonious use of these resources and scientific developments offer opportunities for breaking new ground in the application of artificial intelligence (AI) technologies for education. Techniques developed in the field of AI for Education (AIEd) such as student/tutor/domain modelling, integrated assessment, tutorial dialog, automated scaffolding, data mining and pedagogical agents can further improve the effectiveness of learning in groups. On the other hand, multi-party interaction and massive online learning platforms present AIEd researchers with exciting next generation challenges. The IJAIEd Special Issue on Intelligent Support for Learning in Groups (ISLG) includes four articles that give us a brief peek at recent advances in some of these Int J Artif Intell Educ (2014) 24:1–7 DOI 10.1007/s40593-013-0013-5

A satisfaction-based model for affect recognition from conversational features in spoken dialog systems

Detecting user affect automatically during real-time conversation is the main challenge towards our greater aim of infusing social intelligence into a natural-language mixed-initiative High-Fidelity (Hi-Fi) audio control spoken dialog agent. In recent years, studies on affect detection from voice have moved on to using realistic, non-acted data, which is subtler. However, it is more challenging to perceive subtler emotions and this is demonstrated in tasks such as labeling and machine prediction. This paper attempts to address part of this challenge by considering the role of user satisfaction ratings and also conversational/dialog features in discriminating contentment and frustration, two types of emotions that are known to be prevalent within spoken human-computer interaction. However, given the laboratory constraints, users might be positively biased when rating the system, indirectly making the reliability of the satisfaction data questionable. Machine learning experiments were conducted on two datasets, users and annotators, which were then compared in order to assess the reliability of these datasets. Our results indicated that standard classifiers were significantly more successful in discriminating the abovementioned emotions and their intensities (reflected by user satisfaction ratings) from annotator data than from user data. These results corroborated that: first, satisfaction data could be used directly as an alternative target variable to model affect, and that they could be predicted exclusively by dialog features. Second, these were only true when trying to predict the abovementioned emotions using annotator’s data, suggesting that user bias does exist in a laboratory-led evaluation.

Co-constructing Grounded Symbols—Feedback and Incremental Adaptation in Human–Agent Dialogue

Grounding in dialogue concerns the question of how the gap between the individual symbol systems of interlocutors can be bridged so that mutual understanding is possible. This problem is highly relevant to human–agent interaction where mis- or non-understanding is common. We argue that humans minimise this gap by collaboratively and iteratively creating a shared conceptualisation that serves as a basis for negotiating symbol meaning. We then present a computational model that enables an artificial conversational agent to estimate the user’s mental state (in terms of contact, perception, understanding, acceptance, agreement and based upon his or her feedback signals) and use this information to incrementally adapt its ongoing communicative actions to the user’s needs. These basic abilities are important to reduce friction in the iterative coordination process of co-constructing grounded symbols in dialogue.

Giving a Personality to a Spoken Dialogue Agent by its Self-disclosures

Giving a Personality to a Spoken Dialogue Agent by its Self-disclosures

Reasoning dynamically about what one says

In this article we make sdrt’s glue logic for computing logical form dynamic. This allows us to model a dialogue agent’s understanding of what the update of the semantic representation of the dialogue would be after his next contribution, including the effects of the rhetorical moves that he is contemplating performing next. This is a pre-requisite for developing a model of how agents reason about what to say next. We make the glue logic dynamic by using a dynamic public announcement logic (pal). We extend pal with a particular variety of default reasoning suited to reasoning about discourse—this default reasoning being an essential component of inferring the pragmatic effects of one’s dialogue moves. We add to the pal language a new type of announcement, known as ceteris paribus announcement, and this is used to model how an agent anticipates the (default) pragmatic effects of his next dialogue move. Our extended pal validates certain intuitive patterns of default inference that existing pals for practical reasoning do not. We prove that the dynamic glue logic has a pspace validity problem, and as such is no more complex than pal with multiple \({\square}\) operators.

Creating a general collaborative dialogue agent with lounge strategy feature

In this paper, we focus on an agent-based architecture for task-oriented dialogue management. We present our deliberation process based on optimizing the length of a dialogue in terms of its turns needed to accomplish a task. As innovative features, the manager accommodates a “lounge strategy” for passing initiative back to the user, and a two-layered structure for dialogue context representation. We show that applying the lounge strategy during a dialogue leads to improving the information exchange rate by approximately 5% compared to “common” dialogue strategies alternative. Furthermore, we present two of our dialogues and analyze them with respect to the agent description. At the end of the paper, we suggest future extensions and modifications to the architecture, and conclude.

A formal model of emotions for an empathic rational dialog agent

Recent research has shown that virtual agents expressing empathic emotions toward users have the potential to enhance human---machine interaction. To provide empathic capabilities to a rational dialog agent, we propose a formal model of emotions based on an empirical and theoretical analysis of the users' conditions of emotion elicitation. The emotions are represented by particular mental states of the agent, composed of beliefs, uncertainties and intentions. This semantically grounded formal representation enables a rational dialog agent to identify from a dialogical situation the empathic emotion that it should express. An implementation and an evaluation of an empathic rational dialog agent have enabled us to validate the proposed model of empathy.

Assessing debate strategies via computational agents

This paper reports our research concerning dialogue strategies suitable for adoption by a human–computer debating system. We propose a set of strategic heuristics for a computer to adopt to enable it to function as a dialogue participant. In particular, we consider means of assessing the proposed strategy. A system involving two agents in dialogue with each other and a human–agent debate system are constructed and subsequently used to facilitate the evaluations. The evaluations suggest that the proposed strategy can enable the computer to act as an effective dialogue participant. It is anticipated that this work will contribute towards the development of computerised dialogue systems and help to illuminate research issues concerning strategies in dialectical systems.

Development and evaluation of intelligent machine tools based on knowledge evolution in M2M environment

In the near future, the foreseen improvement in machine tools will be in the form of a knowledge evolution-based intelligent device. The goal of this study is to develop intelligent machine tools having knowledge-evolution capability in Machine to Machine (M2M) wired and wireless environment. The knowledge evolution-based intelligent machine tools are expected to be capable of gathering knowledge autonomously, producing knowledge, understanding knowledge, applying reasoning to knowledge, making new decisions, dialoguing with other machines, etc. The concept of the knowledge-evolution intelligent machine originated from the process of machine control operation by the sense, dialogue and decision of a human expert. The structure of knowledge evolution in M2M and the scheme for a dialogue agent among agent-based modules such as a sensory agent, a dialogue agent and an expert system (decision support agent) are presented in this paper, and work-offset compensation from thermal change and recommendation of cutting condition are performed on-line for knowledge-evolution verification.

Evaluation of a hierarchical reinforcement learning spoken dialogue system

We describe an evaluation of spoken dialogue strategies designed using hierarchical reinforcement learning agents. The dialogue strategies were learnt in a simulated environment and tested in a laboratory setting with 32 users. These dialogues were used to evaluate three types of machine dialogue behaviour: hand-coded, fully-learnt and semi-learnt. These experiments also served to evaluate the realism of simulated dialogues using two proposed metrics contrasted with ‘Precision-Recall’. The learnt dialogue behaviours used the Semi-Markov Decision Process (SMDP) model, and we report the first evaluation of this model in a realistic conversational environment. Experimental results in the travel planning domain provide evidence to support the following claims: (a) hierarchical semi-learnt dialogue agents are a better alternative (with higher overall performance) than deterministic or fully-learnt behaviour; (b) spoken dialogue strategies learnt with highly coherent user behaviour and conservative recognition error rates (keyword error rate of 20%) can outperform a reasonable hand-coded strategy; and (c) hierarchical reinforcement learning dialogue agents are feasible and promising for the (semi) automatic design of optimized dialogue behaviours in larger-scale systems.