Information-seeking Dialogues Research Articles

Information-seeking dialogue for explainable artificial intelligence: Modelling and analytics

Explainable artificial intelligence has become a vitally important research field aiming, among other tasks, to justify predictions made by intelligent classifiers automatically learned from data. Importantly, efficiency of automated explanations may be undermined if the end user does not have sufficient domain knowledge or lacks information about the data used for training. To address the issue of effective explanation communication, we propose a novel information-seeking explanatory dialogue game following the most recent requirements to automatically generated explanations. Further, we generalise our dialogue model in form of an explanatory dialogue grammar which makes it applicable to interpretable rule-based classifiers that are enhanced with the capability to provide textual explanations. Finally, we carry out an exploratory user study to validate the corresponding dialogue protocol and analyse the experimental results using insights from process mining and argument analytics. A high number of requests for alternative explanations testifies the need for ensuring diversity in the context of automated explanations.

FaithDial: A Faithful Benchmark for Information-Seeking Dialogue

Abstract The goal of information-seeking dialogue is to respond to seeker queries with natural language utterances that are grounded on knowledge sources. However, dialogue systems often produce unsupported utterances, a phenomenon known as hallucination. To mitigate this behavior, we adopt a data-centric solution and create FaithDial, a new benchmark for hallucination-free dialogues, by editing hallucinated responses in the Wizard of Wikipedia (WoW) benchmark. We observe that FaithDial is more faithful than WoW while also maintaining engaging conversations. We show that FaithDial can serve as training signal for: i) a hallucination critic, which discriminates whether an utterance is faithful or not, and boosts the performance by 12.8 F1 score on the BEGIN benchmark compared to existing datasets for dialogue coherence; ii) high-quality dialogue generation. We benchmark a series of state-of-the-art models and propose an auxiliary contrastive objective that achieves the highest level of faithfulness and abstractiveness based on several automated metrics. Further, we find that the benefits of FaithDial generalize to zero-shot transfer on other datasets, such as CMU-Dog and TopicalChat. Finally, human evaluation reveals that responses generated by models trained on FaithDial are perceived as more interpretable, cooperative, and engaging.

A Large-scale Analysis of Mixed Initiative in Information-Seeking Dialogues for Conversational Search

Conversational search is a relatively young area of research that aims at automating an information-seeking dialogue. In this article, we help to position it with respect to other research areas within conversational artificial intelligence (AI) by analysing the structural properties of an information-seeking dialogue. To this end, we perform a large-scale dialogue analysis of more than 150K transcripts from 16 publicly available dialogue datasets. These datasets were collected to inform different dialogue-based tasks including conversational search. We extract different patterns of mixed initiative from these dialogue transcripts and use them to compare dialogues of different types. Moreover, we contrast the patterns found in information-seeking dialogues that are being used for research purposes with the patterns found in virtual reference interviews that were conducted by professional librarians. The insights we provide (1) establish close relations between conversational search and other conversational AI tasks and (2) uncover limitations of existing conversational datasets to inform future data collection tasks.

Multi-Stage Conversational Passage Retrieval: An Approach to Fusing Term Importance Estimation and Neural Query Rewriting

Conversational search plays a vital role in conversational information seeking. As queries in information seeking dialogues are ambiguous for traditional ad hoc information retrieval (IR) systems due to the coreference and omission resolution problems inherent in natural language dialogue, resolving these ambiguities is crucial. In this article, we tackle conversational passage retrieval, an important component of conversational search, by addressing query ambiguities with query reformulation integrated into a multi-stage ad hoc IR system. Specifically, we propose two conversational query reformulation (CQR) methods: (1) term importance estimation and (2) neural query rewriting. For the former, we expand conversational queries using important terms extracted from the conversational context with frequency-based signals. For the latter, we reformulate conversational queries into natural, stand-alone, human-understandable queries with a pretrained sequence-to-sequence model. Detailed analyses of the two CQR methods are provided quantitatively and qualitatively, explaining their advantages, disadvantages, and distinct behaviors. Moreover, to leverage the strengths of both CQR methods, we propose combining their output with reciprocal rank fusion, yielding state-of-the-art retrieval effectiveness, 30% improvement in terms of NDCG@3 compared to the best submission of Text REtrieval Conference (TREC) Conversational Assistant Track (CAsT) 2019.

Optimization of Information-Seeking Dialogue Strategy for Argumentation-Based Dialogue System

Argumentation-based dialogue systems, which can handle and exchange arguments through dialogue, have been widely researched. It is required that these systems have sufficient supporting information to argue their claims rationally; however, the systems often do not have enough of such information in realistic situations. One way to fill in the gap is acquiring such missing information from dialogue partners (information-seeking dialogue). Existing information-seeking dialogue systems are based on handcrafted dialogue strategies that exhaustively examine missing information. However, the proposed strategies are not specialized in collecting information for constructing rational arguments. Moreover, the number of system's inquiry candidates grows in accordance with the size of the argument set that the system deal with. In this paper, we formalize the process of information-seeking dialogue as Markov decision processes (MDPs) and apply deep reinforcement learning (DRL) for automatically optimizing a dialogue strategy. By utilizing DRL, our dialogue strategy can successfully minimize objective functions, the number of turns it takes for our system to collect necessary information in a dialogue. We conducted dialogue experiments using two datasets from different domains of argumentative dialogue. Experimental results show that the proposed formalization based on MDP works well, and the policy optimized by DRL outperformed existing heuristic dialogue strategies.

Polar questions, social actions and epistemic stance

AbstractThis article describes the relationships between epistemic stance, social actions and form variants of polar questions in Estonian institutional information-seeking dialogues and compares the results with ordinary conversation in English, Dutch, and Danish. Estonian and English use a polarized system where the unknowing stance is formulated by particles and/or inversion, and knowing stance by declaratives and sentence-final markers. Dutch and Danish use an asymmetrical system where unknowing stance uses an inversion and knowing stance uses frequently all question formats. Information-seeking dialogues contain a large number of questions used to establish resources necessary to implement the second-pair-part of the adjacency pair in the interaction.

Computer-assisted safety argument review – a dialectics approach

There has been increasing use of argument-based approaches in the development of safety-critical systems. Within this approach, a safety case plays a key role in the system development life cycle. The key components in a safety case are safety arguments, which are designated to demonstrate that the system is acceptably safe. Inappropriate reasoning in safety arguments could undermine a system's safety claims which in turn contribute to safety-related failures of the system. The review of safety arguments is therefore a crucial step in the development of safety-critical systems. Reviews are conducted using dialogues where elements of the argument and their relations are proposed and scrutinised. This paper investigates an approach of conducting argument review using dialectical models. After studying five established dialectical models with varying strengths and drawbacks, a new dialectical model specially designed to support persuasion and information-seeking dialogues has been proposed to suit the requir...

Business Intelligence Expert System on SOX Compliance over the Purchase Orders Creation Process

The objective of this work is to define a decision support system over SOX (Sarbanes-Oxley Act) compatibility and quality of the Purchase Orders Creation Process based on Artificial Intelligence and Theory of Argumentation knowledge and techniques. This proposed model directly contributes to both scientific research artificial intelligent area and business practices. From business perspective it empowers the use of artificial intelligent models and techniques to drive decision making processes over financial statements. From scientific and research area the impact is based on the combination of 1) an Information Seeking Dialog Protocol in which a requestor agent inquires the business case, 2) a Facts Valuation based Protocol in which the previously gathered facts are analyzed, 3) the already incorporated initial knowledge of a human expert via initial beliefs, 4) the Intra-Agent Decision Making Protocol based on deductive argumentation and 5) the semi automated Dynamic Knowledge Learning Protocol. Last but not least the suggested way of integration of this proposed model in a higher level multiagent intelligent system in which a Joint Deliberative Dialog Protocol and an Inter-Agent Decision Deductive Argumentation Making Protocol are described.

Interactive question answering and constraint relaxation in spoken dialogue systems

AbstractWe explore the relationship between question answering and constraint relaxation in spoken dialogue systems and develop dialogue strategies for selecting and presenting information succinctly. In particular, we describe methods for dealing with the results of database queries in information-seeking dialogues. Our goal is to structure the dialogue in such a way that the user is neither overwhelmed with information nor left uncertain as to how to refine the query further. We present two sets of evaluation results for a restaurant selection task: one is a system performance evaluation experiment involving twenty subjects, the other is an experimental evaluation of the use of suggestions involving sixteen subjects.

Comments on ‘Arguing ‘for’ the Patient. Informed Consent and Strategic Maneuvering in Doctor–Patient Interaction’

Schulz and Rubinelli’s project ‘Informed consent and strategic maneuvering in doctor–patient interaction’ provides an excellent opportunity for studying argumentation in a specific institutional context because a medical consultation is a special communicative activity type that may involve argumentative discussion. Before engaging in empirical research regarding such a consultation it is necessary to make a conceptional analysis of this type of doctor–patient interaction. One first needs to give a general characterization of the type of interaction concerned: what is the structure of the interaction in a doctor–patient consultation in terms of speech acts, role taking and time constraints? For doing so a better understanding is required of the type of difference of opinion that will be at issue in such a consultation. What type of standpoint initiates the discussion? Which parts can be distinguished in the activity type of medical consultation and which of them are typically or potentially argumentative? What are the roles of the two participants in each of these cases? Is it the doctor or the patient who initiates the discussion by putting forward a standpoint or can this be done by either of them? Schulz and Rubinelli characterize doctor–patient interaction as an ‘info-suasive dialogue.’ This characterization is, however, problematic. The authors claim that the interaction partly can be seen as an instance of an information-seeking dialogue and partly an argumentative encounter, ‘at a higher level.’ But in a consultation argumentation is not necessarily required. The patient may be in complete agreement with everything the doctor says and the doctor may not expect the patient to disagree with him so that no anticipation of doubt is necessary. Since argumentation is not a constitutive part of this activity type it is not automatically a ‘persuasive’ type of dialogue. Of course the moment one the parties has reason to believe that the other party is not or will not be in full agreement with him,

Grouping Utterances in Spoken Dialogues

In this paper, the interaction patterns and their automatic analysis in spontaneous spoken information-seeking dialogues are studied. First, based on previous work from discourse analysis (i.e., exchange as basic interaction unit in Birmingham School) and Sytemic Functional Grammar (i.e., Halliday's speech function), a principled scheme is proposed to model interaction patterns with utterance groups. Then a dialogue corpus is annotated with this scheme and further analyzed. Some main factors affecting the structure of utterance group are distinguished. Based on these, an algorithm is established to analyze utterance groups and is evaluated in the corpus. The results achieve a correct rate of 55.4%~84.2% for overall utterance tags, depending on the different recognition performances of the extended sentence type and utterance topic.

Dialogue Systems as Proof Editors

This paper shows how a dialogue system for information-seeking dialogues can be implemented in a type-theory-based syntax editor, originally developed for editing mathematical proofs. The implementation gives a simple logical metatheory to such dialogue systems and also suggests new functions for them, e.g., a local undo operation. The method developed provides a logically based declarative way of implementing simple dialogue systems that is easy to port to new domains.

Expert Advice and Argumentation: Some Remarks on the Work of Douglas Walton

Appeal to expert judgement has become a wide-spread and unavoidable element in public debates in modern society. The many and fundamental argumentative complications that they raise have not received proportional attention in argumentation studies so far. A prominent exception is a recent book by Douglas Walton, devoted entirely to arguments involving expert opinion (Walton, 1997). Confronting some examples from the field of Science and Society with Walton's earlier work, the need can be traced for a more elaborate and sophisticated treatment of the many issues involved, particularly for the model of information seeking dialogue and the treatment of source reasoning. With these issues as points of reference, it is examined in what way this challenge has been taken up in the new book. A few further additions are suggested.

Generating queries and replies during information-seeking interactions

Analysis of naturally occurring information-seeking dialogues indicates that they usually consist of a number of distinct discourse segments, such as a greeting segment, a request issued by a user, an optional clarification segment, a transfer of information segment and a final closing segment. The clarification interaction is often initiated by the information provider and it may be due to one of the following reasons: (1) there is confusion regarding the user's intentions, (2) there is insufficient information to formulate a plan to satisfy a recognized intention, or (3) there is difficulty in formulating a plan that satisfies a recognized intention. Once the information provider determines the user's intention and formulates a plan to achieve this intention, the information transfer phase is initiated to inform the user about the proposed plan.In this paper, we present a mechanism for generating queries during the clarification stage and answers during the information transfer stage. Given a hierarchical representation of the alternatives possibly intended by a user and the probabilities of these alternatives, our mechanism determines the hierarchy level at which a query must be directed and the query to be posed in order to determine the alternative intended by the user. Once the user's intentions are ascertained, the mechanism determines whether additional information is required and the manner in which queries may be posed to acquire this information. When a user's intentions cannot be satisfied by means of a single plan, our mechanism enters into a negotiation process to alter the user's specifications until a valid plan is formulated. In the final stages of the interaction, the mechanism determines the information to be transferred and generates an answer to effect the transfer. The mechanisms for negotiation and for the generation of queries and answers described in this paper have been implemented in a system calledRADAR, a computerized information providing system that functions as a travel agent.

Using dynamic user models in the recognition of the plans of the user

This paper is concerned with information-seeking dialogues in a restricted domain (we consider a consultation system for a Computer Science Department, delivering information about the various tasks that the users may want to perform: for example, how to access the library, get information about the courses of the Department, etc.) and presents a framework where a plan recognition and a user modeling component are integrated to cooperate in the task of identifying the user's plans and goals. The focus of the paper is centered on the techniques used for building the user model and exploiting it in the determination of the user's intentions. For this task, we use stereotypes and we propose some inference rules for expanding the user model by inferring the user's beliefs from both the sentences s/he utters and the information stored in the plan library of the system, that describes the actions in the domain. Moreover, we introduce some disambiguation rules that are applied to the information in the user model for restricting the set of ambiguous hypotheses on the user's plans and goals to the most plausible ones. This also simplifies a further clarification dialogue if it is necessary for a precise identification of the user's intentions.

Unified logic programming approach to the abduction of plans and intentions in information-seeking dialogues

We propose a framework that supports the recognition of plans and intentions behind speech acts through abductive inferences over discourse sentences. These inferences allow each agent to have an active and intelligent participation in dialogues, namely, in cooperative information-seeking dialogues. In our framework, the possible actions, events, states, and world knowledge are represented by extended logic programs (LP with explicit negation), and the abductive inference porcess is modeled by the framework proposed by Pereira et al. [13], which is based on the Well Founded Semantics augmented with explicit negation (WFSX) and contradiction removal semantics (CRSX). It will be shown how this framework supports abductive planning with Event Calculus [5], and some examples will be shown [10, 14] in the domain of information-seeking dialogues. Finally, some open problems will be pointed out.

Cases, scripts, and information-seeking strategies: On the design of interactive information retrieval systems

The support of effective interaction of the user with the other components of the system is a central problem for information retrieval. In this paper, we present a theory of such interactions taking place within a space of information-seeking strategies, and discuss how such a concept can be used to design for effective interaction. In particular, we propose a model of information retrieval system design based on the ideas of: a multidimensional space of information-seeking strategies; dialogue structures for information seeking; cases of specific information-seeking dialogues; anti, scripts as distinguished prototypical cases. We demonstrate the use of this model by discussing in some detail the MERIT system, a prototype information retrieval system, that incorporates these design principles.

A classification of ellipsis based on a corpus of information seeking dialogues

The standard classification of ellipsis has determined the way it is handled in natural language understanding (NLU) systems. This work provides a novel classification of ellipsis based on the analysis of ellipsis usage rather than forms in a corpus of information seeking dialogues. The aim is to demonstrate that pragmatic analysis is necessary for the interpretation of ellipsis. The context, in terms of the dialogue participants' belief states, determines interpretation and in turn the interpretation of subsequent utterances. The dialogues produced in a NLU system using this classification are presented.

A meta-rule approach to flexible plan recognition in dialogue

Although a number of researchers have demonstrated that reasoning on a model of the user's plans and goals is helpful in language understanding and response generation, current models of plan inference cannot handle naturally occurring dialogue. This paper argues that model building from less than ideal dialogues has a great deal in common with processing ill-formed input. It defines well-formedness constraints for information-seeking dialogues and contends that strategies for interpreting ill-formed input can be applied to the problem of modeling the user's plan during an ill-formed dialogue. It presents a meta-rule approach for hypothesizing the cause of dialogue ill-formedness, and describes meta-rules for relaxing the plan inference process and enabling the consideration of alternative hypotheses. The advantages of this approach are that it provides a unified framework for handling both well-formed and ill-formed dialogue, avoids unnatural interpretations when the dialogue is proceeding smoothly, and facilitates a nonmonotonic plan recognition system.

Modeling the illocutionary aspects of information-seeking dialogues

A dialogue model is presented that incorporates the illocutionary aspects of information-seeking dialogues. Directive, commissive, and assertive types of dialogue acts (e.g., asking, offering, rejecting, answering, and evaluating) are represented by a complex transition network. The model determines all legitimate types and sequences of dialogue acts and regulates the role assignments (e.g., when the information seeker and the information provider temporarily exchange their roles). Finally, an approach to integrating the illocutionary layer with other layers—which deal with thematical and rhetorical coherence—is outlined. Such an integrated model can serve as a basis for the design of human-computer dialogue systems.