Spoken Dialogue Interface Research Articles

Evaluation of natural language understanding based speech dialog interface's effectiveness regarding car navigation system usability performance

Recently, new cell phone services enable taking input by speaking to a quasi-agent using highly accurate speech recognition technologies. However, there are two problems when equipping a vehicle with these technologies. First, we do not understand yet the effect on driver’s usability performance of a car navigation system equipped with spoken dialogue interface. Second, the corresponding design techniques to obtain sufficiently good navigation results has not yet been determined. In this study, we first discuss problems involving speech recognition technologies and the types of speech interface used in this study. Second, we experimentally compare in pseudo-driving conditions the effect on driver’s physiology of various speech user interfaces: the Command Input (CI) interface, the Natural Language Understanding (NLU) interface, and the NLU/Spoken Dialogue Management (SDM) interface. According to the analysis of physiological signals, parasympathetic nervous system activity index value was higher when using the NLU/SDM interface than when using the CI interface. Furthermore, both subjective assessment result and time satisfaction degree of NLU/SDM interface were higher than others. Therefore, we can conclude that the NLU/SDM-type interface has better usability performance.

Designing a spoken dialogue interface to an intelligent cognitive assistant for people with dementia.

Intelligent cognitive assistants support people who need help performing everyday tasks by detecting when problems occur and providing tailored and context-sensitive assistance. Spoken dialogue interfaces allow users to interact with intelligent cognitive assistants while focusing on the task at hand. In order to establish requirements for voice interfaces to intelligent cognitive assistants, we conducted three focus groups with people with dementia, carers, and older people without a diagnosis of dementia. Analysis of the focus group data showed that voice and interaction style should be chosen based on the preferences of the user, not those of the carer. For people with dementia, the intelligent cognitive assistant should act like a patient, encouraging guide, while for older people without dementia, assistance should be to the point and not patronising. The intelligent cognitive assistant should be able to adapt to cognitive decline.

A Dynamic Spoken Dialogue Interface for Ambient Intelligence Interaction

In this paper, we present the interpretation and generation processes of a spoken dialogue interface for ambient intelligence. The interface is automatically created for each specific environment and the interpretation and generation vary depending on the environment and its context. These processes rely on a dialogue tree structure. Several modules process the tree structure and the context information to produce specific dialogues for the current environment state. The interface has been implemented and evaluated in an ambient intelligence environment. Satisfactory objective and subjective evaluation results are shown at the end of the paper.

Applying automated deduction to natural language understanding

Very few natural language understanding applications employ methods from automated deduction. This is mainly because (i) a high level of interdisciplinary knowledge is required, (ii) there is a huge gap between formal semantic theory and practical implementation, and (iii) statistical rather than symbolic approaches dominate the current trends in natural language processing. Moreover, abduction rather than deduction is generally viewed as a promising way to apply reasoning in natural language understanding. We describe three applications where we show how first-order theorem proving and finite model construction can efficiently be employed in language understanding. The first is a text understanding system building semantic representations of texts, developed in the late 1990s. Theorem provers are here used to signal inconsistent interpretations and to check whether new contributions to the discourse are informative or not. This application shows that it is feasible to use general-purpose theorem provers for first-order logic, and that it pays off to use a battery of different inference engines as in practice they complement each other in terms of performance. The second application is a spoken-dialogue interface to a mobile robot and an automated home. We use the first-order theorem prover spass for checking inconsistencies and newness of information, but the inference tasks are complemented with the finite model builder mace used in parallel to the prover. The model builder is used to check for satisfiability of the input; in addition, the produced finite and minimal models are used to determine the actions that the robot or automated house has to execute. When the semantic representation of the dialogue as well as the number of objects in the context are kept fairly small, response times are acceptable to human users. The third demonstration of successful use of first-order inference engines comes from the task of recognising entailment between two (short) texts. We run a robust parser producing semantic representations for both texts, and use the theorem prover vampire to check whether one text entails the other. For many examples it is hard to compute the appropriate background knowledge in order to produce a proof, and the model builders mace and paradox are used to estimate the likelihood of an entailment.

A spoken language interface with a mobile robot

We describe a spoken dialogue interface with a mobile robot, which a human can direct to specific locations, ask for information about its status, and supply information about its environment. The robot uses an internal map for navigation, and communicates its current orientation and accessible locations to the dialogue system. In this article, we focus on linguistic and inferential aspects of the human–robot communication process.

Interface for Barge-in Free Spoken Dialogue System Based on Sound Field Reproduction and Microphone Array

A barge-in free spoken dialogue interface using sound field control and microphone array is proposed. In the conventional spoken dialogue system using an acoustic echo canceller, it is indispensable to estimate a room transfer function, especially when the transfer function is changed by various interferences. However, the estimation is difficult when the user and the system speak simultaneously. To resolve the problem, we propose a sound field control technique to prevent the response sound from being observed. Combined with a microphone array, the proposed method can achieve high elimination performance with no adaptive process. The efficacy of the proposed interface is ascertained in the experiments on the basis of sound elimination and speech recognition.

Adaptive interface for spoken dialog

Speech has become increasingly important in human–computer interaction. Spoken dialog interfaces rely on automatic speech recognition, speech synthesis, language understanding, and dialog management. A main issue in dialog systems is that they typically are limited to pre-programmed vocabularies and sets of sentences. The research reported here focuses on developing an adaptive spoken dialog interface capable of acquiring new linguistic units and their corresponding semantics during the human–computer interaction. The adaptive interface identifies unknown words and phrases in the users utterances and asks the user for the corresponding semantics. The user can provide the meaning or the semantic representation of the new linguistic units through multiple modalities, including speaking, typing, pointing, touching, or showing. The interface then stores the new linguistic units in a semantic grammar and creates new objects defining the corresponding semantic representation. This process takes place during natural interaction between user and computer and, thus, the interface does not have to be rewritten and compiled to incorporate the newly acquired language. Users can personalize the adaptive spoken interface for different domain applications, or according to their personal preferences. [Work supported by NSF.]

Handling of user interruption to achieve timing‐free utterances for spoken dialogue interface

AbstractThis paper considers the spoken dialogue interface, which is the basic system for the multimodal interface by speech, and discusses the handling of the interruption that ensures the user the timing‐free utterances. In the interruption by the user, if the utterance by the system is designed with the sentence as the unit, as in the conventional system, there will be produced a difference between the designed content of the utterance and the actually uttered content or the content received by the user. From such a viewpoint, the following method is proposed.The unit for the utterance planning is defined as the information to be transmitted by a sentence, and which information is received is always monitored through the management of the turn‐taking between the speakers of the dialogue. As a result of experiment, the handling of smooth interruption is recognized in more than half the subjects. Using the proposed method, the difference between the utterance planned by the system and the utterance received by the user is reduced. It is verified also that the interface is made more convenient by the handling of the interruption. The time required for the user to complete the task is reduced by 7 percent, and the number of positive topic proposals is increased by 21 percent.

PEGASUS: A spoken dialogue interface for on-line air travel planning

This paper describes PEGASUS, a spoken dialogue interface for on-line air travel planning that we have recently developed. PEGASUS leverages off our spoken language technology development in the ATIS domain, and enables users to book flights using the American Airlines EAASY SABRE system. The input query is transformed by the speech understanding system to a frame representation that captures its meaning. The tasks of the System Manager include transforming the semantic representation into an EAASY SABRE command, transmitting it to the application backend, formatting and interpreting the resulting information, and managing the dialogue. Preliminary evaluation results suggest that users can learn to make productive use of PEGASUS for travel planning, although much work remains to be done.