Human-machine Dialogue Research Articles

Effects of system response delays on elderly humans\u2019 cognitive performance in a virtual training scenario

Observed influences of system response delay in spoken human-machine dialogues are rather ambiguous and mainly focus on perceived system quality. Studies that systematically inspect effects on cognitive performance are still lacking, and effects of individual characteristics are also often neglected. Building on benefits of cognitive training for decelerating cognitive decline, this Wizard-of-Oz study addresses both issues by testing 62 elderly participants in a dialogue-based memory training with a virtual agent. Participants acquired the method of loci with fading instructional guidance and applied it afterward to memorizing and recalling lists of German nouns. System response delays were randomly assigned, and training performance was included as potential mediator. Participants’ age, gender, and subscales of affinity for technology (enthusiasm, competence, positive and negative perception of technology) were inspected as potential moderators. The results indicated positive effects on recall performance with higher training performance, female gender, and less negative perception of technology. Additionally, memory retention and facets of affinity for technology moderated increasing system response delays. Participants also provided higher ratings in perceived system quality with higher enthusiasm for technology but reported increasing frustration with a more positive perception of technology. Potential explanations and implications for the design of spoken dialogue systems are discussed.

Practical Application of Ontologies

This article discusses the scope of computer ontologies. A classification of the main problems is given, for the solution of which it is advisable to use the ontological approach. Examples of the use of ontologies in various fields of human activity related to knowledge-based systems are given. The scope of application of computer ontologies is discussed in this article . A classification of the main problems is given, for the solution of which it is advisable to use the ontological approach. The areas of application of ontologies are identified as following:
 
 Designing knowledge-oriented systems, which refers to the creation, development, implementation, interaction with users of such objects as knowledge bases, databases and data warehouses. Besides, the same area includes the integration of existing information repositories, even with a very different data structure.
 Knowledge management (decision making systems).The usage of the ontological approach allows to raise information management technologies to a new level: to introduce the technology of providing explanations to the user about the management process, which can reveal the essence of the situations and their causes, as well as to justify the proposed action scenarios. That is, to provide human-machine dialogue that allows to communicate in a natural language.
 Semantic technology of information search and the ability to build search results using ontological classifiers. This use of ontologies enhances the relevance and continentity of search results. It provides contextual search capabilities, making the most of the natural language vocabulary display.
 Classification tasks: the classification structure (taxonomy) is an integral part of any ontology, we can talk about the presence of ontology elements in special classifications and indexing systems (for example, in library classification codes).
 Support for linguistically oriented information technologies, that is, providing solutions to complex text processing tasks by organizing a system of interactions of various levels of text processing, including the ontological level associated with the construction of a semantically adequate model of the subject area and various linguistic levels associated with linguistic properties and relations of subject objects area.
 Support for transdisciplinarity, namely the formation of common conceptual spaces, providing an adequate understanding of the information exchanged by experts in various fields of human activity.
 
 Examples of using ontologies in various spheres of human activity related to systems based on knowledge, such as education, the creation of the Semantic Web, etc are given.

Automatic classification of speech overlaps: Feature representation and algorithms

Overlapping speech is a natural and frequently occurring phenomenon in human–human conversations with an underlying purpose. Speech overlap events may be categorized as competitive and non-competitive. While the former is an attempt to grab the floor, the latter is an attempt to assist the speaker to continue the turn. The presence and distribution of these categories are indicative of the speakers’ states during the conversation. Therefore, understanding these manifestations is crucial for conversational analysis and for modeling human–machine dialogs. The goal of this study is to design computational models to classify overlapping speech segments of dyadic conversations into competitive vs. non-competitive acts using lexical and acoustic cues, as well as their surrounding context. The designed overlap representations are evaluated in both linear – Support Vector Machines (SVM) – and non-linear – feed-forward (FFNN), convolutional (CNN) and long short-term memory (LSTM) neural network – models. We experiment with lexical and acoustic representations and their combinations from both speaker channels in feature and hidden space. We observe that lexical word-embedding features significantly increase the overall F1-measure compared to both acoustic and bag-of-ngrams lexical representations, suggesting that lexical information can be utilized as a powerful cue for overlap classification. Our comparative study shows that the best computational architecture is an FFNN along with a combination of word embeddings and acoustic features.

Reports of the Workshops of the 32nd AAAI Conference on Artificial Intelligence

The AAAI-18 workshop program included 15 workshops covering a wide range of topics in AI. Workshops were held Sunday and Monday, February 2–7, 2018, at the Hilton New Orleans Riverside in New Orleans, Louisiana, USA. This report contains summaries of the Affective Content Analysis workshop; the Artificial Intelligence Applied to Assistive Technologies and Smart Environments; the AI and Marketing Science workshop; the Artificial Intelligence for Cyber Security workshop; the AI for Imperfect-Information Games; the Declarative Learning Based Programming workshop; the Engineering Dependable and Secure Machine Learning Systems workshop; the Health Intelligence workshop; the Knowledge Extraction from Games workshop; the Plan, Activity, and Intent Recognition workshop; the Planning and Inference workshop; the Preference Handling workshop; the Reasoning and Learning for Human-Machine Dialogues workshop; and the the AI Enhanced Internet of Things Data Processing for Intelligent Applications workshop.

Explorations in multiparty casual social talk and its relevance for social human machine dialogue

Much talk between humans is face-to-face, casual, multiparty, and of indefinite duration. Such casual conversation or social talk facilitates social bonding and mutual co-presence rather than strictly being used to exchange information in order to complete well-defined practical tasks. Artificial partners capable of participating as a speaker or listener in such talk would be useful for companionship, educational, and social contexts. However, to adequately model social talk, such applications require dialogue structure beyond simple question/answer routines. While there is a body of theory on multiparty casual talk, there is a lack of quantitative work in the area. Our work focuses on the anatomy of casual talk, in particular phases of chat, highly interactive dialogue exchanges, and chunks, longer contributions from single participants in the dialogue. We outline the current knowledge on the structure of casual talk and describe our investigations in this domain. Our research finds that distributions of the durations of chat and chunk phases vary with chat being shorter than chunk phases. Chat is also more common at the start of conversations, with chunks becoming more prominent as the conversation progresses. Laughter and overlap are more common in chat phases than chunk phases. We discuss how these insights can inform the design and implementation of truly social machine dialogue partners.

RETRACTED ARTICLE: Man-machine dialogue system optimization based on cloud computing

This paper studies the optimization and implementation of human-machine dialog system based on cloud computing technology. Firstly, the coarse-perceived hash generation based on the formant frequency is studied, and the detail-aware hash generation based on the energy difference in the time domain is studied. Then, the combination of coarse-perceived hash and detail-aware hash is elaborated. Secondly, the algorithm is tested and emulated. The algorithm matches the coarse-perceived hash sequence with fewer bits, and returns the voice number with similar rough features to the index voice after matching successfully, and matches the detail-aware hash sequence of the corresponding number voice. Then, the exact match of the result is gotten. Finally, it is concluded that the algorithm in this paper can effectively recognize the human voice.

Combining topic-based model and text categorisation approach for utterance understanding in human-machine dialogue

Combining topic-based model and text categorisation approach for utterance understanding in human-machine dialogue

Combining topic-based model and text categorisation approach for utterance understanding in human-machine dialogue

In the present paper, we suggest an implementation of an automatic understanding system of the statement in human-machine communication. The architecture we adopt is based on a stochastic approach that assumes that the understanding of a statement is nothing but a simple theme identification process. Therefore, we present a new theme identification method based on a documentary retrieval technique which is text (document) classification (Bawakid and Oussalah, 2010). The method we suggest was validated on a basic platform that gives information related to university schooling management (querying a student database), taking into consideration a textual input in French. This method has achieved a theme identification rate of 95% and a correct utterance understanding rate of about 91.66%.

Интерактивный специализатор подмножества языка Java, основанный на методе частичных вычислений

Specialization is a program optimization approach that implies the use of a priori information about values of some variables. Specialization methods are being developed since 1970s (mixed computations, partial evaluation, supercompilation). However, it is surprising, that even after three decades, these promising methods have not been put into the wide programming practice. One may wonder: What is the reason? Our hypothesis is that the task of specialization requires much greater human involvement into the specialization process, the analysis of its results and conducting computer experiments than in the case of common program optimization in compilers. Hence, specializers should be embedded into integrated development environments (IDE) familiar to programmers and appropriate interactive tools should be developed. In this paper we provide a work-in-progress report on results of development of an interactive specializer based on partial evaluation for a subset of the Java programming language. The specializer has been implemented within the popular Eclipse IDE. Scenarios of the human-machine dialogue with the specializer and interactive tools to compose the specialization task and to control the process of specialization are under development. An example of application of the current version of the specializer is shown. The residual program runs several times faster than the source one.

DREAM: Classification scheme for dialog acts in clinical research query mediation

Clinical data access involves complex but opaque communication between medical researchers and query analysts. Understanding such communication is indispensable for designing intelligent human–machine dialog systems that automate query formulation. This study investigates email communication and proposes a novel scheme for classifying dialog acts in clinical research query mediation. We analyzed 315 email messages exchanged in the communication for 20 data requests obtained from three institutions. The messages were segmented into 1333 utterance units. Through a rigorous process, we developed a classification scheme and applied it for dialog act annotation of the extracted utterances. Evaluation results with high inter-annotator agreement demonstrate the reliability of this scheme. This dataset is used to contribute preliminary understanding of dialog acts distribution and conversation flow in this dialog space.

Measuring the differences between human-human and human-machine dialogs

In this paper, we assess the applicability of user simulation techniques to generate dialogs which are similar to real human-machine spoken interactions.To do so, we present the results of the comparison between three corpora acquired by means of different techniques. The first corpus was acquired with real users.A statistical user simulation technique has been applied to the same task to acquire the second corpus. In this technique, the next user answer is selected by means of a classification process that takes into account the previous dialog history, the lexical information in the clause, and the subtask of the dialog to which it contributes. Finally, a dialog simulation technique has been developed for the acquisition of the third corpus. This technique uses a random selection of the user and system turns, defining stop conditions for automatically deciding if the simulated dialog is successful or not. We use several evaluation measures proposed in previous research to compare between our three acquired corpora, and then discuss the similarities and differences with regard to these measures.

Virtual training and coaching of health behavior: Example from mindfulness meditation training

ObjectiveComputer-based virtual coaches are increasingly being explored for patient education, counseling, and health behavior training and coaching. The objective of this research was to develop and evaluate a Virtual Mindfulness Coach for training and coaching in mindfulness meditation. MethodsThe coach was implemented as an embodied conversational character, providing mindfulness training and coaching via mixed initiative, text-based, natural language dialog with the student, and emphasizing affect–adaptive interaction. (The term ‘mixed initiative dialog’ refers to a human–machine dialog where either can initiate a conversation or a change in the conversation topic.) ResultsFindings from a pilot evaluation study indicate that the coach-based training is more effective in helping students establish a regular practice than self-administered training using written and audio materials. The coached group also appeared to be in more advanced stages of change in terms of the transtheoretical model, and have a higher sense of self-efficacy regarding establishment of a regular mindfulness practice. ConclusionThese results suggest that virtual coach-based training of mindfulness is both feasible, and potentially more effective, than a self-administered program. Of particular interest is the identification of the specific coach features that contribute to its effectiveness. Practice implicationsVirtual coaches could provide easily accessible and cost-effective customized training for a range of health behaviors. The affect–adaptive aspect of these coaches is particularly relevant for helping patients establish long-term behavior changes.

Research on Chamfer Software of Complex Parts

In order to meet the needs of enterprises for chamfering complex parts, based on the customization of commercial CAD/CAM software, chamfer programming software is developed. According to user’s machining demands for a part, a chain of edges of a part is extracted from its 3D model. With preprocessing of the chain of edges, the continuity of the chain is estimated, and the start and end point of those edges are automatic obtained. Furthermore, with human-machine dialogue, machining parameters is set by users. By definition of the primary and secondary surfaces of the chain of edges, and interpolation of the edges, the positions of cutter location point and postures of cutter are calculated. Finally the interference of tool path is checked, and tool path is simulated. The software solves the programming problem of chamfering complex parts.

Voice-based Human-Machine Interaction Modeling for Automated Information Services

Voice based human-machine dialogs are becoming more and more important part of informative services. The implementation of voice dialogs enables to realize some of the aims of telecommunication services more successfully and efficiently. The main aim is to enable the communication according the principle "anytime-anywhere". The importance of voice dialogs is also caused by the fact that principle "anytime-anywhere" often could be realized only using mobile and portable devices. Those devices typically have small keyboards and screens and hence voice based interface has advantages over traditional keyboard and screen based interface. The paper presents the model of multimodal interface which core element is the recognition of voice commands. The model targets the informative services provided by the Lithuanian medical and social security enterprises. Paper shows that recognition accuracy of Lithuanian voice commands could be increased significantly if the foreign language which has closer to Lithuanian phonetic structure engine is adapted. Ill. 2, bibl. 11, tabl. 1 (in English; abstracts in English and Lithuanian).http://dx.doi.org/10.5755/j01.eee.110.4.300

Phrase classes in two-level language models for ASR

In this work, we propose and compare two different approaches to a two-level language model. Both of them are based on phrase classes but they consider different ways of dealing with phrases into the classes. We provide a complete formulation consistent with the two approaches. The language models proposed were integrated into an Automatic Speech Recognition (ASR) system and evaluated in terms of Word Error Rate. Several series of experiments were carried out over a spontaneous human–machine dialogue corpus in Spanish, where users asked for information about long-distance trains by telephone. It can be extracted from the obtained results that the integration of phrases into classes when using the language models proposed leads to an improvement of the performance of an ASR system. Moreover, the obtained results seem to indicate that the history length with which the best performance is achieved is related to the features of the model itself. Thus, not all the models show the best results with the same value of history length.

Automatic annotation of context and speech acts for dialogue corpora

Richly annotated dialogue corpora are essential for new research directions in statistical learning approaches to dialogue management, context-sensitive interpretation, and context-sensitive speech recognition. In particular, large dialogue corpora annotated with contextual information and speech acts are urgently required. We explore how existing dialogue corpora (usually consisting of utterance transcriptions) can be automatically processed to yield new corpora where dialogue context and speech acts are accurately represented. We present a conceptual and computational framework for generating such corpora. As an example, we present and evaluate an automatic annotation system which builds ‘Information State Update’ (ISU) representations of dialogue context for the Communicator (2000 and 2001) corpora of human–machine dialogues (2,331 dialogues). The purposes of this annotation are to generate corpora for reinforcement learning of dialogue policies, for building user simulations, for evaluating different dialogue strategies against a baseline, and for training models for context-dependent interpretation and speech recognition. The automatic annotation system parses system and user utterances into speech acts and builds up sequences of dialogue context representations using an ISU dialogue manager. We present the architecture of the automatic annotation system and a detailed example to illustrate how the system components interact to produce the annotations. We also evaluate the annotations, with respect to the task completion metrics of the original corpus and in comparison to hand-annotated data and annotations produced by a baseline automatic system. The automatic annotations perform well and largely outperform the baseline automatic annotations in all measures. The resulting annotated corpus has been used to train high-quality user simulations and to learn successful dialogue strategies. The final corpus will be made publicly available.

Dialogue-based CALL: a case study on teaching pronouns

We describe a computer assisted language learning (CALL) system that uses human–machine dialogue as its medium of interaction. The system was developed to help students learn the basics of the Māori language and was designed to accompany the introductory course in Māori running at the University of Otago. The student engages in a task-based conversation with the CALL system, which is designed to provide practice in specific language constructions as well as to provide opportunities to negotiate meaning within the constraints of the target language constructions. The focus of the paper is on evaluating the CALL system with students. We began with a series of observations of a small number of students using an early version of the system. These real-use observations informed the development of the system which was used in a quanititative study with a different student cohort later in the year. Learning outcomes were assessed through written tests on the Māori system of personal pronouns. Our results show that students who attended the CALL tutorials scored at least as well as students who attended regular tutorials on immediate post-tests. The CALL tutorial groups scored less well on delayed post-tests and some possible reasons for this are discussed. Some caveats for the study are also discussed along with suggestions for further research.

Learning effective and engaging strategies for advice-giving human-machine dialogue

We describe a system that automatically learns effective and engaging dialogue strategies, generated from a library of dialogue content, using reinforcement learning from user feedback. Besides the more usual clarification and verification components of dialogue, this library contains various social elements like greetings, apologies, small talk, relational questions and jokes. We tested the method through an experimental dialogue system that encourages take-up of exercise and shows that the learned dialogue policy performs as well as one built by human experts for this system.

Multi-agent human–machine dialogue: issues in dialogue management and referring expression semantics

This paper presents an implemented model of dialogue management for simple dialogues involving multiple speakers. In our model, the user is one speaker, and the system ‘plays’ a number of other speakers. We present a number of principles governing dialogue management in such cases, which relate to turn-taking and the identification of the addressees of utterances. We also consider how to extend a syntactic and semantic treatment of first- and second-person personal pronouns, and of addressee terms, in order to deal with the multi-speaker scenario. We give some examples of our current system, and conclude by outlining some extensions of the system to include disagreements, interruptions, and private communication between subgroups of dialogue participants.

Towards a taxonomy of error-handling strategies in recognition-based multi-modal human–computer interfaces

In this paper, we survey the different types of error-handling strategies that have been described in the literature on recognition-based human–computer interfaces. A wide range of strategies can be found in spoken human–machine dialogues, handwriting systems, and multi-modal natural interfaces. We then propose a taxonomy for classifying error-handling strategies that has the following three dimensions: the main actor in the error-handling process (machine versus user), the purpose of the strategy (error prevention, discovery, or correction), and the use of different modalities of interaction. The requirements that different error-handling strategies have on different sets of interaction modalities are also discussed. The main aim of this work is to establish a classification that can serve as a tool for understanding how to develop more efficient and more robust multi-modal human–machine interfaces.