Dialogue Management System Research Articles

An Approach Towards to Real Time AI Desktop Voice Assistant

The advent of artificial intelligence (AI) has revolutionized human-computer interaction, making it more natural and intuitive. This research paper presents the development and implementation of an AI Desktop Voice Assistant designed to enhance productivity and accessibility for users. The voice assistant leverages advanced speech recognition, natural language processing (NLP), and machine learning techniques to understand and execute user commands. Key functionalities include voice-activated application control, web searches, personalized reminders, and real-time information retrieval. Our system integrates with widely-used APIs and services, providing a seamless user experience across various tasks. The paper discusses the architecture of the voice assistant, detailing the integration of components such as the speech recognition engine, NLP models, and the dialogue management system. We explore the challenges encountered during development, including accurate speech recognition in noisy environments and handling ambiguous user commands. Solutions implemented to address these challenges are also presented. Furthermore, we conduct a usability study to evaluate the effectiveness and user satisfaction of the voice assistant. The results indicate a high level of user engagement and satisfaction, demonstrating the practical benefits of incorporating AI-driven voice interfaces into desktop environments. Our findings contribute to the ongoing research in human-computer interaction, suggesting pathways for future enhancements in AI voice assistants..

A study of interactive robot architecture through the practical implementation of conversational android.

This study shows an autonomous android robot that can have a natural daily dialogue with humans. The dialogue system for daily dialogue is different from a task-oriented dialogue system in that it is not given a clear purpose or the necessary information. That is, it needs to generate an utterance in a situation where there is no clear request from humans. Therefore, to continue a dialogue with a consistent content, it is necessary to essentially change the design policy of dialogue management compared with the existing dialogue system. The purpose of our study is to constructively find out the dialogue system architecture for realizing daily dialogue through implementing an autonomous dialogue robot capable of daily natural dialogue. We defined the android’s desire necessary for daily dialogue and the dialogue management system in which the android changes its internal (mental) states in accordance to the desire and partner’s behavior and chooses a dialogue topic suitable for the current situation. The developed android could continue daily dialogue for about 10 min in the scene where the robot and partner met for the first time in the experiment. Moreover, a multimodal Turing test has shown that half of the participants had felt that the android was remotely controlled to some degree, that is, the android’s behavior was humanlike. This result suggests that the system construction method assumed in this study is an effective approach to realize daily dialogue, and the study discusses the system architecture for daily dialogue.

An Open Source Virtual Reality Training Framework for the Criminal Justice System

We developed an open-source training framework to practice conversation skills in a controlled and immersive virtual reality (VR) environment. Virtual characters with different biographies were developed with which a conversation using natural language is possible. The virtual characters integrate a dialog management system (ChatScript) to provide different biographical memories. Natural language processing for the German language is integrated by using Kaldi, an open-source speech recognition toolkit. As the framework allows for interchangeable content there are many different possible application cases to apply within the criminal justice system. The VR frameworks code is available under an open-source license. In this article, an overview of the framework’s functionality is given as well as an outlook on possible areas of application. Statements about user acceptance and usability cannot yet be made, as relevant data have first to be gathered through a concrete application case.

Dialog management system based on user persona

Natural language processing (NLP) components are responsible for analysing and contextualising human-like discussions between chatbots or any voice browser or with any live users are known as dialogue management systems (DMS). Dialog management systems, also known as plug-ins, allow the chatbot to complete this functionality with ease. The dialogue management system features a module called the agent for dialogue management that allows the DMS to contextualise information and deliver replies. Chatter-bots frequently employ dialogue management systems, such as ChatScript, to regulate the conversation structure based on themes. In the developed application which emulates the behaviour of a DMS, the functionalities like voice assisted navigation, functional keys implementation, language neutral search are implemented. The system is developed by taking into consideration of user experience as the primary factor. The system facilitates physically disabled users to perform all the above mentioned functionalities using voice commands with approx.90% of accuracy.

Development of Dialogue Management System for Banking Services

Rapid increase in conversational AI and user chat data lead to intensive development of dialogue management systems (DMS) for various industries. Yet, for low-resource languages, such as Azerbaijani, very little research has been conducted. The main purpose of this work is to experiment with various DMS pipeline set-ups to decide on the most appropriate natural language understanding and dialogue manager settings. In our project, we designed and evaluated different DMS pipelines with respect to the conversational text data obtained from one of the leading retail banks in Azerbaijan. In the work, the main two components of DMS—Natural language Understanding (NLU) and Dialogue Manager—have been investigated. In the first step of NLU, we utilized a language identification (LI) component for language detection. We investigated both built-in LI methods such as fastText and custom machine learning (ML) models trained on the domain-based dataset. The second step of the work was a comparison of the classic ML classifiers (logistic regression, neural networks, and SVM) and Dual Intent and Entity Transformer (DIET) architecture for user intention detection. In these experiments we used different combinations of feature extractors such as CountVectorizer, Term Frequency-Inverse Document Frequency (TF-IDF) Vectorizer, and word embeddings for both word and character n-gram based tokens. To extract important information from the text messages, Named Entity Extraction (NER) component was added to the pipeline. The best NER model was chosen among conditional random fields (CRF) tagger, deep neural networks (DNN), models and build in entity extraction component inside DIET architecture. Obtained entity tags fed to the Dialogue Management module as features. All NLU set-ups were followed by the Dialogue Management module that contains a Rule-based Policy to handle FAQs and chitchats as well as a Transformer Embedding Dialogue (TED) Policy to handle more complex and unexpected dialogue inputs. As a result, we suggest a DMS pipeline for a financial assistant, which is capable of identifying intentions, named entities, and a language of text followed by policies that allow generating a proper response (based on the designed dialogues) and suggesting the best next action.

An Interactive Dialogue Management System for Spoken Language

The spoken dialogue system is one of the most important human-machine communication ways. Human-machine communication can be described as an interaction between the user and the computer. This field is full of research points, so it is considered a good attractive environment for many researchers. The spoken dialogue system is of great importance in the process of communicating commercial applications, and facilitating the connecting process between the human and machine which may take different faces. The main objective of this research will be building an interactive dialogue management system for spoken dialogue system in an ideal way, By answering the following main question: How can we build an interactive dialogue management system for spoken dialogue system in an ideal way has the ability to accomplish the Naturalness, Usability, Mixed initiative, Co-operativity, Robustness, and Exploration. This research will be a mixed-method research and will adopt a descriptive survey design in collecting information by Survey questionnaires, Interviews to a sample of the target population, and while secondary data will be found from books, journals, and The Internet. The most important conclusion of the research is the spoken dialogue system is to be less complexity and use uncertainty model; this way must be acceptable by the user and the system itself.

Human-Robot Interaction Based on Dialog Management Using Sentence Similarity Comparison Method

Advances in developing dialogue systems regarding speech recognition, language understanding, and speech synthesis. Dialogue systems to support human interaction with a robot efficiently by using spoken language. Facilities that provide convenience in carrying out daily activities for someone, such as older people, are necessary. The existence of Human-Robot Interaction (HRI), so that this interaction can give orders to the robot to do work that cannot be done by humans. This study presents a dialogue management system for HRI with a comparison sentence similarity method between TF-IDF (Term Frequency-Inverse Document Frequency) Cosine Similarity Algorithm and Jaccard Coefficient and using Finite State Machine (FSM). Dialogue Management is a way to find the response of the answer. When the user says something or in other words, is responsible for managing the flow of the conversation to command the robot. TF-IDF is used to give the weight of the term relationship and comparison between Cosine Similarity and Jaccard Coefficient for comparison method to determine the classification of similarity sentences from the dialogue manager to improve the intent of the dialogue, for the FSM method to set the sequence flow dialogue. We use Google Cloud Speech API as an engine for speech to text using Kinect V2 as an audio sensor. There are eight scenarios created in this system. The speech recognition process using Google Speech for an average of 2.62 seconds shows a reasonably fast response. TF-IDF Cosine Similarity method can produce enough accuracy of 97.43%, and Jaccard Coefficient indicates an accuracy level of 91.57%. The state of the FSM method can be considered as an efficient structure for building dialogue management.

Variational Denoising Autoencoders and Least-Squares Policy Iteration for Statistical Dialogue Managers

The use of Reinforcement Learning (RL) approaches for dialogue policy optimization has been the new trend for dialogue management systems. Several methods have been proposed, which are trained on dialogue data to provide optimal system response. However, most of these approaches exhibit performance degradation in the presence of noise, poor scalability to other domains, as well as performance instabilities. To overcome these problems, we propose a novel approach based on the incremental, sample-efficient Least-Squares Policy Iteration (LSPI) algorithm, which is trained on compact, fixed-size dialogue state encodings, obtained from deep Variational Denoising Autoencoders (VDAE). The proposed scheme exhibits stable and noise-robust performance, which significantly outperforms the current state-of-the-art, even in mismatched noise environments.

Cas-GANs: An Approach of Dialogue Policy Learning based on GAN and RL Techniques

Dialogue management systems are commonly applied in daily life, such as online shopping, hotel booking, and driving booking. Efficient dialogue management policy helps systems to respond to the user in an effective way. Policy learning is a complex task to build a dialogue system. There are different approaches have been proposed in the last decade to build a goal-oriented dialogue agent to train the systems with an efficient policy. The Generative adversarial network (GAN) is used in the dialogue generation, in previous works to build dialogue agents by selecting the optimal policy learning. The efficient dialogue policy learning aims to improve the quality of fluency and diversity for generated dialogues. Reinforcement learning (RL) algorithms are used to optimize the policies because the sequence is discrete. In this study, we have proposed a new technique called Cascade Generative Adversarial Network (Cas-GAN) that is combination of the GAN and RL for dialog generation. The Cas-GAN can model the relations between the dialogues (sentences) by using Graph Convolutional Networks (GCN). The graph nodes are consisting of different high level and low-level nodes representing the vertices and edges of the graph. Then, we use the maximum log-likelihood (MLL) approach to train the parameters and choose the best nodes. The experimental results compared with the HRL, RL agents and we got state-of-the-art results.

CASE: A HAL 9000 for 2021.

An intelligent agent, like HAL 9000, can plan and manage activities for a planetary base and can interact with users through a dialog management system.

Speech to speech interaction system using Multimedia Tools and Partially Observable Markov Decision Process for visually impaired students

In general, visually impaired students need of another person’s to teach them with the help of computers and book. However, a number of students are not aware of using the computers and understanding the concepts by self. In order to solve this issue, a speech to speech interaction system is developed on the basis of a novel dialogue management system. This interaction is developed by combining Multimedia tools and Partially Observable Markov Decision Process (POMDP) with agenda based model used in the proposed dialogue management system to learn the speech signals from user and system will reply accordingly. The proposed system helps visually impaired students to learn easily using a novel dialogue management system. Word Error Rate, Recognition cum retrieval rate and Misrecognition Retrieval Rate are calculated for the proposed POMDP with Agenda Based dialogue management system. The experimental results are compared with Finite-State Based dialogue management system, Frame Based dialogue management system, and Probabilistic dialogue management system. The experimental results proved that the good performance of the proposed POMDP with Agenda Based dialogue management system. The proposed model is trained with 125 speakers out of which 46 were visually impaired and tested with 95 untrained speakers out of which 32 are visually impaired.

Developing a Conversational Virtual Standardized Patient to Enable Students to Practice History-Taking Skills.

Although traditional virtual patient simulations are designed to teach and assess clinical reasoning skills, few employ conversational dialogue with the patients. The virtual standardized patients (VSPs) described herein represent standardized patients that students interview using natural language. Students take histories and develop differential diagnoses of the VSPs as much as they would with standardized or actual patients. The student-VSP interactions are recorded, creating a comprehensive record of questions and the order in which they were asked, which can be analyzed to assess information-gathering skills. Students document the encounter in an electronic medical record created for the VSPs. The VSP was developed by integrating a dialogue management system (ChatScript) with emotionally responsive 3D characters created in a high-fidelity game engine (Unity). The system was tested with medical students at the Ohio State University College of Medicine. Students are able to take a history of a VSP, develop a differential diagnosis, and document the encounter in the electronic medical record. Accuracy of the VSP responses ranged from 79% to 86%, depending on the complexity of the case, type of history obtained, and skill of the student. Students were able to accurately develop an appropriate differential diagnosis on the basis of the information provided by the patient during the encounter. The VSP enables students to practice their history-taking skills before encounters with standardized or actual patients. Future developments will focus on creating an assessment module that will automatically analyze VSP sessions and provide immediate student feedback.

Aplication of techno-marketing programme package of Indonesian orthodox black tea to analyze the Russian Federation market

In order to increase market share as well as competitiveness of Indonesian orthodox black tea in Russian Federation as the biggest Indonesian tea export destination, a research to analyze Russian Federation market applying tea techno-marketing programme package especially to finding suggestions in improving process production has been conducted. This programme consist of Data Base Management System, Knowledge Base Management System, Model Base Management System, and Dialogue Management System. Four data analysis techniques were applied namely (1) market trend, (2) comparative performa index (CPI), (3) constant market share (CMS), and (4) quality function deployment (QFD). The results showed that the Russian Federation market prefer the medium grown tea which is dominated by broken grade (91%). Minimum sensory test score for appearance of made tea, taste, aroma, and colour of liquor as well as infused leaf to be accepted in Russian Federation market were medium. To comply with a request of Russian Federation market, main production process aspects had to be improved, were the quality of tea shoots, had to be 65% minimum, and the rolling programme in factory to result in 90% of broken grades.

Utilizing Human-to-Human Conversation Examples for a Multi Domain Chat-Oriented Dialog System

This paper describes the design and evaluation of a method for developing a chat-oriented dialog system by utilizing real human-to-human conversation examples from movie scripts and Twitter conversations. The aim of the proposed method is to build a conversational agent that can interact with users in as natural a fashion as possible, while reducing the time requirement for database design and collection. A number of the challenging design issues we faced are described, including (1) constructing an appropriate dialog corpora from raw movie scripts and Twitter data, and (2) developing an multi domain chat-oriented dialog management system which can retrieve a proper system response based on the current user query. To build a dialog corpus, we propose a unit of conversation called a tri-turn (a trigram conversation turn), as well as extraction and semantic similarity analysis techniques to help ensure that the content extracted from raw movie/drama script files forms appropriate dialog-pair (query-response) examples. The constructed dialog corpora are then utilized in a data-driven dialog management system. Here, various approaches are investigated including example-based (EBDM) and response generation using phrase-based statistical machine translation (SMT). In particular, we use two EBDM: syntactic-semantic similarity retrieval and TF-IDF based cosine similarity retrieval. Experiments are conducted to compare and contrast EBDM and SMT approaches in building a chat-oriented dialog system, and we investigate a combined method that addresses the advantages and disadvantages of both approaches. System performance was evaluated based on objective metrics (semantic similarity and cosine similarity) and human subjective evaluation from a small user study. Experimental results show that the proposed filtering approach effectively improve the performance. Furthermore, the results also show that by combing both EBDM and SMT approaches, we could overcome the shortcomings of each.

An easy to author dialogue management system for serious games

This article describes an architecture for a dialogue management system to be employed in serious games for natural language interaction with nonplayer characters. The aim of this work is to improve the learning experience by enhancing the immersiveness felt by the player. The system is implemented in a Service Oriented Architecture perspective and it exposes its functionalities through Web services. It also provides an easy to use authoring tool, which allows cultural heritage experts to define the character's knowledge without the need for learning a scripting language. We tested the system by embedding it in a simple serious game, where the user could interact with the virtual representation of a XVI century Geneoese artist, Luca Cambiaso, in order to acquire knowledge about his life and artworks. These early lab test results showed a high level of appreciation for the usability of the system and for the user engagement. Tests also showed that the system is a good didactic tool, since players demonstrated a similar level of knowledge acquisition to that achieved by reading a text.

A dialogue management system using a corpus-based framework and a dynamic dialogue transition model

We propose a novel practical dialogue management system that satisfies the requirements of robust dialogue management, efficient domain knowledge construction, and flexible architecture for mainten...

A dialogue management system using a corpus-based framework and a dynamic dialogue transition model

We propose a novel practical dialogue management system that satisfies the requirements of robust dialogue management, efficient domain knowledge construction, and flexible architecture for maintenance and extensibility. The proposed system uses a corpus-based framework and a dynamic dialogue transition network model, which work in a cooperative and complementary manner. The former supports automatic generation of domain knowledge from an annotated corpus, whereas the latter manages dialogue flows robustly. The system can also automatically carry out user-intention analyses and response generation since it retrieves the most similar utterance and its response pair by estimating similarity between the input utterance and corpus utterances. Therefore, the system can control a new domain dialogue by updating the corpus. In our experiments on two different corpora, the system achieved F0.5-scores of 91% and 90% in terms of user intention recognition with a task completion rate of 95% and 91%.

Erratum: "Assisting E-Government Users with Animated Talking Faces" by P. Foglia, F. Giuntoli, C. A. Prete, and M. Zanda

The references were inadvertently removed from an article entitled "Assisting E-Government Users with Animated Talking Faces" in the January-February 2007 issue of <interactions> . We apologize for this oversight. The information is available below. 1. Cullen, R., O'Connor D., Veritt A. An Evaluation of Local Government Web sites in New Zealand. In The World of EGovernment (ed. Curtin G.C., Sommer M.H., Sommer V.), Haworth, 2003. 2. Foglia P., Prete C.A., Zanda M. Modelling Public Administration Portals. In Encyclopaedia of Portal Technology and Application (ed. Tatnall A.), Idea Group (in press). 3. Haxby J.V., Gobbini M.I., Furey M.L., Ishai A., Schouten, J.L., Pietrini P. Distributed and overlapping representations of faces and objects in ventral temporal cortex. Science, Sept. 28; 293(5539):2405-7, 2001. 4. Nielsen J. Talking-Head video is boring online, AlertBox, Dec. 5, 2005. 5. Qiu L., Benbasat I. An Investigation into the effects of text-to-speech voice and 3D avatars on the perception of presence and flow of live help in electronic commerce. ACM Transactions on Computer Human Interaction (TOCHI), 12(4); 2005. 6. Reis F. E-Government: Internet based interaction with the European businesses and citizens; Statistics in Focus; Eurostat, 2005. 7. Stewart O., Huerta J.M. Transition relevance place: a proposal for adaptive user interface in natural language dialog management systems. ACM SIGCHI Conference Work in progress, 2006.

Bayesian networks based multi-modality fusion for error handling in human–robot dialogues under noisy conditions

In this paper, we introduce probabilistic model based architecture for error handling in human–robot spoken dialogue systems under adverse audio conditions. In this architecture, a Bayesian network framework is used for interpretation of multi-modal signals in the spoken dialogue between a tour-guide robot and visitors in mass exhibition conditions. In particular, we report on experiments interpreting speech and laser scanner signals in the dialogue management system of the autonomous tour-guide robot RoboX, successfully deployed at the Swiss National Exhibition (Expo.02). A correct interpretation of a user’s (visitor’s) goal or intention at each dialogue state is a key issue for successful voice-enabled communication between tour-guide robots and visitors. To infer the visitors’ goal under the uncertainty intrinsic to these two modalities, we introduce Bayesian networks for combining noisy speech recognition with data from a laser scanner, which are independent of acoustic noise. Experiments with real-world data, collected during the operation of RoboX at Expo.02 demonstrate the effectiveness of the approach in adverse environment. The proposed architecture makes it possible to model error-handling processes in spoken dialogue systems, which include complex combination of different multi-modal information sources in cases where such information is available.

“LentInfo” Information—Providing System for the Festival Lent Programme

This paper presents an application, “LentInfo”, which is a system used to provide information about programmes for the Festival Lent in Slovenia. The Festival Lent consists of different open-air theatre and music performances and raws more than 400,000 visitors per year. This application is based on a Hidden Markov Model (HMM) speech recogniser, and the dialogue construction and management is done using the CSDP (Common Spoken Dialogue Platform) dialogue management system. It is represented as a finite-state structure. The dialogue can be specified in a script using simple syntax description. The dialogue manager is multi-application oriented, so it can easily be upgraded for new applications. If some new concepts are needed, only new actions need be added to the existing ones. Currently, prompt messages are prerecorded, but it is also possible to include a speech synthesis system depending on the needs of the application. Error recovery during the dialogue is done with user confirmation of the recognised input speech. The results are presented for tests performed in the year 2001. The results are analyzed according to the phone type (fixed/mobile), signal to noise ratio, dialogue path, etc. Although some calls where carried out using mobile phones from noisy festival places, the performance of the system decreased only slightly under these conditions.