Human-computer Dialogue System Research Articles

Simulation modeling of the system of managing the psychophysiological state of a person based on non-verbal extralinguistic characteristics

Today, when artificial intelligence systems are being created, the time of constant dia-logue between humans and computer, there is a great demand for intelligent control systems and human-machine systems. These systems are very complex and require analysis and re-search. The publication is devoted to the simulation computer modeling of control of the psy-chological and physical state of a person. The basis of the human-computer dialogue system is the speech-to-text conversion sys-tem. The use of this system is the basis for the creation of computer language control systems. Unlike systems that work on the principle of identifying a single user response to a request and a proposal or a question from a password database, it is necessary to expand the lan-guage interface of the computer. It is necessary for the computer to accept commands from the operator and execute them only if the voice and its timbre match those registered in the database. Such a system will allow you to differentiate access to the computer and ensure se-curity for the user, preserve his privacy or save his personal data. After analyzing the speech activity of the operator, it became clear that it was necessary to develop a system of command control and «speech-to-text» conversion in the fused speech stream. This task is solved by comparing the words of the commands and is part of the problem of automatic recognition and understanding of the operator's human language. On the basis of such systems, a computer language control system can be created. The analysis has shown that unlike text-dependent identification systems, systems with a dialog method implement not only a one-time operator response to a query or question from the password database, but also the extension of its full-fledged language interface. The computer receives commands from the operator and executes them only if the voice matches the one registered in the database. This analysis is effective in the context of real-life speech activity of the operator. The literature review showed that for the development of sys-tems of command control and "speech-to-text" transformation in a stream of fused speech, it is necessary to use "mechanisms" of automatic recognition and understanding of the opera-tor's language.

The Latest Progress in Human-Computer Dialogue System

The Human-Computer dialogue system allows a computer to make conversations with humans through natural languages, and some designs can accomplish tasks given by humans. The development of the Human-Computer dialogue system can significantly expand the range of users for computers. The profound ELIZA Program designed scripts containing keywords and their corresponding rule-based sentence transformation to respond to the user. As the computation power increases, Human-Computer dialogue systems are handling more varied and complicated scenarios by introducing deeper and more complex artificial neural networks. The designer of a task-oriented system can choose between the pipeline method and the end-to-end method. The pipeline method is basically a pipeline that lets the user input go through three parts handling Natural Language Understanding, Dialogue Management, and Natural Language Generation, to generate an appropriate response back to the user. The end-to-end method, however, uses joint models to allow the parts to interact with others and get more efficient in handling the information. With the development of ChatGPT, more general language models and more varied methodologies are becoming more prevalent in dialogue systems.

Intent recognition model based on sequential information and sentence features

In human–computer dialogue systems, intent recognition is crucial for determining the intentions or purposes of users during interactions with the system, enabling the system to provide appropriate responses or actions. This paper proposes an intent recognition model that integrates sequential information and sentence structural features. Specifically, the approach utilizes a CNN to capture local salient features in the text, followed by a BILSTM to extract sequential information within the local context. The sequential information is then fed into a multi-head attention mechanism to focus on more relevant sequential details. Additionally, the original data is processed by BERT to extract sentence structural features. Finally, the sequential information features and sentence structural features are concatenated and fused to achieve enhanced intent recognition performance. This approach effectively leverages contextual and semantic information within the text, leading to improved accuracy in intent recognition. Experimental results demonstrate the effectiveness of the proposed method in intent recognition and its high relevance for practical applications.

Joint Intent Detection Model for Task-oriented Human-Computer Dialogue System using Asynchronous Training

How to accurately understand low-resource languages is the core of the task-oriented human-computer dialogue system. Language understanding consists of two sub-tasks, i.e., intent detection and slot filling. Intent detection still faces challenges due to semantic ambiguity and implicit intentions with users’ input. Moreover, separately modeling intent detection and slot filling significantly decrease the correctness and relevance between questions and answers. To address these issues, we propose a joint intent detection method using asynchronous training strategy. The proposed method firstly encodes local text information extracted by CNN and relationship information among words emphasized by attention structure. Later, a joint intent detection model with asynchronous training strategy is proposed by either fusing hidden states of intent detection and slot filling layers, or adopting the key information to fine-tune the whole network, greatly increasing the relevance of intent detection and slot filling subtasks. The accuracy achieved by the proposed method tested on an open-source airline travel dataset and a self-collected electricity service dataset, i.e., ATIS and ECSF, are 97.49% and 89.68%, respectively, which proves the effectiveness of joint learning and asynchronous training.

An Effective Artificial Intelligence-Enabled Error Detection and Accuracy Estimation Technique for English Speech Recognition System

Error detection and accuracy estimation in automated speech recognition (ASR) systems act a vital part in the design of human-computer spoken dialogue systems, as recognition error can hamper accurate systems in understanding the end user intentions. The major aim is to identify the errors in an utterance, and therefore, the dialogue manager can provide proper clarifications to the user. Therefore, the design of accurate error detection and accuracy determination techniques becomes essential in the ASR system. With this motivation, this paper presents a novel artificial intelligence-enabled accuracy estimation and error detection technique for the English speech recognition system (AIEDAE-ESRS). The goal of the AIEDAE-ESRS technique is to perform three actions such as confidence estimation, out-of-vocabulary (OOV) word identification, and error type categorization. In addition, the AIEDAE-ESRS technique performs different levels of preprocessing including sampling of input speech signal, bandpass filtering, and noise removal. Besides, a new deep neural network with hidden Markov model- (DNN-HMM-) based speech recognition technology is designed, which also aims to estimate the accuracy and error. Finally, the hyperparameters of the DNN-HMM model can be optimally chosen by the use of flower pollination algorithm (FPA) and thereby accomplished improved recognition performance. In order to demonstrate the better performance of the AIEDAE-ESRS technique, a series of simulations were conducted and the results are examined under varying aspects. English voice recognition system’s accuracy estimation and error detection were made possible using artificial intelligence (AIEDAE-ESRS). There are three steps in the AIEDAE-ESRS method: confidence estimation; identifying out-of-vocabulary words (OOV); and categorizing mistake types. The simulation results reported the enhanced performance of the AIEDAE-ESRS methodology over current advanced approaches. Our AIEDAE-ESRS methodology outperforms existing methodologies by a factor of ten. The simulation results demonstrated that the AIEDAE-ESRS methodology outperformed previous approaches in terms of efficiency. The improved experimental results indicated that the AIEDAE-ESRS technique produced superior results across a variety of measures.

Research on the Construction of Human-Computer Interaction System Based on a Machine Learning Algorithm

In this paper, we use machine learning algorithms to conduct in-depth research and analysis on the construction of human-computer interaction systems and propose a simple and effective method for extracting salient features based on contextual information. The method can retain the dynamic and static information of gestures intact, which results in a richer and more robust feature representation. Secondly, this paper proposes a dynamic planning algorithm based on feature matching, which uses the consistency and accuracy of feature matching to measure the similarity of two frames and then uses a dynamic planning algorithm to find the optimal matching distance between two gesture sequences. The algorithm ensures the continuity and accuracy of the gesture description and makes full use of the spatiotemporal location information of the features. The features and limitations of common motion target detection methods in motion gesture detection and common machine learning tracking methods in gesture tracking are first analyzed, and then, the kernel correlation filter method is improved by designing a confidence model and introducing a scale filter, and finally, comparison experiments are conducted on a self-built gesture dataset to verify the effectiveness of the improved method. During the training and validation of the model by the corpus, the complementary feature extraction methods are ablated and learned, and the corresponding results obtained are compared with the three baseline methods. But due to this feature, GMMs are not suitable when users want to model the time structure. It has been widely used in classification tasks. By using the kernel function, the support vector machine can transform the original input set into a high-dimensional feature space. After experiments, the speech emotion recognition method proposed in this paper outperforms the baseline methods, proving the effectiveness of complementary feature extraction and the superiority of the deep learning model. The speech is used as the input of the system, and the emotion recognition is performed on the input speech, and the corresponding emotion obtained is successfully applied to the human-computer dialogue system in combination with the online speech recognition method, which proves that the speech emotion recognition applied to the human-computer dialogue system has application research value.

Impact of Using Bidirectional Encoder Representations from Transformers (BERT) Models for Arabic Dialogue Acts Identification

In Human-Computer dialogue systems, the correct identification of the intent underlying a speaker's utterance is crucial to the success of a dialogue. Several researches have studied the Dialogue Act Classification (DAC) task to identify Dialogue Acts (DA) for different languages. Recently, the emergence of Bidirectional Encoder Representations from Transformers (BERT) models, enabled establishing state-of-the-art results for a variety of natural language processing tasks in different languages. Very few researches have been done in the Arabic Dialogue acts identification task. The BERT representation model has not been studied yet in Arabic Dialogue acts detection task. In this paper, we propose a model using BERT language representation to identify Arabic Dialogue Acts. We explore the impact of using different BERT models: AraBERT Original (v0.1, v1), AraBERT Base (v0.2, and v2) and AraBERT Large (v0.2, and v2), which are pretrained on different Arabic corpora (different in size, morphological segmentation, language model window, …). The comparison was performed on two available Arabic datasets. Using AraBERTv0.2-base model for dialogue representations outperformed all other pretrained models. Moreover, we compared the performance of AraBERTv0.2-base model to the state-of-the-art approaches applied on the two datasets. The comparison showed that this representation model outperformed the performance both state-of-the-art models.

An Evaluation of Chinese Human-Computer Dialogue Technology

Abstract There is a growing interest in developing human-computer dialogue systems which is an important branch in the field of artificial intelligence (AI). However, the evaluation of large-scale Chinese human-computer dialogues is still a challenging task. To attract more attention to dialogue evaluation work, we held the fourth Evaluation of Chinese Human-Computer Dialogue Technology (ECDT). It consists of few-shot learning in spoken language understanding (SLU) (Task 1) and knowledge-driven multi-turn dialogue competition (Task 2), the data sets of which are provided by Harbin Institute of Technology and Tsinghua University. In this paper, we will introduce the evaluation tasks and data sets in detail. Meanwhile, we will also analyze the evaluation results and the existing problems in the evaluation.

SPEECH / TEXT NOTE APP USING JAVASCRIPT AND HTML

Speech recognition is a vast research field for researchers in modern era. Earlier, the human language was processed by the computer system for speech recognition. Thus, the main objective is to develop recognition system which improves human to human communication by enabling human-machine communication by processing of text or speech. Various applications of speech recognition systems are present and these all includes various research challenges. Speech to text or text to speech is a part of Natural Language Processing which is a subfield of Artificial Intelligence. In Speech Recognition, spoken words/sentences are translated into text by computer. It is also known as Speech to Text (STT). Speech / Text note app could be very useful in number of applications. Especially in personal assistant bot, dictation, voice command based control system, audio transcriptions, quick notes with audio support, voice based authentication, etc. Web Speech API , It's a very powerful browser interface that allows you to record human speech and convert it into text. We will also use it to do the opposite - reading out strings in a human-like voice.Speech / Text note app can be classified into two main areas, dictation and human-computer dialogue systems.

An Evaluation of Chinese Human-Computer Dialogue Technology

The human-computer dialogue has recently attracted extensive attention from both academia and industry as an important branch in the field of artificial intelligence (AI). However, there are few studies on the evaluation of large-scale Chinese human-computer dialogue systems. In this paper, we introduce the Second Evaluation of Chinese Human-Computer Dialogue Technology, which focuses on the identification of a user's intents and intelligent processing of intent words. The Evaluation consists of user intent classification (Task 1) and online testing of task-oriented dialogues (Task 2), the data sets of which are provided by iFLYTEK Corporation. The evaluation tasks and data sets are introduced in detail, and meanwhile, the evaluation results and the existing problems in the evaluation are discussed.

Computational Models of Miscommunication Phenomena.

Miscommunication phenomena such as repair in dialogue are important indicators of the quality of communication. Automatic detection is therefore a key step toward tools that can characterize communication quality and thus help in applications from call center management to mental health monitoring. However, most existing computational linguistic approaches to these phenomena are unsuitable for general use in this way, and particularly for analyzing human-human dialogue: Although models of other-repair are common in human-computer dialogue systems, they tend to focus on specific phenomena (e.g., repair initiation by systems), missing the range of repair and repair initiation forms used by humans; and while self-repair models for speech recognition and understanding are advanced, they tend to focus on removal of "disfluent" material important for full understanding of the discourse contribution, and/or rely on domain-specific knowledge. We explain the requirements for more satisfactory models, including incrementality of processing and robustness to sparsity. We then describe models for self- and other-repair detection that meet these requirements (for the former, an adaptation of an existing repair model; for the latter, an adaptation of standard techniques) and investigate how they perform on datasets from a range of dialogue genres and domains, with promising results.

Strategies adopted by talkers faced with fluctuating and competing-speech maskers

Studying how interlocutors exchange information efficiently during conversations in less-than-ideal acoustic conditions promises to both further the understanding of links between perception and production and inform the design of human-computer dialogue systems. The current study explored how interlocutors' speech changes in the presence of fluctuating noise. Pairs of talkers were recorded while solving puzzles cooperatively in quiet and with modulated-noise or competing speech maskers whose silent intervals were manipulated to produce either temporally sparse or dense maskers. Talkers responded to masked conditions by both increasing the amount of speech produced and locally changing their speech activity patterns, resulting in a net reduction in the proportion of speech in temporal overlap with the maskers, with larger relative reductions for sparse maskers. An analysis of talker activity in the vicinity of masker onset and offset events showed a significant reduction in onsets following masker onsets, and a similar increase in onsets following masker offsets. These findings demonstrate that talkers are sensitive to masking noise and respond to its fluctuations by adopting a "wait-and-talk" strategy.

The further development of CSIEC project driven by application and evaluation in English education

AbstractIn this paper, we present the comprehensive version of CSIEC (Computer Simulation in Educational Communication), an interactive web‐based human–computer dialogue system with natural language for English instruction, and its tentative application and evaluation in English education. First, we briefly introduce the motivation for this project, survey the related works and illustrate the system structure with flow diagram. Then we describe its pedagogical functions, especially free chatting and chatting on a given topic. We summarise the free Internet usage within 6 months and introduce its integration into English classrooms, as well as the formal evaluation results of the integration. The evaluation findings show that the chatting function has been improved and fully used by the users, and the application of the CSIEC system in English instruction can motivate the learners to use English and enhance their learning process. Lastly, we discuss the application‐driven approach of system development and draw some conclusions for future improvement.

An AI Framework to Teach English as a Foreign Language: CSIEC

Computer Simulation in Educational Communication (CSIEC) is not only an intelligent web‐based human‐computer dialogue system with natural language for English instruction but also a learning assessment system for learners and teachers. Its multiple functions— including grammar‐based gap‐filling exercises, scenario show, free chatting, and chatting on a given topic—can satisfy the various requirements for students with different backgrounds and learning abilities. After a brief explanation of the conception of the dialogue system, as well as a survey of related works, I will illustrate the system structure and describe its pedagogical functions with the underlying AI techniques, such as natural language processing and rule‐based reasoning, in detail. I will summarize the free Internet usage within a six‐month period and its integration into English classes in universities and middle schools. The evaluation findings about the class integration show that the chatting function has been improved and frequently utilized by users, and the application of the CSIEC system on English instruction can motivate learners to practice English and enhance their learning process. Finally, I will conclude with potential improvements.

Principles of electronic speech processing with applications for people with disabilities

In the first part of this paper the principles and the state of the art of speech processing, and especially speech synthesis and recognition, are explained. Then, a speech-based human-computer dialogue system is discussed. The next section gives a brief overview of the available recommendations, guidelines and standards that are directly related with the application of speech technologies. The last part of the paper is dedicated to applications of speech technology for the disabled. The main focus is on blind and partially sighted people and those with hearing loss. Concerning the blind, many multilingual text-to-speech synthesis systems exist, and some polyglot ones, that can convert printed and electronic documents to audio, but further research is needed for structured text, tables and above all graphics to be efficiently transformed into speech. For the deaf persons, there are still big challenges in the development of adequate communication aids. Although a high-speed transformation of speech into text is possible with state-of-the-art speech recognizers (and thus a quasi real-time information transfer from a hearing to a deaf person), the automatic gesture recognition, needed for the reverse transfer, is still in research state. Other applications discussed in this paper include speech-based cursor control for those with physical disabilities, transformation of dysarthric speech into intelligible speech, voice output communication aids for the language impaired and those without speech, and accessibility options for public terminals and Automated Teller Machines through the incorporation of speech technologies. The paper concludes with an outlook and recommendations for research areas that need further study.

Recent Progress in Corpus-Based Spontaneous Speech Recognition

This paper overviews recent progress in the development of corpus-based spontaneous speech recognition technology. Although speech is in almost any situation spontaneous, recognition of spontaneous speech is an area which has only recently emerged in the field of automatic speech recognition. Broadening the application of speech recognition depends crucially on raising recognition performance for spontaneous speech. For this purpose, it is necessary to build large spontaneous speech corpora for constructing acoustic and language models. This paper focuses on various achievements of a Japanese 5-year national project Spontaneous Speech: Corpus and Processing Technology that has recently been completed. Because of various spontaneous-speech specific phenomena, such as filled pauses, repairs, hesitations, repetitions and disfluencies, recognition of spontaneous speech requires various new techniques. These new techniques include flexible acoustic modeling, sentence boundary detection, pronunciation modeling, acoustic as well as language model adaptation, and automatic summarization. Particularly automatic summarization including indexing, a process which extracts important and reliable parts of the automatic transcription, is expected to play an important role in building various speech archives, speech-based information retrieval systems, and human-computer dialogue systems.

Abductive Knowledge Base Updates for Contextual Reasoning

We show in this paper how procedures that update knowledge bases can naturally be adapted to a number of problems related to contextual reasoning. The fact that the update procedures are abductive in nature is favourably exploited to tackle problems related to human-computer dialogue systems. We consider as examples aspects of pronoun resolution,goal formulation , and the problem of restoring the consistency of a knowledge base after some knowledge update is carried out. We state these problems in terms of the update problem and abductive reasoning and show how procedures that update knowledge bases yield some interesting results. We also explain how these procedures can naturally be used to model various forms of hypothetical reasoning such as hypothesizing inconsistencies and performing some “look ahead” form of reasoning. We do not claim thaT the problems presented here are solved entirely within the update framework. However, we believe that the flexibility of the representation and of the problem-solving approach suggest that the problems could be solved by adding more details about each problem. What is most interesting in our understanding is that all the aforementioned problems are expressed and tackled within the same framework.

Modeling the illocutionary aspects of information-seeking dialogues

A dialogue model is presented that incorporates the illocutionary aspects of information-seeking dialogues. Directive, commissive, and assertive types of dialogue acts (e.g., asking, offering, rejecting, answering, and evaluating) are represented by a complex transition network. The model determines all legitimate types and sequences of dialogue acts and regulates the role assignments (e.g., when the information seeker and the information provider temporarily exchange their roles). Finally, an approach to integrating the illocutionary layer with other layers—which deal with thematical and rhetorical coherence—is outlined. Such an integrated model can serve as a basis for the design of human-computer dialogue systems.